ZuZu during her first bath….could a puppy face look any...

“I like an escalator because an escalator can never break, it can only become stairs. There would never be an escalator temporarily out of order sign, only an escalator temporarily stairs. Sorry for the convenience.
— Mitch Hedburg”

- (via Logan Leger)

It’s a bushel ‘o pugs!

Wed, 11 Nov 2009 10:42:00 -0700

It’s a bushel ‘o pugs!

TrackThePack: From PHP to Rails

Wed, 11 Nov 2009 10:35:48 -0700

Recently I pushed an update live for TrackThePack that was probably a little bigger than “just” an update.

The feature set for the web app didn’t change very much (though there were certainly a few much needed things that were added). But the biggest change was that I scrapped every bit of the PHP code and rewrote the entire thing in Ruby/Rails.

El History

I built TrackThePack as an “internal” tool back in 2006 when my wife and I ran a little toy store. We were constantly ordering new toys from suppliers and my OCD self wanted to know where they are. Plus, who doesn’t love tracking packages? The problem was that I was getting really annoyed with trying to juggle half a dozen tracking numbers and going to four different carrier sites to track them.

So, to solve my little problem, I decided to build something. It was running strictly on my dev server and I had no intention of it seeing the light of day. But a few months after I built it, I had mentioned it to a few friends and they prodded me to make the thing public so others could use it.

So I whipped up a little user authentication class for it and pushed it live. Still not thinking anybody would use it. But sure enough, within a few weeks there were a few hundred new users (no idea how…I told maybe 3 people about the thing).

I rarely made updates to the code, just fixed a few bugs here and there, but the site really started picking up steam with Lifehacker did a bit about it. At that point I didn’t have much of a choice…the site started taking on a life of its own.

Egyptian Script Kiddie

Then, in May 2007, some script kiddie in Egypt hacked in to my server and ganked everything…databases and all. Thankfully I was doing nightly backups, but unthankfully I had not encrypted any of the user passwords in the database. (DOH!!!!) Total amateur move, for sure, but it was what it was. I quickly built in some encryption and then emailed all the users telling them the bad news and that they’d need to change their passwords ASAP.

Obviously stuff like that could be a major blow to a web app as you’ve broken the user’s trust, but thankfully people stuck around and helped the site continue to grow.

The Decision to Get Switchy

So, after all of that, I happened to start picking up Ruby/Rails dev and had started developing my stuff exclusively in Rails. I first ported TutorialOutpost, and then PugSpot from PHP to Rails. So TrackThePack was my last “major” app that needed porting.

I say “needed”…it’s not like PHP is some awful language that isn’t capable of doing the job, it just ultimately comes down to preference. Coding in PHP is a chore for me, while Ruby is actually fun. I enjoy doing it.

Anywho, I wasn’t in any big hurry to port TrackThePack. It doesn’t make any money (yet…that’s changing soon) and since we sold the toy store, I wasn’t tracking near as many packages as I was before. But around 5 months ago TrackThePack really started picking up steam again. A few major retailers started using using TrackThePack as a place to send their customers to track their orders and I figured I needed to get down to business and start looking at this thing in the long term.

In the long term, as in a few years, I didn’t want to still be pushing PHP around. So now was as good a time as any to convert the thing to Rails.

Thus the Process Begins

When you’ve got an app like TrackThePack, the core functionality of the app revolves around connecting to third-party APIs. In this case: UPS, FedEx, DHL, and USPS. So the very first thing I had to do was connect to these web services.

Obviously it’s slow as christmas to poll these web services every time a user loads the page, and it gets even hairier when you’ve got half a dozen tracking numbers all doing that at the same time.

Retrieving Yo Data

So what we do is when we retrieve the data from the web services, we parse the XML into individual “events” and store them in the database. Then, we save the last time the feed was updated and won’t refresh the data for 30-120 minutes (depends on various factors). That way we aren’t polling the APIs too much and it speeds up the loading of tracking data exponentially. Not to mention, most of the carriers don’t update their data very often…so no more than once a day. So refreshing the data a dozen times a day doesn’t do anybody any good.

We’re using net/http to request the data and Hpricot to parse the XML.

Scrapin’

Which brings me to the next thing. Smartpost, UPS Mail Innovations, and DHL Global Mail are all services that each carrier uses to partner with USPS to deliver your package. So FedEx will start the shipment, but will hand it over to USPS to do the final delivery. None of the the carriers provide tracking data in their APIs for those, so what we have to do is scrape the data from their actual websites. It’s a major pain because if they do even minor changes to the HTML on the page (specifically the HTML that lists the tracking data) it breaks junk on our end. So…that’s a constant battle but it is what it is.

For the scraping we’re just using open-uri and Hpricot. Hpricot is seriously a lifesaver and makes the whole process amazingly simple.

Auto-Detection

One of the major differences between TrackThePack and some of the other tracking services is that we auto-detect the type of number you entered. The bit of code that determines the type of tracking number is all sorts of nasty (a conglomerate of regex and if/else and case statements). Unfortunately many of the carriers like to use numbers with the same format. So, in those cases what we have to do is poll all the possible services and see who responds with valid tracking data. It slows down the process, but until the carriers decide to use unique identifiers, it is what it is.

Googley Maps

With each tracking number we include a map with icons of the progress. Since many times there are 3-4 status updates for a single location (Arrival, Processing, Departure, etc), we select the unique cities/states/countries from that number’s events and then plot them on a Google Map. We use Geokit Rails to convert city/state/country names into lat/long plots.

Mobile/Email Notification

This feature is still a little buggy, but it’s getting better. Right now, if you enter in your mobile number and an email address that you want to be notified, we’ll send you a notification when your package is both out for delivery and delivered. We just check for the the string “deliver” on status updates and if it’s new, we’ll send you a notification.

The mobile stuff is a text message, but every phone has it’s own email address (5558037475@txt.att.net) so we just collect the user’s number and network if they want to be notified and send the text message as an email.

iPhone Version

I’ve had dozens of people ask for a native iPhone app, but I’m still not convinced that’s really necessary. So for now I’m focusing on really making the iPhone version of the site itself pretty solid.

If you visit TrackThePack on your iPhone, the app (using request.user_agent data) forwards you to specific iPhone controller. I had initially just planned on using CSS to style the current site, but the more I worked on it, the more it made sense to use a completely stripped down version with only the absolute necessary HTML/CSS so it’d load faster.

Plugins/Gems

Overall it’s a really simple app, but here are some of the plugins/gems we’re using to make life easier.

AssetPackager - For compressing/minifying Javascript/CSS on deploy
Exceptional - For error monitoring.
Geokit Rails - For map plotting on the Google Maps in each listing
Message Block - For unifying/simplifying error messages and notices.
PermalinkFu - For creating permalinks on the usernames.
Restful Authentication - For managing user authentication.
site_config - We’re juggling probably a dozen different API keys and settings for all the carriers…so this makes that a lot easier.
WillPaginate - To a paginate the listing of numbers
Whenever - For setting up cron jobs

Server Setup

When the app was built with PHP it was on a server with MediaTemple…and it was awful. I can’t count the number of times our database server was down or the server was just plain slow.

Thankfully we’re now using Slicehost and we’re able to run the whole app on a 512MB slice there (though we’ll probably up that shortly to help things speed up even more).

Commercial: The Future

Above I mentioned that TrackThePack is currently not making a dime, but that actually is something that will be changing shortly.

While I’ve been building out the “Personal” version of the site, I’ve also been building out the “Commercial” version.

Right now, a the large majority of online stores (large and small) send customers a tracking number and tell them to go to the carrier site to track it. So what we’ll be doing is offering online store owners the ability to embed tracking data and maps into their own sites so they can keep their customers at their site instead of sending them elsewhere. I won’t go into to many more details, but this service will be ready in the next week or two.

The Personal version will always remain free…no plans on changing that. So, please continue enjoying that. :)

Shpigford

ZuZu during her first bath….could a puppy face look any...

The myth of the page fold: evidence from user testing

Eat Me: The Food and Philosophy of Kenny Shopsin

Ummmm WHATTT?!?!?! Digital Tattoo Interface

My favorite is the scenery in the background :) SNS: Balancing...

pie0: World’s Tallest Dog and World’s Smallest Dog meet. Old...

mrgan: A refreshingly wholesome (and entertaining) look at...

"I like an escalator because an escalator can never break, it can only become stairs. There would..."

It’s a bushel ‘o pugs!

TrackThePack: From PHP to Rails

El History

Egyptian Script Kiddie

The Decision to Get Switchy

Thus the Process Begins

Scrapin’

Auto-Detection

Googley Maps

Mobile/Email Notification

iPhone Version

Plugins/Gems

Server Setup

Commercial: The Future