Sean's blog

URL Shortening – use a service or do it yourself?

Jess saw the news about tr.im today as she was using Twitter and asked me which URL-shortening service she should use. URL shortening services allow you to include those monster URLs that some sites seem to be capable of producing in the short-format tweets that twitter is based on. If you’ve ever tried to send someone a link to a Google search, you may have seen this. A search for “url shortening” on Google produces this whopping 158-character strin… no this is not a string, it’s a hawser or a python, or something:

http://www.google.com/search?hl=en&safe=off&client=firefox-a&tbo=p&rls=com.ubuntu%3Aen-US%3Aofficial&tbs=nws%3A1&q=url+shortening&aq=f&aqi=&aql=&oq=&gs_rfai=

I sometimes wonder why sites who use URLs that large bother serving pages – I’m sure we’re only months away from a browser that can render the contents of the Location: header field! If – like me – you see an URL like that and edit out all the redundant cruft in there, you’ll be sending your friend a nice, short 46-character URL that does the same thing:

http://www.google.com/search?q=url+shortening

But if you were to include this in a tweet (or you’re not like me, and don’t look at gargantuan URLs and think “what a waste of characters”) you might want to shorten it even further. From 46 characters, a hypothetical URL-shortening service might give you something like

http://sh.rt/abc123

… for just 19 characters. Which one should you choose? Some might say “shorter the better”, but what it really comes down to is finance. URL shortening is an invisible service, often provided for free by individuals or groups employing the Slashdot Business Plan:

1. Good idea
2. …?
3. Profit!

For short messaging services, IM and possibly mobile browsing, URL shortening is a “Good idea”. The problem is that if it’s done right, nobody knows who is doing it (beyond the domain name in the URL they briefly see) – there’s no opportunity to build a brand, add value, nothing! The upshot is that you can tell if someone is doing URL shortening ‘right’, because they won’t be able to win any revenue, and they’ll have to close shop!

An URL-shortening service closing shop is a minor disaster. The web is built on links from one point to another. When the URL-shortener dies, there’s no connection between the short URL that’s left on one site and the long URL you would once have been redirected to. That’s quite a risk to take, so you’d want to make your choice of URL-shortening service carefully, and possibly even consider paying for it.

URL-shortening is not rocket science. To make your own URL-shortening service, all you have to do (besides make it work 100% of the time, forever) is to provide a short URL on your domain which when requested will redirect the User-Agent to the original long URL. That’s it.

I persuaded Jess to let me write one for her using Spinneret. Her shop is lolyco.com – lolyco at 6 characters is shorter than tinyurl, so there’s your first check to see if you should roll your own short URLs: do you have a short domain name? The problem is how to get the service in. The shop is a PHP project served by Apache. Spinneret is a separate Java application server, so the method I chose is to use mod_proxy and the ProxyPass directive to hang my URL-shortening service on the “/u/” (‘u’ is short for URL) path – any requests to http://lolyco.com/u/..anything… are forwarded by mod_proxy to the URL shortening service which runs as a web service on a different port on the same server. Just for the sake of sharing, the Lolyco short URLs can be listed at:

http://lolyco.com/u/short-urls.html

And just for example – the first short URL is for this blog article:

http://lolyco.com/u/0

Not bad huh? That’s only 21 characters – the first 60 or so URLs will be that length, the next few thousand will be 22 characters long. There’s yet another bonus for Jess in doing it this way. Whenever she uses a lolyco.com short URL in a tweet to refer to a page at lolyco.com, a visitor’s browser will only have to resolve (look up) one domain name: lolyco.com, so that should make the DIY version seem even quicker to a visitor than a third-party URL shortener.

Just in case you’re a trainspotter, I’ve done no optimisation on the URL shortener (there are some obvious ones) and Apache Bench running on the same small VPS the short URL service is tells me that it reliably serves nearly 3,000 redirects per second (100 concurrent) at around 15Mbit/s average transfer rate with mean total time per request of 35 ms (0.35ms per concurrent request). In the event that lolyco.com ever receives more than 1,000 requests per second, I may consider optimising the service.