Sean's blog

Pos Malaysia Shipping Widget for your website

(Update 29th Dec 2010: The widget died while I was moving code around. It’s back again for a while until I put some pages together at spider.my explaining how the new API is going to solve all your shipping woes)

(Update 11th Jan 2011: Widget has been off and on for some time while I’ve been beating the API into shape. It should continue to work in its present form for the foreseeable future. Check out the spider.my API to see what the widget uses for shipping data.)

I’ve never written a widget before! This one hasn’t turned out bad – try changing the parcel weight and destination. See? It works! And you can have it*1 on your website too! All you need to do is to add this little bit of HTML:

<div id="mySpiderShippingWidget"><script type="text/javascript" src="http://spider.my/static/js/shippingwidget.js"></script></div>

…and next time you look at your webpage, it’ll have Pos Malaysia shipping rates on it, courtesy of spider.my and a very dodgy browser-security workaround to allow cross-domain … actually it isn’t really AJAX, it’s more like Asynchronous JavaScript And More JavaScript, but you’ll have to wait for the HowTo post to see how I do it (or just view my page source).

At the start of the week I said I’d demonstrate a how a Pos Malaysia API (Application Programmer’s Interface) should work. As a recap, it was to have two major parts: a simple, reliable, fast technique for obtaining a single quote in a format that can be readily used by any application, and a technique for providing a complete set of rating tables for applications that need higher performance or reliability, but also want easy access to updated rates.

It has actually been a very solid week’s work, but it’s now all done. On Monday, I demonstrated what the problem is. Pos Malaysia’s online shipping quotation page is OK*2 for humans to look at, but is too ‘fat’ and arbitrarily formatted for efficient and reliable use in e-commerce. I hinted at an AJAX replacement by caching quotes on my ‘old-technique’ page.

On Tuesday I started ‘ripping’ Pos Malaysia’s shipping rates. The whole problem is the difficulty of access to their rates, so having my own set would make the rest of the demo much easier. To avoid upsetting the Pos Malaysia sysadmin, I wrote a set of persistent classes for the rating data on spider.my, so that I would only have to make the hundreds of requests needed to rip the data just once.

On Wednesday I tidied up the server-side access to the ripped rates and provided a reliable AJAX interface which accepts a weight and a destination and returns quotes from all qualifying methods. I currently only have Pos Laju Document and Parcel rates, so if you ask for a weight over 1kg, you get only 1 quote, 2 quotes for anything lighter (now more quotes since Pos Parcel rates added). At this stage, part 1 of the job is finished: there’s now a simple technique for Pos Malaysia’s customers and partners to instantly acquire a single quote.

The method for complete rating data updates was quite straightforward to provide once the server-side persistence was finished. On Thursday I provided a rates download facility which can give full rating data for all countries and all methods, all methods for one country, or all countries for a single method. I have an e-commerce project ‘on a back burner’ at the moment which will use stored shipping rates to simplify and speed up its checkout. With Pos Malaysia’s current online facility that’s practically impossible. If Pos provided this kind of API it would be straightforward.

As a demonstration piece, I started writing a ‘widget’ that can be used on anybody’s blog or website on Thursday night. I’ve never written much in the way of javascript before this week, so I’ve been spending a lot of time at w3schools.com! I thought I’d finished the widget last night, but discovered the “Same Origin Policy” which prevents javascript on a page from one website loading data from another website. There seem to be a lot of very complicated solutions for this problem, but this afternoon I stumbled on a simple one.

*1 – Yes, you can put it on your blog or website. It’s not very pretty, but if you’d like to try it out I would be very pleased that you did. Let me know what you think of it functionally. One last thing – this is very experimental, so it might stop working from time to time (might, but might also keep going forever), and watch out for the rates: they don’t include Pos’ silly random surcharges.

*2 – OK… but it needs some basic maintenance! There are very many quotes which claim to allow shipments up to 999kg. It looks like Pos has never got round to setting the maximum values for all their countries. Also the country names – some of them are oddly styled like ‘Ukraine (Kiev)’ and ‘Serbia & Montenegro’. The latter hasn’t existed since 2006, according to Wikipedia! The technique Pos uses to request a quote from their server doesn’t actually work with names like these (try getting a quote for shipping to ‘Serbia & Montenegro’ from Pos’ website). The names form the URL for the quote, but contain invalid characters for URLs, so some countries return errors instead of quotes!