Sean's blog

Pos Malaysia shipping quotes for e-commerce

Pos Malaysia

I maintain some Malaysian shipping modules for the popular e-commerce software osCommerce which allow local e-shops to include up-to-date automatic shipping quotes from Pos Malaysia (Pos Laju and Pos Air Parcel) in the order total. If you run an e-shop, being able to pass on accurate shipping costs to your customers – compared with ‘free’ or fixed-price shipping – means you can give your customers better prices than your competitors. If you can offer your customer a choice of shipping options, this also allows them to save money or expedite their delivery. Doing all of this automatically means you don’t have to attend to your online business – it completes orders and collects money for you, and tells you when to pack deliveries!

Pos Malaysia have a shipping quotation service at their website. It’s obviously intended for human visitors, as it also has advertising, branding and links to other services on it. I use this web page in my osCommerce module. In order to give an automatic, up-to-date quote, an osCommerce shop using my modules must send a request to Pos Malaysia’s website, read the entire page, and use pattern matching to extract the shipping price.

The current technique is very wasteful of resources, the page that must be searched for the shipping price is 13 to 15KB – for just a few characters! The technique is also horrendously sensitive. If Pos Malaysia change the appearance of the page, the pattern-matching technique can’t work and the shipping module fails to return a quote, which will usually mean that the osCommerce e-shop won’t be able to complete the order. The Internet being the ‘instant’ experience that it is, the sales opportunity is probably lost.

One further annoyance is that the shipping module requesting one or two (for different shipping methods) quotes from Pos Malaysia’s website introduces a considerable delay in the user’s experience of the e-shop. When a customer goes to the checkout page on which the shipping cost appears, the server starts to build the page, sends a request to Pos Malaysia’s website, waits for Pos Malaysia’s webserver to build its response, reads the (large) response, performs pattern matching on it (repeating the last 4 steps for each additional shipping method) and finally sends the page to your customer’s browser. Several-second delays are not uncommon.

Pos Malaysia quotation page timings

Take a look at these two resource timing graphs from the Chromium browser. If you haven’t used Chromium yet, and you’re a developer, there are some great tools in there – and it’s shaping up to be a very competent browser too. The first image is from Pos Malaysia’s website, a shipping quote for a parcel (surface and air quotes on the same page). The blue bar is for the page itself – the other bars are for resources the page needs to be rendered. We only need the page, as we’re not interested in images, stylesheets and scripts, only in the text of the shipping amounts. There’s a redirect (a 302 Found) which further slows things down, and then a 104ms wait (perhaps the page is being built by Pos’s server?) and then nearly a quarter of a second while the ~14KB of the page text is transferred.

AJAX load time from spider.my's currency conversion page

Compare that with this image, which shows the resources being loaded from spider.my’s currency conversion page. The line to note is that for the AJAX update of the exchange rate. The latency is horrible, as spider.my is hosted in the USA, but the time to load the AJAX update is only 1ms – hundreds of times faster than Pos’ shipping quote page. That’s understandable because the reply from spider.my is only about 100bytes long!

What I think Pos Malaysia should do is to provide an API to Malaysian e-business so that their postal services can be more efficiently and reliably built into local business systems. I think they should offer 2 APIs – one a simple AJAX responder for use on blog pages and low-volume shops, and the other a shipping rates updater for larger-volume businesses. The simple responder would take the place of the current osCommerce modules and should be able to easily serve thousands of shipping quote requests per second on any old PC that Pos has lying around. I envisage some heavy client-side caching for the high-volume API, so apart from days on which Pos update their rates, the server should see very little traffic at all for a more complex solution suitable for large businesses.

It was obvious to me when I first wrote the shipping modules a few years ago that they were inefficient and fragile, so I emailed and wrote to Pos Malaysia, even met their regional management to try to get someone to improve their end of the shipping quote system. A few years have passed and nothing has happened, so I’m going to write it myself and try to get someone from Pos to have a look at it and hopefully host it themselves. It’s not difficult and it’ll make a nice test of Spinneret. After all, when I finally get round to making Spinneret available, it will be FOSS buatan Malaysia!

If Pos wants the system, I’d be quite happy to do all the necessary work to integrate it into Pos Malaysia’s current IT systems FOR FREE. I don’t want to maintain the osCommerce modules in their current condition any more, and I think Pos exposing a decent API to local e-businesses would make a big difference to the ability of Malaysian businesses to compete online. That would make my life easier, and hopefully earn me a few free cups of tea at the kopitiam.