64 bit Java on slamd64 (64 bit slackware)

April 5th, 2009
slamd64

slamd64 - 64bit Slackware

slamd64 is Fred Emmott’s 64bit Slackware project. I installed it on a couple of servers that provide the bulk of the processing power behind spider.my recently. Just like installing Slackware, installing slamd64 on a server is a matter of downloading the first CD ISO in the set, and following the instructions.

The only real difference I noticed is the startup splash screen. I’m only ever going to see that once, as I never connect monitors to my servers unless they’re broken. I thought the splash screen was a nice touch. I use Slackware because I have conservative tastes in server software, but I think I’d have been disappointed if I’d had to use uname to tell I was using the new distro!

For those who won’t believe a thing until they see the screenshots, I’m not going to disappoint you -here’s the output from uname in an xterm on one of the servers running slamd64. See the ‘x86_64’? That’s what you get for buying extra RAM and installing a 64bit operating system. The SC440 I have still running Slackware reports ‘i686’ for the machine hardware name.

slamd64 uname output

slamd64 uname output

Just in case you think a picture of an xterm doesn’t count as a screenshot, here’s that startup splash screen – nice huh?

slamd64 boot screen

slamd64 boot screen

More screenshots later, keep salivating! The servers I installed slamd64 on are Dell PowerEdge SC440 (yes, they really do work with 8GB of RAM!). They’re not the most powerful servers ever, but they are cheap, quiet and use very little power. I’ve just increased the RAM on two of them to 8GB and 6GB (rather than throw away the old memory), so I had to change my linux kernel to use the RAM beyond the 4GB / 32bit limit. I didn’t consider building a PAE kernel for long – it just seems too ‘clunky’.

It all seemed too easy. I couldn’t wait to get my Java applications started to take advantage of the extra memory. It’s not so much that I want to go ‘bloated’, more that I’m running out of spare capacity to start new instances of my application while I’m developing. I was pleased to see that a slamd64 first-CD only install includes the 64bit JRE, so I could just copy my application onto the upgraded servers and start it.

I actually installed slamd64 before fitting the extra memory, so I was a bit upset to see my Java application being killed by the OOM (Out Of Memory) killer, but pleased to see it reported briefly at 1.9g by ‘top’on a 2GB system! Before I installed slamd64, the most I ever saw top report was 1.7g for a java process, before the inevitable OutOfMemoryError.

After I installed the 8GB, I ran the application again with the 64bit JVM and got a shock! You can see from the top output that my application is using nearly 3 times the memory (3.7g in the picture)  it does when run with the 32bit (1.7g, running concurrently in the top output) JVM! My application is a mixture of unchanging Hashtables and arrays, so perhaps I’ve managed to produce a worst-case 64bit example. I didn’t intend to!

slamd64 java processes in top

slamd64 java processes in top (first row is 32bit JVM, below is 64bit JVM, same application)

The good news is, I can run the 32bit application under slamd64 and enjoy a larger heap. I can start the 32 bit JVM with maximum heap size up to ‘java -Xmx3700M’. A simple array-of-big-strings test program runs to 3.6GB reported by top before it runs out of heap.

I’ve obviously got some work to do before I fully expoit the potential of the 64 bit JVM. The good news is I can keep running the 32 bit JVM under slamd64, and get access to an extra 2GB of RAM that way. The bad news is, it appears as though the 32-bit JVM can only run in the bottom 4GB of RAM (but you knew that, didn’t you?). To use all 8GB with a 32-bit JVM would require either PAE, or running the JVMs inside Virtual Machines such as VirtualBox.

Maybank2u Agent M with Pidgin on Ubuntu

March 30th, 2009
Maybank2u Agent M

Maybank2u Agent M

The Malaysian web can be a frustrating experience if you’re not using any Microsoft products. Maybank2u have an ‘online banking buddy’ to help their customers ask questions and avoid giving them answers. We were recently wondering about changing online merchant payment processors, and considered giving Maybank a go.

We like to try out new stuff on the Internet, so we were quite keen to try our “but how much does it cost?” question. We don’t have any Microsoft products on our home network, for financial, security, performance and utility reasons. The ‘Agent M’ promoted on Maybank’s website at the moment is a Windows Live Messenger chatbot. “Oh cool” we thought, and fired up Pidgin, a free and (previously) very reliable instant messaging client. It didn’t work!

It was a bit of a slow-motion pain signing up for MSN accounts so we could use Pidgin on MSN. The pain was made worse by Streamyx suffering from 50% packet loss today. I’ve got bored with reporting packet loss every few days. It’s so frequent that Streamyx is unusably slow that I just want to weep. Never mind, we’ve got a DiGi EDGE connection as a fall-back, which was very handy today.

Pidgin Instant Messenger

Pidgin Instant Messenger

Pidgin just didn’t seem to connect properly to the MSN network, but neither did it report any failure. Whatever it was, it wasn’t connecting to Agent M, or for that matter any of the other MSN accounts we set up to test it with. Finally, some online searching suggested that connecting Pidgin to the MSN network has been a bit problematic since January. All you need to do is add the ‘msn-pecan’ plugin – a Windows Live Messenger protocol – via synaptic (or sudo apt-get install msn-pecan). You’ll have to delete and add your MSN account, but choose the ‘WLM’ protocol. Then you can connect to Agent M.

While clicking randomly on webpages in frustration at how long it took to connect to Agent M, we discovered that the Maybank e-commerce merchant program can only accept payments from credit cards issued in Malaysia. Since many of our customers are overseas, that means it’s unsuitable for our needs. An afternoon wasted.

The thing that really bites is that it was Microsoft’s fault that I couldn’t connect to Maybank. Microsoft keep their protocols secret to prevent third party makers of software from competing with them. Open source software developers (like the Pidgin project) are constantly having to struggle to work out what new technique Microsoft have come up with to break their applications. In this case, Microsoft have deliberately made it more difficult for me to contact my bank.

I don’t really understand companies like Maybank locking themselves into a single-vendor arrangement as they have with Agent M. There are plenty of other IM protocols in existence – including open source ones, such as XMPP. Maybank customers needn’t even have an IM client – Maybank could provide an in-browser chat client such as JWChat, if they didn’t fancy the Java applet route. Or perhaps being insulated from annoying customers by a chatbot is even more effective if it doesn’t actually work?

Java Damerau Levenshtein update

March 30th, 2009

There’s currently a fairly serious issue with character encoding at spider.my – the knock-on effect of which is far too many entries in the keyword table. The MySQL Damerau Levenshtein UDF I wrote is fast, but the disks in my database server are not. A spelling suggestion was taking several seconds to compute – far too slow for an interactive application.

I wrote a quick Java RMI server to load all the keywords into memory, and used the java code I wrote (but never previously used) to scan through for the best match. It’s still far from ideal, but seems to execute in less than 10% of the time the database takes to do the same thing. This isn’t an example of Java’s performance supremacy over C++, but the advantage of writing a dedicated application to do a single task.

The reason I wrote the Java code was to help understand the original C code the UDFs were based on. When I came to use them to get the spelling suggestions back online, I realised there was a fairly serious bug in them. Because of a transposition of string lengths, an example such as damlevlim(“speling”, “spelling”, 3) was returning 3 instead of 1. That bug is now squashed.

If you want to test the code, it’s now the Java damlevlim(…) method that’s providing the spelling suggestions at spider.my

Here is a class file with the static methods in, there’s no license expressed or implied on these, so take the method you want and embed it in your code, after testing it with the main() method provided.

Java Levenshtein / Damerau-Levenshtein methods

2009 Apr 16 update – see the Damerau Levenshtein page for latest version

DiGi and Streamyx: The tortoise and the hare

March 15th, 2009

I still can’t be bothered phoning this one in, it is after all just bad and not actually completely broken. It is also Sunday, so the weekend-workers at TM would probably rather tackle something really major. I mentioned a couple of nights ago that Streamyx was bad to some international sites, well it still is – here’s a ping to news.bbc.co.uk:

sean@taiguima:~$ ping -n bbcnews.com
PING bbcnews.com (212.58.224.138) 56(84) bytes of data.
64 bytes from 212.58.224.138: icmp_seq=1 ttl=112 time=529 ms
64 bytes from 212.58.224.138: icmp_seq=2 ttl=112 time=514 ms
64 bytes from 212.58.224.138: icmp_seq=3 ttl=112 time=252 ms
64 bytes from 212.58.224.138: icmp_seq=5 ttl=112 time=507 ms
64 bytes from 212.58.224.138: icmp_seq=6 ttl=112 time=255 ms
64 bytes from 212.58.224.138: icmp_seq=7 ttl=112 time=284 ms
64 bytes from 212.58.224.138: icmp_seq=8 ttl=112 time=256 ms
64 bytes from 212.58.224.138: icmp_seq=9 ttl=112 time=267 ms
64 bytes from 212.58.224.138: icmp_seq=10 ttl=112 time=264 ms
64 bytes from 212.58.224.138: icmp_seq=13 ttl=112 time=521 ms
64 bytes from 212.58.224.138: icmp_seq=15 ttl=112 time=523 ms
64 bytes from 212.58.224.138: icmp_seq=17 ttl=112 time=529 ms
64 bytes from 212.58.224.138: icmp_seq=18 ttl=112 time=255 ms
64 bytes from 212.58.224.138: icmp_seq=20 ttl=112 time=529 ms
64 bytes from 212.58.224.138: icmp_seq=21 ttl=112 time=256 ms
64 bytes from 212.58.224.138: icmp_seq=22 ttl=112 time=540 ms
64 bytes from 212.58.224.138: icmp_seq=23 ttl=112 time=259 ms
64 bytes from 212.58.224.138: icmp_seq=24 ttl=112 time=256 ms
64 bytes from 212.58.224.138: icmp_seq=25 ttl=112 time=538 ms
64 bytes from 212.58.224.138: icmp_seq=26 ttl=112 time=249 ms
64 bytes from 212.58.224.138: icmp_seq=27 ttl=112 time=249 ms
64 bytes from 212.58.224.138: icmp_seq=29 ttl=112 time=535 ms
64 bytes from 212.58.224.138: icmp_seq=30 ttl=112 time=245 ms
64 bytes from 212.58.224.138: icmp_seq=31 ttl=112 time=537 ms
64 bytes from 212.58.224.138: icmp_seq=32 ttl=112 time=521 ms
^C
— bbcnews.com ping statistics —
33 packets transmitted, 25 received, 24% packet loss, time 32100ms
rtt min/avg/max/mdev = 245.494/387.271/540.330/135.008 ms

This is actually quite a good one! I forgot the -n switch (so the lines aren’t too long to fit nicely in the blog!) the first time, and that was 50% packet loss. You can see the massive variation between times. Here’s a DiGi ping, taken about the same time:

root@box0045:/home/sean# ping -n bbcnews.com
PING bbcnews.com (212.58.224.138) 56(84) bytes of data.
64 bytes from 212.58.224.138: icmp_seq=1 ttl=114 time=957 ms
64 bytes from 212.58.224.138: icmp_seq=2 ttl=114 time=1456 ms
64 bytes from 212.58.224.138: icmp_seq=3 ttl=114 time=736 ms
64 bytes from 212.58.224.138: icmp_seq=4 ttl=114 time=698 ms
64 bytes from 212.58.224.138: icmp_seq=5 ttl=114 time=736 ms
64 bytes from 212.58.224.138: icmp_seq=6 ttl=114 time=574 ms
64 bytes from 212.58.224.138: icmp_seq=7 ttl=114 time=536 ms
64 bytes from 212.58.224.138: icmp_seq=8 ttl=114 time=690 ms
64 bytes from 212.58.224.138: icmp_seq=9 ttl=114 time=632 ms
64 bytes from 212.58.224.138: icmp_seq=10 ttl=114 time=549 ms
64 bytes from 212.58.224.138: icmp_seq=11 ttl=114 time=671 ms
64 bytes from 212.58.224.138: icmp_seq=12 ttl=114 time=633 ms
64 bytes from 212.58.224.138: icmp_seq=13 ttl=114 time=592 ms
64 bytes from 212.58.224.138: icmp_seq=14 ttl=114 time=672 ms
64 bytes from 212.58.224.138: icmp_seq=15 ttl=114 time=692 ms
64 bytes from 212.58.224.138: icmp_seq=16 ttl=114 time=551 ms
64 bytes from 212.58.224.138: icmp_seq=17 ttl=114 time=692 ms
64 bytes from 212.58.224.138: icmp_seq=18 ttl=114 time=694 ms
64 bytes from 212.58.224.138: icmp_seq=19 ttl=114 time=573 ms
64 bytes from 212.58.224.138: icmp_seq=20 ttl=114 time=714 ms

— bbcnews.com ping statistics —
20 packets transmitted, 20 received, 0% packet loss, time 19005ms
rtt min/avg/max/mdev = 536.086/702.613/1456.054/195.820 ms, pipe 2

See? No packet loss! The times are large and there’s large variation between times because this is a cellular modem. Ping times and packet loss may not be of much interest to you, but if you want to read the news at http://news.bbc.co.uk/ and you have a choice of these two Internet connections, you’ll get a counter-intuitive result.

My Streamyx connection is specified as 512Kbit/s, EDGE is about 200Kbit/s (I think). Using Streamyx, the BBC page ‘half’ appeared after a minute and 5 seconds. That half never completed. I gave up at 4 minutes. Using DiGi, the whole page appeared around 1 minute, with the pictures slowly appearing over the next 2 minutes. The page was fully loaded around 3 minutes.

Those times sound appalling, and they are – I suspect the international links may be unusually slow today. the BBC News front page is also very, very ‘fat’, with lots of images to load – nearly half a Megabyte in total. I cleared my cache in between loads of the BBC News front page, so every little icon had to be loaded each time. If you visit the page a second time, it loads in under 20 seconds on DiGi and fails to load again on Streamyx.

It is just like the old story of the tortoise and the hare. Streamyx is fast – when it works. It occasionally impresses me with its performance. You couldn’t ever say the same for DiGi EDGE – it’s physically a slow way to transfer data, there’s no getting away from that. But it never fails to deliver the goods. Eventually.

– Update –

It did cheer up for a bit, and was quite fast, up until lunchtime 2 days later. And then it was unusable.

Streamyx bad to international sites

March 14th, 2009

Not sure I caught the start of this one, been mostly working on some local projects. I noticed earlier (midnight?) that some USA-hosted blogs were slow or not loading at all. It’s 1:30am now. Have a look at these pings:

root@box0043:/home/sean# ping -n bbcnews.com
PING bbcnews.com (212.58.224.138) 56(84) bytes of data.
64 bytes from 212.58.224.138: icmp_seq=1 ttl=112 time=463 ms
64 bytes from 212.58.224.138: icmp_seq=3 ttl=112 time=465 ms
64 bytes from 212.58.224.138: icmp_seq=4 ttl=112 time=302 ms
64 bytes from 212.58.224.138: icmp_seq=7 ttl=112 time=303 ms
64 bytes from 212.58.224.138: icmp_seq=8 ttl=112 time=304 ms
64 bytes from 212.58.224.138: icmp_seq=10 ttl=112 time=459 ms
64 bytes from 212.58.224.138: icmp_seq=11 ttl=112 time=459 ms
64 bytes from 212.58.224.138: icmp_seq=13 ttl=112 time=305 ms
64 bytes from 212.58.224.138: icmp_seq=14 ttl=112 time=302 ms
64 bytes from 212.58.224.138: icmp_seq=15 ttl=112 time=304 ms
64 bytes from 212.58.224.138: icmp_seq=16 ttl=112 time=461 ms
64 bytes from 212.58.224.138: icmp_seq=17 ttl=112 time=305 ms
64 bytes from 212.58.224.138: icmp_seq=18 ttl=112 time=456 ms
64 bytes from 212.58.224.138: icmp_seq=19 ttl=112 time=455 ms
64 bytes from 212.58.224.138: icmp_seq=20 ttl=112 time=304 ms
64 bytes from 212.58.224.138: icmp_seq=24 ttl=112 time=305 ms
^C
— bbcnews.com ping statistics —
24 packets transmitted, 16 received, 33% packet loss, time 23234ms
rtt min/avg/max/mdev = 302.275/372.512/465.295/77.473 ms

Here’s another:

root@box0043:/home/sean# ping -n slashdot.org
PING slashdot.org (216.34.181.45) 56(84) bytes of data.
64 bytes from 216.34.181.45: icmp_seq=3 ttl=235 time=307 ms
64 bytes from 216.34.181.45: icmp_seq=7 ttl=235 time=436 ms
64 bytes from 216.34.181.45: icmp_seq=9 ttl=235 time=306 ms
64 bytes from 216.34.181.45: icmp_seq=10 ttl=235 time=307 ms
64 bytes from 216.34.181.45: icmp_seq=11 ttl=235 time=436 ms
64 bytes from 216.34.181.45: icmp_seq=13 ttl=235 time=431 ms
64 bytes from 216.34.181.45: icmp_seq=15 ttl=235 time=311 ms
64 bytes from 216.34.181.45: icmp_seq=18 ttl=235 time=307 ms
64 bytes from 216.34.181.45: icmp_seq=20 ttl=235 time=437 ms
64 bytes from 216.34.181.45: icmp_seq=21 ttl=235 time=434 ms
64 bytes from 216.34.181.45: icmp_seq=24 ttl=235 time=308 ms
^C
— slashdot.org ping statistics —
24 packets transmitted, 11 received, 54% packet loss, time 23239ms
rtt min/avg/max/mdev = 306.792/366.162/437.873/63.400 ms

See the huge variation in ping times? And that level of packet dropping means the BBC and slashdot pages don’t load at all or take a very long time (minutes). I’m not sure what this is – maybe TM try to sneak some maintenance work in late at night when they think nobody is working! I can’t be bothered to phone this one in tonight, we’ll see if it’s still this bad in the morning.