I think possibly one of the most practical upgrades in VMware Server 2 appears to be the ability to automatically turn on virtual machines in a stagnated order. I have fond memories of turning on a server with 10 virtual machines, and when they all turn on at once, the hard disk grinds to a halt. This forced me to turn on the machines manually afterwards.

+1 VMware in my books.

I needed another Python fix, and I need one pretty badly. I spent the weekend wondering why it appears to be impossible to edit the GUIDs inside an Exchange mailbox store (read: NOT the GUIDs stored in AD for Exchange). Anyways, I digress.

My goals were simple. I wanted to use Python, wanted something to do with traffic, and wanted to play around with Glade/PyGTK and graphing stuff. My end result was a little app that allows you to specify a GPX file, and it plots the waypoints (and calculates the moving average!). Pretty simple, pretty useless, but pretty fun. I really do like pretty pictures.

I ended up using matplotlab for the graphing part, but I don’t really like how the graphs look. I will likely use Pycha (which dips into Cairo) for my future projects – but we’ll see when that point comes. (If data sensitivity wasn’t an issue, I would totally use Google Charts, since I’m a sucker for APIs).

I showed a few of my co-workers my graph and one replied – oh! that’s really cool. (I think only two of my co-workers are actually interested in my geekyness). He then emailed me tonight a .kmz file containing a colorized file of his speed. I looked at the kml and noticed it appeared to be dynamically allocated judging by the top speed. Well, as you could guess, I surely had to modify my code to include colors.

Within an hour I had a semi-working example, and within two hours will easily be done with this blog post. The code might not be perfect, but it first parses the xml and returns the max speed for the trip. Next, it colorizes the speeds based on a scale of 0-255, with 0 being blue for fast and 255 for being yellow, or slow. I was going to study for the CCNA tonight, but it looks like writing Python is just too much fun.

So what, you might ask, are those dips? Good question. They are huge speed bumps (and the tall blue mound in the middle is a really steep hill).

I recently have been playing with parsing GPX files and spitting out the results into a special KML file. I initially wrote a parser using minidom, yet after running this the first time – and my Core2Duo laptop reaching 100% utilization for 10 seconds – I realized I needed to re-write it using something else.

I spent a little time reading the different parsers for XML and eventually read more about cElementTree. And it is included with Python2.5, sweet.

I quickly rewrote the code and did some tests. First, the two bits of code for parsing my GPX file:

minidom-speed.py

#!/usr/bin/python

from xml.dom import minidom
from genshi.template import TemplateLoader

def collect_info():
dom = minidom.parse('airport.gpx')
for node in dom.getElementsByTagName('trkpt'):
lat = node.getAttribute('lat')
lon = node.getAttribute('lon')
speed = node.getElementsByTagName('speed')[0].firstChild.data
speed = float(speed) * 10
coords = '%s,%s' % (lon, lat)
coords_speed = '%s,%s' % (coords, speed)
yield {
'coordinates': coords_speed
}

loader = TemplateLoader(['.'])
template = loader.load('template-speed.kml')
stream = template.generate(collection=collect_info())

f = open('minidom.kml', 'w')
f.write(stream.render())

cet-speed.py

#!/usr/bin/python

import sys,os
import xml.etree.cElementTree as ET
import string
from genshi.template import TemplateLoader

def collect_info():
mainNS=string.Template("{http://www.topografix.com/GPX/1/0}$tag")

wptTag=mainNS.substitute(tag="trkpt")
nameTag=mainNS.substitute(tag="speed")

et=ET.parse(open("airport.gpx"))
for wpt in et.findall("//"+wptTag):
wptinfo=[]
wptinfo.append(wpt.get("lon"))
wptinfo.append(wpt.get("lat"))
wptinfo.append(str(float(wpt.findtext(nameTag)) * 10))
coords_speed = ",".join(wptinfo)
yield {
'coordinates': coords_speed,
}

loader = TemplateLoader(['.'])
template = loader.load('template-speed.kml')
stream = template.generate(collection=collect_info())

f = open('cet.kml', 'w')
f.write(stream.render())

The speed difference is not just noticeable, but very noticeable.

minidom-speed.py

$ python -m cProfile minidom-speed.py
4405376 function calls (3787047 primitive calls) in 32.142 CPU seconds

cet-speed.py

$ python -m cProfile cet-speed.py
1082061 function calls (904167 primitive calls) in 6.736 CPU seconds

A quarter as many calls and almost 5x faster – at least that’s how I interpret the results. Much better!

Status: ✅

You can likely tell that I’ve been having some fun with graphing and mapping recently. I was reading a few articles about GIS and stumbled upon a pretty darn cool project at Webopticon, which included cool pictures. I showed it to a friend thinking they would find it interesting, and then realized: oh! KML has an altitude attribute. That could be interesting.
One of my projects is to create maps of Sydney’s traffic, so I have been experimenting heavily with Mapnik and OSM. I figured I could have some fun and finally parse some gps tracks and display the data.

I first started off trying to play around with the KML files my gps logger natively stores. After a while I realized it shouldn’t be this hard to parse the XML, and realized it also stores data in gpx format. I opened up one of the gpx files and immediately saw how much easier it would be to work with. I quickly created a parser for the xml in Python (using the dom method, yet I think I’m going to rewrite it using sax), and then with the aid of an article by Sean Gillies, converted the needed objects into KML. I used the speed attribute (with some magnification) as the altitude, and voila, a pretty picture.
This picture is as Victoria Road crosses James Rouse Drive – a spot that is always congested in the morning.
I’ll likely post some code shortly, I would like to rewrite the parsing section to use something event-driven – hopefully it will be a little faster.

Two of my colleagues were having just a little bit too much fun with my blog, so I decided to have some fun back. Over a period of 10 minutes, they managed to leave 10+ comments. Luckily I have full control over my server, and was able to quickly create my practical joke.

$HTTP["remoteip"] == "123.45.678.910" {
url.redirect = (
    "^/(.*)" => "http://www.urbandictionary.com/define.php?term=annoying+person",
    "" => "http://www.urbandictionary.com/define.php?term=annoying+person",
    "/" => "http://www.urbandictionary.com/define.php?term=annoying+person"
)
}

My new guest VMs under Xen seem to be having issues where upon each reboot, the network interface gets incremented by 1. For instance, it starts at eth0, then goes to eth1, then eth2, and eventually ethX. There are two issues to fix: 1) get the count back to 0, and 2) stop it from counting again.

I was able to get them to decrease by looking in the /etc/udev/rules.d/70-persistent-net.rules file and removing all entries.

Next, I was able to prevent this by simply inserting a MAC address to the interface in the configuration. For instance, one of my domU’s has this entry:

vif         = [ 'mac=00:D0:59:83:DC:B5,bridge=xenbr0' ]

Lastly, I made sure (as I would with any server) to create an entry in the /etc/network/interfaces file.

auto eth0
iface eth0 inet static
address 192.168.1.16
gateway 192.168.1.1
netmask 255.255.255.0

Works like a charm.

I have managed to install mapnik 0.4, 0.5, 0.5.1 and various SVN releases in-between on Ubuntu. While this isn’t in itself exciting, I think I manage to stumble at every installation. I typically forget to add the flags when building, so, to prevent myself from stumbling again, I’m going to write them out here.

Build mapnik

$ python scons/scons.py PYTHON=/usr/bin/python \
PGSQL_INCLUDES=/usr/include/postgresql \
PGSQL_LIBS=/usr/lib/postgresql BOOST_INCLUDES=/usr/include/boost BOOST_LIBS=/usr/lib

Then install it

$ sudo python scons/scons.py install PYTHON=/usr/bin/python \

PGSQL_INCLUDES=/usr/include/postgresql \

PGSQL_LIBS=/usr/lib/postgresql BOOST_INCLUDES=/usr/include/boost BOOST_LIBS=/usr/lib

Then proceed as normal.

I have finally decided to do another remodel of this site. I had a few goals before starting:

• Use one image
• Use the YUI-CSS framework
• Use Django
• Make it easily extendable

So far, I think I’ve accomplished these goals. The site is easier to read, easy to modify, and has a few new features. More entries to come!

I returned to my computer today to notice I had the following error:

(145, "Table './databasename/comments_freecomment' is marked as crashed and should be repaired")

Darn. The solution is quite easy, however:

mysqlcheck -uUsername -pPassword databasename comments_freecomment

Now you know what you already know, you can fix it:

mysqlcheck -r -uUsername -pPassword databasename comments_freecomment

If that doesn’t work, you can try a slightly different method. First, go to the location where your databases are stored on the disk (most likely something like /var/lib/mysql/databasename). Next, stop the database – and try to free up as much memory as possible. Then run:

myisamchk -r comments_freecomment

If that doesn’t work, try to force it:

myisamchk -r comments_freecomment -f

Hope that helps!