GUI to Plot Driving Speed

May 27, 2008

I needed another Python fix, and I need one pretty badly. I spent the weekend wondering why it appears to be impossible to edit the GUIDs inside an Exchange mailbox store (read: NOT the GUIDs stored in AD for Exchange). Anyways, I digress.

My goals were simple. I wanted to use Python, wanted something to do with traffic, and wanted to play around with Glade/PyGTK and graphing stuff. My end result was a little app that allows you to specify a GPX file, and it plots the waypoints (and calculates the moving average!). Pretty simple, pretty useless, but pretty fun. I really do like pretty pictures.

I ended up using matplotlab for the graphing part, but I don't really like how the graphs look. I will likely use Pycha (which dips into Cairo) for my future projects -- but we'll see when that point comes. (If data sensitivity wasn't an inssue, I would totally use Google Charts, since I'm a sucker for APIs).

Another Baby Step

April 29, 2008

I showed a few of my co-workers my graph and one replied -- oh! that's really cool. (I think only two of my co-workers are actually interested in my geekyness). He then emailed me tonight a .kmz file containing a colorized file of his speed. I looked at the kml and noticed it appeared to be dynamically allocated judging by the top speed. Well, as you could guess, I surely had to modify my code to include colors.

Within an hour I had a semi-working example, and within two hours will easily be done with this blog post. The code might not be perfect, but it first parses the xml and returns the max speed for the trip. Next, it colorizes the speeds based on a scale of 0-255, with 0 being blue for fast and 255 for being yellow, or slow. I was going to study for the CCNA tonight, but it looks like writing Python is just too much fun.

So what, you might ask, are those dips? Good question. They are huge speed bumps (and the tall blue mound in the middle is a really steep hill).

Event vs. DOM Driven Parsing of XML

April 29, 2008

I recently have been playing with parsing GPX files and spitting out the results into a special KML file. I initially wrote a parser using minidom, yet after running this the first time -- and my Core2Duo laptop reaching 100% utilization for 10 seconds -- I realized I needed to re-write it using something else.

I spent a little time reading the different parsers for XML and eventually read more about cElementTree. And it is included with Python2.5, sweet.

I quickly rewrote the code and did some tests. First, the two bits of code for parsing my GPX file:

minidom-speed.py

#!/usr/bin/python

from xml.dom import minidom
from genshi.template import TemplateLoader

def collect_info():
    dom = minidom.parse('airport.gpx')
    for node in dom.getElementsByTagName('trkpt'):
        lat = node.getAttribute('lat')
        lon = node.getAttribute('lon')
        speed = node.getElementsByTagName('speed')[0].firstChild.data
        speed = float(speed) * 10
        coords = '%s,%s' % (lon, lat)
        coords_speed = '%s,%s' % (coords, speed)
        yield {
            'coordinates': coords_speed
            }

loader = TemplateLoader(['.'])
template = loader.load('template-speed.kml')
stream = template.generate(collection=collect_info())

f = open('minidom.kml', 'w')
f.write(stream.render())

cet-speed.py

#!/usr/bin/python

import sys,os
import xml.etree.cElementTree as ET
import string
from genshi.template import TemplateLoader

def collect_info():
    mainNS=string.Template("{http://www.topografix.com/GPX/1/0}$tag")

    wptTag=mainNS.substitute(tag="trkpt")
    nameTag=mainNS.substitute(tag="speed")

    et=ET.parse(open("airport.gpx"))
    for wpt in et.findall("//"+wptTag):
        wptinfo=[]
        wptinfo.append(wpt.get("lon"))
        wptinfo.append(wpt.get("lat"))
        wptinfo.append(str(float(wpt.findtext(nameTag)) * 10))
        coords_speed = ",".join(wptinfo)
        yield {
            'coordinates': coords_speed,
            }
        
loader = TemplateLoader(['.'])
template = loader.load('template-speed.kml')
stream = template.generate(collection=collect_info())

f = open('cet.kml', 'w')
f.write(stream.render())

The speed difference is not just noticeable, but very noticeable.

minidom-speed.py

$ python -m cProfile minidom-speed.py
4405376 function calls (3787047 primitive calls) in 32.142 CPU seconds

cet-speed.py

$ python -m cProfile cet-speed.py
1082061 function calls (904167 primitive calls) in 6.736 CPU seconds

A quarter as many calls and almost 5x faster -- at least that's how I interpret the results. Much better!

Baby Steps at Graphing Traffic

April 27, 2008

You can likely tell that I've been having some fun with graphing and mapping recently. I was reading a few articles about GIS and stumbled upon a pretty darn cool project at Webopticon, which included cool pictures. I showed it to my girlfriend thinking she would find it interesting, and then realized: oh! KML has an altitude attribute. That could be interesting.

One of my projects is to create maps of Sydney's traffic, so I have been experimenting heavily with Mapnik and OSM. I figured I could have some fun and finally parse some gps tracks and display the data.

Details!

I first started off trying to play around with the KML files my gps logger natively stores. After a while I realized it shouldn't be this hard to parse the XML, and realized it also stores data in gpx format. I opened up one of the gpx files and immediately saw how much easier it would be to work with. I quickly created a parser for the xml in Python (using the dom method, yet I think I'm going to rewrite it using sax), and then with the aid of an article by Sean Gillies, converted the needed objects into KML. I used the speed attribute (with some magnification) as the altitude, and voila, a pretty picture.

This picture is as Victoria Road crosses James Rouse Drive -- a spot that is always congested in the morning.

I'll likely post some code shortly, I would like to rewrite the parsing section to use something event-driven -- hopefully it will be a little faster.

Redirecting Fun with Lightty

April 23, 2008

Two of my colleagues were having just a little bit too much fun with my blog, so I decided to have some fun back. Over a period of 10 minutes, they managed to leave 10+ comments. Luckily I have full control over my server, and was able to quickly create my practical joke.

$HTTP["remoteip"] == "123.45.678.910" {
    url.redirect = (
        "^/(.*)" => "http://www.urbandictionary.com/define.php?term=annoying+fuck",
        "" => "http://www.urbandictionary.com/define.php?term=annoying+fuck",
        "/" => "http://www.urbandictionary.com/define.php?term=annoying+fuck"
    )
}