Sydney Commute Times Mapped Part 1

EDIT 12-03-2025: I accidentially broke the maps when deleting my AWS account, as the mbtiles were hosted there. Oops.

I quite like open data. I like data based on open standards (or mostly open standards) even better. Many transport operators around the world have started releasing their timetable data using (mostly) open standards, e.g. GTFS. One of the nice things about using a standard is that clever people have created tools to work with the timetable data, and those tools can now be used to manipulate timetable data from hundreds of agencies. The magnificent OpenTripPlanner is one such tool, and it works well with 131500’s GTFS data.

New South Wales Planning & Infrastructure have released a draft plan for how they hope to shape Sydney’s growth, which is where they detail the idea of a “city of cities”. I thought it would be interesting to mash these smaller “cities” with 131500’s transport data, and then display a map with the shortest commute to the nearest city. Various cities, I believe including Melbourne, have goals of re-achieving a “20-minute” city, or something similar (i.e. X% of the population can reach X% of the city within X minutes).

This map is the first stage. It only displays the commute time to St Leonards from every Mesh Block in the greater Sydney area. I used the open source tool OpenTripPlanner to computer the commute times, with OpenStreetMaps to support walking distances. The next map I release will probably have all the regional cities, and a similar styled map depicting time to nearest “centre”.

Mapping Mesh Blocks with TileMill

This quick tutorial will detail how to prepair the ABS Mesh Blocks to be used with MapBox’s TileMill. Beyond scope is how to install postgresql, postgis and TileMill. There is a lot of documentation how to do these tasks.

First, we create a database to import the shapefile and population data into:

Using ‘psql’ or ‘SQL Query’, create a new database:

CREATE DATABASE transport WITH TEMPLATE postgis20 OWNER postgres;
# Query returned successfully with no result in 5527 ms.

It is necessary to first import the Mesh Block spatial file using something like PostGIS Loader.

We then create a table to import the Mesh Block population data:

CREATE TABLE tmp_x (id character varying(11), Dwellings numeric, Persons_Usually_Resident numeric);

And then load the data:

COPY tmp_x FROM '/home/kelvinn/censuscounts_mb_2011_aust_good.csv' DELIMITERS ',' CSV HEADER;

It is possible to import the GIS information and view it in QGIS:

Now that we know the shapefile was imported correctly we can merge the population with spatial data. The following query is used to merge the datasets:

UPDATE mb_2011_nsw
SET    dwellings = tmp_x.dwellings FROM tmp_x
WHERE  mb_2011_nsw.mb_code11 = tmp_x.id;

UPDATE mb_2011_nsw
SET    pop = tmp_x.persons_usually_resident FROM tmp_x
WHERE  mb_2011_nsw.mb_code11 = tmp_x.id;

We can do a rough validation by using this query:

SELECT sum(pop) FROM mb_2011_nsw;

And we get 6916971, which is about right (ABS has the 2011 official NSW population of 7.21 million).

Finally, using TileMill, we can connect to the PostgGIS database and apply some themes to the map.

host=127.0.0.1 user=MyUsername password=MyPassword dbname=transport
(SELECT * from mb_2011_nsw JOIN westmead_health on mb_2011_nsw.mb_code11 = westmead_health.label) as mb

After generating the MBTiles file I pushed it to my little $15/year VPS and used TileStache to serve the tiles and UTFGrids. The TileStache configuration I am using looks something like this:

{
  "cache": {
    "class": "TileStache.Goodies.Caches.LimitedDisk.Cache",
    "kwargs": {
        "path": "/tmp/limited-cache",
        "limit": 16777216
    }
  },
  "layers": 
  {
    "NSWUrbanDensity":
    {
        "provider": {
            "name": "mbtiles",
            "tileset": "/home/user/mbtiles/NSWUrbanDensity.mbtiles"
        }
    },
    "NSWPopDensity":
    {
        "provider": {
            "name": "mbtiles",
            "tileset": "/home/user/mbtiles/NSWPopDensity.mbtiles"
        }
    }
  }
}

Mapping Urban Density in Sydney

EDIT 12-03-2025: I broke the maps when I deleted my AWS account, which I forgot as hosting the mbtiles.

Five years ago I started exploring different mapping technologies by detailing instructions on installing Mapnik and mod_tile. Times have changed significantly in the last five years, and thanks a lot to the products offered by MapBox. After playing with TileMill, MBTiles, Leaflet and UTFGrids, it is great how many annoyances have been fixed by MapBox. I find it enjoyable making maps now, as I no longer need to worry about patching code just to get it to run, or mucking about with oddities in web browser.

Each night this week I have created a new map using Mesh Block spatial data from the Australian Bureau of Statistics (Mesh Blocks are the smallest area used when conducting surveys). I am thankful to live in a country that provides a certain amount of open data, and the ABS should be applauded for the amount of data they provide. They provide spatial data about Mesh Blocks, as well as population counts for this spatial data. It is relatively easy to merge the two and then visualise them using TileMill.

First up - population density of Sydney, i.e. persons reported to be living in each mesh block. Darker red indicates a higher population count.

I find it interesting to see how many people live in certain Mesh Blocks. You will notice that Mesh Blocks with high population levels tend to be nearer public transport - either major roads with frequent bus service, or train stations.

We can look at the urban densities by determining dwellings per hectare, and do this per Mesh Block. The definition I used for urban densities comes from Ann Forsyth in “Measuring Density: Working Definitions for Residential Density and Building Intensity” (pdf). Ann discusses the need to consider net or gross densities, depending on the type of land use. At the Mesh Block level the land use type appears to be singular: Industrial, Parkland, Commercial, Residential, and Transport. Because the land use type was generally singular I have not adjusted to gross/net, but still used Ann’s definitions of certain density bands:

  • Very low density: 11 dw/ha
  • Low density: 11-22 dw/ha
  • Medium density: 23-45 dw/ha
  • High density: 45 dw/ha

“dw/ha” is dwellings per hectare. I decided to map the four density levels, which can be relatively easily achieved using TileMill. See below for an example.

You can zoom in and scroll over any Mesh Block in Sydney to find out more. Additional installation information on how I did this can be found on this special page: Mapping Mesh Block Data.

Hiking the W Circuit (Torres del Paine)

I have just returned from hiking the “W”, a famous circuit through the Torres del Paine, in Patagonia. Although I did some research before doing the trek, it turns out I made a number of assumptions that turned out to be incorrect. I will detail in this entry what I learned, a few things that worked well, and a few things that did not work so well.

I will try to avoid posting spoilers of the major sites, and instead focus on logistics of doing the trek. First things first: much of this content is from what I learned at a talk given at the Erratic Rock hostel, at 3PM the day before entering the park. The single biggest suggestion I would give you is to go to this talk.

Next, I should clear the two biggest assumptions I had before arriving, so you can plan accordingly.

  1. “Pirate” camping is frowned upon - probably even illegal in the park. You must stay at a designated camp-site.
  2. You can only cook at designated areas, which means you need to plan lunch meals that do not need to be cooked.
  3. Water from streams is apparently safe to drink. Nobody uses filters. Just fill up at a place with flowing water, which comes straight from glaciers, and not where there is a horse crossing.
  4. There are paid refugios and free refugios.
  5. The paid refugios (shelters) have showers.
  6. The transportation to and from the park is timed perfectly for all the trekkers, but it is crucial you plan for what bus you want to take.
  7. It gets really cold at night, at least it did in March. If you bring your own sleeping bag, then make sure it is rated at least down to -5C, maybe -10C, otherwise you won’t be getting much sleep.

How do you get to the Torres del Paine?

This naturally depends where you are coming from. I were in Santiago, so flew to Punta Arenas and took a bus to Puerto Nateles. One thing that was a little unclear was if the bus would stop at the airport in Punta Arenas or not. I decided not to risk it, so caught a mini-bus from the airport to the bus terminal (3000CLP), and caught a bus to Puerto Nateles almost immediately. It turns out the bus did stop at the airport and picked on person up - there were no other seats left. I would suggest you email or call the bus company (Bus Ferdnandez) and make sure they pick you up.

One of the first decisions you will need to make is if you should camp, or just stay in a refugio? It was obvious the people who were staying in refugios, as their packs were usually quite small, and they smelled really clean. It was obvious the people doing the full trek, as they usually looked tired, and not terribly clean. I were in the later group. Base Camp, right next to Erratic Rock, where the 3PM talk is held, offers gear rental. The prices are reasonable. Equipment needs to be reserved before you arrive during high season.

What are these refugios? Do they need to be booked in advance? The refugios are little shelters at different camp-sites strewn throughout the park. I camped, but I believe there are nice shelters (more similar to cabins), and some that are more like dormitories. Take note that you must camp at one of these designated campsites, and they aren’t all free - more details below in the day-to-day breakdown. The two companies running the refugios are: Fantasticosur and Vertice. I think you should book in advance, but maybe research this. 

What route to take? The route you take will depend strictly on how far you want to trek each day, or are capable, as you must stay at a camp-site. At the Erratic Rock talk they will give you a suggested route, from west to east, which is one I and a group of others followed. A map of the hiking area will be provided at check in.

The “Erratic Rock Route” goes like this:

Day 1

Catch 7:30 bus into Torres del Paine - your hostel/hotel can surely organise. Our bus cost 15000CLP return. You will enter park at about 10:00 and pay the park entrance fee. The fee is 18000CLP. You will also listen to a short talk telling you not to “pirate” camp, and don’t burn down the forest. The bus will wait for you. Continue taking the bus to the second stop, which is right next to Lago Pehoe. Disembark the bus and walk to the catamaran. The boat goes between this second stop and refugio Paine Grande. The boat costs 8000CLP.

You will arrive Paine Grande at about 13:00 and need to start hiking immediately. This day you will hike 11KM to Refugio Grey where you will set-up tent and put down your bags. The stay is 4000CLP. After setting down your bags you will keep going along the trail to the Mirador overlooking the glacier - this hike is about 4KM. Arrive as early as possible, as the sun will set behind the glacier and taking photos will become difficult.

Day 2

Wake up rather early, cook breakfast, and hike from Refugio Grey back to Refugio Paine Grande (11KM, ~3.5hr). Have a quick lunch, and continue hiking to Campamento Italiano (7.6KM, ~2.5hr). This is a free camp-site, so set-up tent, cook dinner, and get ready for bed. There is a water sprout near the top of the camp, so you don’t need to walk to the river for water.

Day 3

Wake up and hike the French Valley (7.5KM each way,  ~3hr each way). Return to camp, collect your gear, and walk to Los Cuernos (5.5KM, ~1.5hr). This stay is 8000CLP, but the showers were really hot. The camp-site fills up pretty early, as people are coming from both directions, so try to arrive earlier rather than later.

Day 4

Hike from Los Cuernos to Campamento Torres. About 9KM out of Los Cuernos you will encounter a big sign that says “SHORTCUT” - take it. This will take you around the backside of one mountain, by a lake, and cut some time off an already pretty long day. It is maybe 3.5hr from the shortcut to Campamento Torres, but it is all uphill. The trail gets very well used after merging with the trail from Los Torres hotel. Campamento Torres is a free camp-site. Consider camping uphill from the bathroom.

Day 5

Wake up quit early and depart for the Base de las Torres for sunrise. Most people leave Campamento Torres by about 6:15am, but when I went it was overcast, and the sun didn’t actually hit the mountain until almost 7:50. The walk takes about 45 minutes. Take your photos, hike back down to camp, pack up, and get back to Hotel Los Torres by 14:00. I ate breakfast at sunrise at the Base de las Torres (trail mix), but had a hot lunch at Campamento Chileno. A mini bus will pick you up by Hotel Los Torres at 14:00 and take you back to Laguna Amarga. This mini bus costs 2500CLP. Your bus will depart back to Puerto Natales at 14:30. Please shower and do laundry as soon as you get back - you probably smell like you’re homeless.

So, what are some things that worked really well for me?

One of the best things I did was keep our packs light. Our packs were both under 10KG, including five days food, sleeping bags, pads, and tent. My pack was probably about 7KG. Probably the biggest regret I heard on the trail was that everyone’s pack was too heavy. I would say most were above 20KG, even for females. A lot of this was due to the food choices made. I read a book on ultralight hiking in university, so knew some basic rules for keeping pack weight down:

  1. Don’t buy food in tin cans or water (e.g. tuna)
  2. Don’t bring fresh fruits or vegetables
  3. Bring plastic or titanium/aluminum cutlery
  4. Don’t bring “it” if you won’t have to use “it” every day
  5. No knives or leatherman (you don’t have tins to open now…)
  6. 1x pants, 2x shirts, 2x socks, 2x underwear (or none), 1x long sleeve, 1x fleece, 1x down vest (maybe if cold), and 1x windbreaker/raincoat. That’s it. You don’t need three jumpers or five pairs of underwear. You probably don’t even need the shirts as a base.
  7. Put duct tape on random things (e.g. trekking poles) instead of bringing a roll of duct tape.
  8. Buy food that cooks quickly, not types of pasta that take 20 minutes. Risotto is pretty efficient (boil, heat it some, take off flame and cover), as are some thing types of pasta in soup.
  9. You can eat out of the bowls you are cooking in - you don’t need pots and plates.
  10. Bottles for only enough for 5 days. A full tube of toothpaste, big bottle of shampoo, and tube of sun screen all ads up the weight. The same is true for pills - you don’t need a full pack of multivitamins, five will do.

I heard stories of people cooking pancakes and french fries somehow, which would taste amazing, but I would rather eat risotto and have a pack 1/3 the weight. I both made it through with not a single blister, whereas the person next to me right now has seven.

Other suggestions of things that  worked well for me include:

  1. Wool or wool blends of everything. It dries fast, keeps you warm, and doesn’t keep odour. Get some wool blend underwear for travelling if you don’t already have some.
  2. Bring a super light day pack (like one of these) that you can toss water and food into for hikes that don’t require the full pack. If your full pack is comfortable enough, then just use that.
  3. Polarized UV protection glasses are a must, as is a hat. There were some seriously sun burned people returning from the trip, despite putting on “two layers of sunscreen”. Wear a hat.
  4. I brought my MSR “dragonfly” stove with me, and the normal cup to cook with. Erratic Rock / Base Camp have a container with half empty gas canisters. If you don’t mind risking running out of gas, grab one from here. Otherwise gas is about 8 bucks.
  5. Find additional people to share food with. Oatmeal is only sold by 1KG packages, which is a lot. Most people were throwing away leftover oatmeal at the end of the trip.

However, there were a few things that did not work that well, or not work as well as expected. The worst thing was a growing pain in my right knee. I’ve always had some pain in my knee after hiking, but by day two I was starting to have severe pain. The trails have quite steep ascends/descends. This destroyed my knee. On day four another walker who knew about my knee passed by and said “I have some trekking poles - I tried them, tripped over myself, and haven’t used them since. Wanna try?” I had never used poles before, thinking they were only for old people with bad knees… Needless to say, I’ll be buying a pair when I get back to Sydney, and I finished the day 4 hike without any pain. If you ever, even once, have had pain in your knees, then rent trekking poles. I had to skip the French Valley because my knees were hurting too much (I guess technically I hiked the “U”, not the “W”). If my pack was not so light, I do not think I would have been able to go up to the Torres.

The second mistake I made was not bringing a flashlight. My logic was I would just go to bed when it got dark, and rise when it got light. This is what I have always done before when trekking. Unfortunately, this does not work that well when you want to be up before the sun rises, e.g. to see the Base de las Torres. I followed (closely) some people with torches, so made it, but came close to tripping quite a few times, and it generally wasn’t that enjoyable. Most people had those LED lights that go on your forehead - that would be advisable.

Finally, I brought a Platypus for water. Hiking back in Oregon there would be some stretches with no water - maybe 4-5 hours of hiking without easy access to water. In the Torres del Paine, there was water nearly every two kilometers, so a Nalgene would have worked well. My better half just used her Nalgene, and it worked fine. It is also easier to fill up, and has measuring lines for how much water is needed for risotto.

That’s all the advice and information I can give about hiking the “W” in the Torres del Paine. The hike is a bit more expensive than I had expected, but there was some great camaraderie with other hikers, and it leaves you with a feeling of accomplishment when finished.

There are a few more sites that detail this trek, including:
How to hike the “W” in Torres del Paine
THE Definitive Guide to Hiking Torres del Paine

February Sydney Python Presentation

In February I gave a presentation to about 80 people at the Sydney Python group hosted by Atlassian. Firstly, Atlassian’s office was beautiful, feeling a little like Google’s Sydney office, but with beer on tap instead of cereal dispensers. Secondly, the talk before me on Cython by Aaron Defazio was exceptionally interesting, garnering lots of questions from the audience. My presentation, more of a show and tell on piping location data to Google’s Latitude through App Engine, was also meant to subtly share my views on the need for innovation in the public sector (all sectors, really).

My slides are below. I used very little text in the slides, but you can probably catch what is going on. The response from the audience was favourable, and I thank Dylan Jay for giving me the opportunity to speak.

Lessons Learned from Kathmandu

My first trip to Kathmandu is now over, so there are some lessons learned I should scribe. Some of these are obvious, and which I abide by whenever travelling, and some I simply forgot in my (very) impromptu trip to Nepal.

  • When agreeing on a price, make 100% sure the other person states the price back to you. I thought a price had been agreed to when the other person responded “ok ok, you are a lucky man”, but this does not count. As they say, reconfirm, reconfirm, reconfirm.
  • When arriving at the airport, make sure you have small bills, too. I had 3x 100RS, 1x 10RS, and then a few 500RS. The price I negotiated was 440RS, and it would have been nice to have paid the exact amount.
  • Kathmandu is polluted and dirty. I cannot emphasise this enough. It is dirtier than probably any other city I have been to. If I come back, I will be bringing masks. I know this sounds silly, to wear a mask, but any local on a motorbike or in a taxi wears a mask, and many just walking around. Instead of the normal cloth masks that many people use, I would probably bring a make with finer grained material - probably N95. I ultimately tried to avoid walking on main roads, but having some activated carbon absorb something would have made it a little less unpleasant.
  • Bring old cloths, and throw them away after the trip. Or just bring black. If you have nice jackets or clothing they will likely come back pretty dirty.
  • Bring some toilet paper. Similar to other parts of Asia, the bathrooms don’t have any.
  • Bring a flashlight. The load shedding makes the city dark, and if you go out, you will want a flashlight. There aren’t any lights. I only used it a few times, but I am really glad I brought two flashlights with us.
  • Bring vitamin C and lots of hand sanitizer. I did, like I always do when I travel, and I’m really glad I did. Everybody is coughing or sick, and everybody spits. It is similar to the situation in China, i.e. everyone spits. Then everybody gets sick. Bring hand sanitizer.
  • If you take a bus somewhere, try to ask when you buy tickets to sit on the left side in the middle. The front is a no-go for me. I typically had seats in the rear right, but on the curvy roads I think the left middle would be safer, as oncoming buses won’t hit you. One bus on the way back had the left side decimated. If you do a search on “nepal bus crashes” in images.google.com, you will quickly see why you don’t want to be in the front row.
  • My hotel rooms all typically had just one power outlet. If you bring multiple electronic devices, bring some way to  charge more than one at a time.
  • Bring clothing to stay warm at night. I travelled to Nepal in winter, and all my rooms got pretty cold at night.

Enjoy!

Migrate Custom Blog to Blogger

For the last ten years I have run this website from various systems. First it was on Wordpress, then Mambo, then Joomla, and since early 2006 it has been running on custom code written using Django. I used this site as a learning tool for Django, re-wrote it after gaining more knowledge of Django, and then re-wrote it again when Google released App Engine. However, I recently realised that for the last few years I have spent more time writing little features than actually writing. I have entire trips that I never wrote because I was too busy writing code.

This week it all changed. I did the unthinkable. I moved this website to Blogger.

After evaluating some of the features of blogger, i.e. custom domains, location storing, ability to filter on labels, custom HTML/CSS, great integration with Picasa, and their mobile app, I realised I could virtually replace everything I had previously custom made.

This post gives a technical description how to migrate a site running Django, but readily applies to any blog running on custom code. I initially spent a fair bit of time trying to figure out how to convert my existing RSS feed into something Blogger could accept, but every solution required troubleshooting. I soon remembered why I love Django so much, and that it would be trivial to generate the correct XML for import.

  1. Create Blogger Template
    I wanted to keep my design, so I hacked it to support Blogger. Take one of the existing templates, edit the HTML, and adjust it for your design. If you’ve worked with templates before this shouldn’t be too difficult.
  2. Generate Sample XML
    The first step was to generate a sample XML file from Blogger to see what would be needed for import. Create a sample post with a unique name and a few labels, and location. In Blogger, go to Settings->Other and click Export Blog. The first 90% of the file will be for your template and other settings, but eventually you will find a section with entry elements in it. Copy this sample element out - this will become your template.
  3. Format Template
    Using the sample section from the blog export, format it so the view you will create populates it correctly. A note of caution: the template needs time in ISO 8601 format, you need the id element, and the location element needs coordinates if there is a name. It won’t import later if there is a name with no coordinates. My template looks like this:

feeds/rss.html

{%  load blog_extras %}
{% for entry in entries %}
    tag:blogger.com,1999:blog-1700991654357243752.post-{% generate_id %}
        {{ entry.publish_date|date:"Y-m-d" }}T10:30:00.000123
        {{ entry.publish_date|date:"Y-m-d" }}T10:30:00.000123
        {% for tag in entry.tags %}
            {% endfor %}

        {{ entry.title }}
        {{ entry.content }}

        

        Joe Bloggs
            https://plus.google.com/12345689843655881853
            [email protected] 
{% endfor %}

This isn’t really RSS, so if you are pedantic you can name it something else. You will notice I loaded some template tags in there (“blog_extras”). This is for generating the random number, as this is needed for the ID element.. Here’s the template tag.

blog_extras.py

# 'import random' at beginning of file
def generate_id():
    id = ""
    for x in xrange(1, 7):
        id = id + str(int(random.uniform(400, 900)))
    id = id + "8"
    return {'obj': id}
register.inclusion_tag('blog/generate_id.html')(generate_id)

/blog/generate_id.html

{{ obj }}
  1. Create Code To Populate View

This section should be easy if you have written your blog in Django. Simply populate the template, what I have shown as “rss.html” above

blog/views.py

def show_rss(self):
    q = Entry.all()
    q = q.filter("genre !=", "blog")
    entries = q.fetch(500)
    return render_to_response("feeds/rss.html", {
        'entries': entries,
        }, mimetype='text/plain')

I did a filter on the model to not include “blog” entries - these are my travel stories, and I exported them separately. Remember that this is all happening on App Engine, so you will need to adjust if using Django’s normal ORM.

  1. Download Entries

Visit the URL you mapped to the “show_rss” function in urls.py, it should generate your list of entries. Copy and paste those entries into the exported XML from Blogger where you took out the original entry element.

  1. Import Entries

Now go to Blogger and import your blog. With any luck you will have imported all your entries. You will probably need to do this a few times as you tweak the text. I had to remove some newlines from my original posts.

Optional Steps

  1. Create Redirect URLS
    Links in Blogger appear to only end in .html, which is a problem for links coming from Django. Luckily, Blogger includes the ability to add redirects. Go to Settings->Other-Search Preferences. You can then edit redirects there. I generated a list of my old URLs and combined that with a list of the new URLs. Hint: you can use Yahoo Pipes to extract a list of URLS from a RSS feed. If you open any of the links in Excel and split on forward slashes, remember that it will cut off leading zeros. Set that field to TEXT during import.

I decided not to create redirects for every entry, as I didn’t really have time, and it only probably matters if somebody links directly to that page. I opened Google Analytics and looked at the Search Engine Optimisation page and sorted it by the most used inbound links. After getting down to entries that only had 1 inbound request per month I stopped creating redirects.

  1. Host Stylesheets and Images Externally

Blogger won’t host host files, so you need to work around this problem. All my images are generally from Picasa, except very specific website related ones. I moved those to Amazon’s S3 and updated the links. I did the same with my CSS. You could probably store them in Google Storage, too.

  1. Create Filters on Labels

If you had any previous groupings you can still link to them using label searches (in my case I actually added the “genre” as a label). The syntax is “/search/label/labelname/”, as you can see in my howtos section link.

  1. Update Webmaster Tools

If your site is part of Google’s Webmaster Tools, you will want to login and take a look that things are OK. You will also probably want to update your sitemap (send Google your atom.xml feed).

How to convert 131500 TDX to GTFS

TDX data has been available for a number of years on 131500.info, but many tools are GTFS specific. I also find GTFS easier to work with.

Luckily, converting from TDX to GTFS is not overly difficult, and below are some instructions. This howto is a bit old, as I am only now copying it from my “Notes” folder to put online to help others.

Note: You can now directly download GTFS from the TransportInfo website: https://tdx.131500.com.au

  1. Signup for an account with EC2 (AWS), unless you have 16GB of memory available on a machine.
  2. Upload TransXChange2GTFS to a place you can download from.
  3. Upload the latest TDX data dump from 131500.info to a place you can download from.
  4. Login to AWS and start an EC2 instance.  I picked a large instance and used 64-bit Ubuntu 10.04, us-east-1 ami-f8f40591
  5. Download the Data and transxchange to /mnt
wget http://ec2-175-41-139-176.ap-southeast-1.compute.amazonaws.com/Data20110127.zip
wget http://cdn.kelvinism.com/transxchange2GoogleTransit.jar

EDIT 16-03-2025: I’ve since removed these files.

  1. Install Sun JRE.
apt-get install python-software-properties
add-apt-repository "deb http://archive.canonical.com/ lucid partner"
apt-get update
apt-get install sun-java6-jre
  1. Check how much memory is available
root@domU-12-31-39-10-31-B1:/mnt# free -m
             total       used       free     shared    buffers     cached
Mem:          7680        626       7053          0         11        329
-/+ buffers/cache:        285       7394
Swap:            0          0          0
  1. Create a configuration file sydney.conf
url=http://131500.info
timezone=Australia/Sydney
default-route-type=2
output-directory=output
useagencyshortname=true
skipemptyservice=true
skiporhpanstops=true
  1. If you’re on the train like me, start screen, and start converting. The number you pick for “-Xmx” obviously needs to fit in the amount of free memory you have.
java -Xmx104000m -jar dist\\transxchange2GoogleTransit.jar Data20120524.zip -c sydney.conf

Goodbye Facebook

I seldom used Facebook, and felt some kind of negative emotion after logging on each time. I don’t know if I am alone in feeling this way, but each time still felt a bit like a secret contest. A bit, not a lot, but sort of subconsciously. Honestly, though, even at an involuntary subconscious level, I don’t like this sort of negative feeling. I would rather be sharing ideas or creating something. Wish me luck.

EDIT 12-05-2025: Still off Facebook, and generally all other social media. I try to chase my dopamine through other means.

131500trains Updates to GTalk

Here’s a little hack I thought I’d share. There are maybe other ways to do this, but this way took all of two minutes, which suits me. My daily routine has me taking a train along the Inner West Line (here in Sydney), and I wanted to know in advance if there was any issues with the line. 131500 provides weekely emails about upcoming trackwork, and there’s an RSS feed of delays; this is what you’d see in many of the Sydney transport-related apps. Another information source is through Twitter. For all of these sources I only really care about the one line I take every day - the quickest answer would seem to be if I could only get updates on Twitter for my line

Luckily this is all quick to do using an only service called ifttt. What I needed to do was create a task searching Twitter for “Inner West from:131500trains”, and if any new items are located it sends me a message on GTalk. This is one example of what I like about having some mostly open data - I can consume it how I want it. Let’s just say I’m looking forward to GovHack 2012

This example could be extended for other purposes, e.g. searching for the word “discount” on some suppliers Twitter stream.