A short while ago I wrote about visualizing transport by using 131500’s TDX data, converted to GTFS, and served by GeoServer. Because I’ve started playing around with Google’s Fusion table, I thought it would be interesting to see what all the transit stops in Sydney look like in FT. So, voila!
Recently I tried editing my Cisco’s ACL at home on the train. It went something like this:
Although it is simple enough to just ask her to “flip the switch on the black box”, I still don’t like doing it. Plus, if she’s not home, I’m stuck. This accident immediately reminded me of one of a trait of the ‘reload’ command: it can be scheduled.
In the case of updating a device remotely, it is as easy as:
router# reload in 2
router# conf t
router(config)# [type in desired commands]
router(config)# exit
router# reload cancel
If the commands are entered in fine, then cancel the reload. If there is a problem, then the router will reboot and resort to the startup config.
If you want to use Graphserver to do some analysis with GTFS, you will need to convert GTFS into the database. This is how I did it.
I used the following AMI. If you have enough memory, you don’t need to do this.
ami-7000f019
Lookup and read the GTFSDB INSTALL.txt document
sudo apt-get install mercurial
hg clone https://gtfsdb.googlecode.com/hg/ gtfsdb
sudo apt-get install python-setuptools
sudo easy_install psycopg2
sudo apt-get install build-essential
ubuntu@domU-12-31-39-00-5D-B8:/mnt/gtfsdb$ pwd
/mnt/gtfsdb
sudo python setup.py install
sudo wget http://cdn.kelvinism.com/google_transit.zip
sudo apt-get install python-psycopg2
EDIT 16-03-2025: I’ve since removed these files.
#default.cfg
[options]
create = True
database = postgresql://nsw:[email protected]:5432/nsw
filename = /mnt/google_transit.zip
geospatial = True
#schema = None
screen
python gtfsdb/scripts/load.py
I’ve had several conversations with neighbors and co-workers about the “lack” of forward thinking, or at least the lack of forward action. Of course, I keep in the back of my mind that we aren’t “experts”, and the more I learn about transport the more I learn how complex it is. Dr. Sussman’s CLIOS process (Complex, Large-
Scale, Integrated, Open Systems) appears more and more true the longer I work in and study transport. There is a plethora of excuses that can be made, but the general conclusion was that the earlier we prepare the better. I can remember working near Zhongshan 7-8 years ago and driving around on huge roads in the middle of empty fields. There weren’t even stoplights at every intersection. It was then that I had an epiphany of how smart the planning was to build the infrastructure before the masses arrived.
Sydney is estimated to increase by some 1.7 million people by 2036, and I can tell you, from a transportation (private and public) standpoint, that sort of scares me. When people ask me why transport is so difficult I justify it by with my uneducated guess that the CBD is next to the ocean, so everybody travels in from just 180 degrees instead of 360. Maybe this is why the NSW government created the “City of Cities” strategy. I realized this within the first few weeks: most people live west but work east.
Tonight (a Saturday) I was bored, and should have been studying, but wanted to create a few visualizations first.
The below maps were created using TDX data released from 131500. After converting it to GTFS I imported it into PostGIS using GTFSDB, and then could serve it via GeoServer. Finally, I could access it via WMS in QGIS. I added the stops into a map of Sydney and added some boundaries, and added the Growth Zones. The result was a map with every bus/train/ferry stop. Darker areas have stops that are closer (not necessarily more frequent service).
One of the first things I noticed is that there isn’t much physical infrastructure in these areas. There also aren’t many transit stops; I suppose this is why the South West Rail Link is going to be so important. I don’t know all of the political ramifications, but let’s hope the North West Rail Link is built as well?
I’ve been cleaning up my personal blog a bit, and I noticed that my tagging system recently broke. I’ve investigated the cause, and it appears to be because I removed some apps but the contenttypes remained. This meant that whenever I tried calling a tag with a TaggedItem that had been deleted, I was getting this error:
'NoneType' object has no attribute '_meta'
The solution is to first list all app_labels for contenttypes, and then delete any not in use.
In [61]: from django.contrib.contenttypes.models import ContentType
In [62]: for ct in ContentType.objects.all(): print ct.app_label
....:
picasaweb
lifestream
readernaut
delicious
mapfeed
comments
...
I could then delete the unused contenttypes.
ct_list = ["delicious", "flickr", "photologue", "twitter"]
for ct_label in ct_list:
for ct in ContentType.objects.filter(app_label=ct_label):
ct.delete()
And no more errors! For more details take a look at David’s article.
I recently had the desire to send small updates to my so called lifestream page via XMPP/GTalk. I played around with Twisted Words and several other Python XMPP clients, but I didn’t really want to keep a daemon running if unnecessary. It turns out imified took a lot of the pain out of it. The steps for me were as follows:
Create an account with imified, and create a URL, e.g. /app/api/
We then configure the urls.conf
urlpatterns = patterns('',
(r'^app/api/$', bot_stream),
)
We then create the necessary views. So, in views.py:
from django.shortcuts import render_to_response
from django.http import HttpResponse
from lifestream.forms import *
from datetime import datetime
from time import time
def bot_stream(request):
if request.method == 'POST':
botkey = request.POST.get('botkey')
username = request.POST.get('user')
msg = request.POST.get('msg')
network = request.POST.get('network')
if username == "[email protected]" or network == "debugger":
blob_obj = Blob(id=time(), body=msg, service_name="Mobile",
link="http://www.kelvinism.com/about-me/", published=datetime.now())
blob_obj.save()
resp = "OK"
else:
resp = "Wrong username %s" % username
else:
resp = "No POST data"
return HttpResponse(resp)
To complete this little example, you can see what I used for my models.py
class Blob(models.Model):
id = models.CharField(max_length=255, primary_key=True)
body = models.TextField(max_length = 1024, null = True, blank = True)
service_name = models.CharField(max_length=50, null=True, blank=True)
link = models.URLField(max_length=255, verify_exists=False, null=True, blank=True)
published = models.DateTimeField(null=True, blank=True)
def __unicode__(self):
return self.id
class Meta:
ordering = ['-published']
verbose_name = 'Blob'
verbose_name_plural = 'Blobs'
def get_absolute_url(self):
return "/about-me/"
It maybe isn’t super elegant, but it works just fine, and maybe can provide a hint if somebody else is contemplating using a homebuilt xmpp solution, or just pawning it off on IMified.
I was having some issues with my personal mail server (Postfix) and my phone (Android). The logs depicted the below issue:
Jan 9 09:19:53 ip-11-222-23-223 postfix/smtpd[12345]: NOQUEUE: reject: RCPT from 12-13-14-15.abc.com.au[12.13.14.15]: 504 5.5.2 <localhost>: Helo command rejected: need fully-qualified hostname; from=<emailaddr kelvinism.com="kelvinism.com"> to=<emailaddr gmail.com="gmail.com"> proto=ESMTP helo=<localhost>
</localhost></emailaddr></emailaddr></localhost>
We can see here that the stock Android email client is doing a ‘helo localhost’. One part of my main.cf file specifies this:
smtpd_helo_required = yes
smtpd_helo_restrictions =
permit_mynetworks,
reject_non_fqdn_helo_hostname,
reject_invalid_helo_hostname,
permit_sasl_authenticated,
permit
To resolve, unfortunately, just change the order to authenticated clients are permitted earlier:
smtpd_helo_required = yes
smtpd_helo_restrictions =
permit_mynetworks,
permit_sasl_authenticated,
reject_non_fqdn_helo_hostname,
reject_invalid_helo_hostname,
permit
You also may need to do the same for smtpd_recipient_restrictions and/or smtpd_sender_restrictions (i.e. put permit_sasl_authenticated above the reject lines).
A few weeks ago I saw an opening to give a 5-10 minute lightening talk at SyPy (Sydney Python), and with two nights to prepare, decided it would be interesting to explore Splunk’s usage of Python. You can see it here
I like using Ubuntu’s UFW command, but today I needed to allow outgoing ICMP. I received results as so:
$ ping 4.2.2.2
PING 4.2.2.2 (4.2.2.2) 56(84) bytes of data.
ping: sendmsg: Operation not permitted
ping: sendmsg: Operation not permitted
ping: sendmsg: Operation not permitted
To allow outbound icmp I edited ‘before.rules’ and added the following lines.
$ sudo vi /etc/ufw/before.rules
# allow outbound icmp
-A ufw-before-output -p icmp -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT
-A ufw-before-output -p icmp -m state --state ESTABLISHED,RELATED -j ACCEPT
I recently had the need to add some HTML to the label for a form field using Django. The solution is pretty easy, except I didn’t see it written explicitly anywhere, and I missed the memo of the function I should be using.
My form first just had the HTML in the form label as so:
from django import forms
class AccountForm(forms.Form):
name = forms.CharField(widget=forms.TextInput(), max_length=15, label='Your Name (<a href="//www.blogger.com/questions/whyname/" target="_blank">why</a>?')
However, when I displayed it, the form was autoescaped.
This is generally a good thing, except my form obviously didn’t display correctly. I tried autoescaping it in the template, but that didn’t work. To resolve this you’ll need to mark that individual label as safe. Thus:
from django.utils.safestring import mark_safe
from django import forms
class AccountForm(forms.Form):
name = forms.CharField(widget=forms.TextInput(), max_length=15, label=mark_safe('Your Name (<a href="//www.blogger.com/questions/whyname/" target="_blank">why</a>?)'))
It will now display correctly:
In [1]: from myproject.forms import *
In [2]: form = AccountForm()
In [3]: form.as_ul()
Out[3]: u'
<li><label for="id_name">Your Name (<a href="//www.blogger.com/questions/whyname/" target="_blank">why</a>?):</label> <input id="id_name" maxlength="15" name="name" type="text"></li>
'
There’s maybe another easier way to do this, but this worked for me.