Lessons Learned: Losing Parts

Published on Thursday, November 24, 2011

I don't know how many times I've lost tiny parts and had to spend time trying to find them. I realized after removing the back of my monitor that I'm starting to learn from those mistakes; instead of just sticking the screws in a box with other screws, and then having to dig them out, I taped them to the back of the monitor.



This sounds small, but after doing it I realised how many times I had to hunt through a box of screws trying to find the exact one that fit. Lesson learned.

Destroying Hard Drives

Published on Friday, October 14, 2011

If you're like me there is a box of hard drives sitting in a dusty corner somewhere. Some are mine, some are others', but they are all in a failed or semi-failed state. So, why have I lugged them around? I've been a bit paranoid about throwing them away. Some of the hard drives are encrypted, others aren't, and the drives from friends certainly aren't. Although the chances of somebody getting the drive from a landfill and restoring it is minimal, I never wanted to take the chance.
So, I kept lugging a bag of drives through each move.
My friend Clinton has recently returned from Europe, and he brought me a gift: a Swiss-made Victorinox, the 'CyberTool'. After playing with it for several minutes I noticed it came with a Torx 8, 10 and 15 bit. This was a reminder that one method of mostly disabling a hard drive is to destroy the platters.
Other methods I have heard are to use a hammer and nails - which I unfortunately don't have in our tiny apartment. Whilst finishing Dexter I started pulling apart the box of hard drives, and it surprisingly didn't take long to disable them.
Step 1: Remove Torx screws





Hint: Don't forget the screw covered under the paper.
Step 2: Use a flathead and pop off the cover






Step 3: Stare at the shiny platters





Step 4: Scratch platters with flathead, and bend platters if possible






Step 5: Dispose of bits and pieces






Warning! The CyberTool didn't have a Torx small enough to open the 2.5" hard drives, but I could just use a pair of pliers to lift up the cover and jam a flathead in there. However, and here's the warning, 2.5" platters are sometimes not made of metal. I forgot about this on the third 2.5" hard drive and covered the desk I share with YS with shards of glass.

Fusion Tables and 131500 Stops

Published on Saturday, June 11, 2011

A short while ago I wrote about visualizing transport by using 131500's TDX data, converted to GTFS, and served by GeoServer. Because I've started playing around with Google's Fusion table, I thought it would be interesting to see what all the transit stops in Sydney look like in FT. So, voila!

Reload a Cisco Router WIthout Worry

Published on Wednesday, June 8, 2011

Recently I tried editing my Cisco's ACL at home on the train. It went something like this:
  • I logged in
  • I started updating the ACL
  • I hit a blackspot in my 3g coverage
  • My command stops at "router(config)#access-"
  • I get a call from my partner saying "what did you do to our internet!"
Although it is simple enough to just ask her to "flip the switch on the black box", I still don't like doing it. Plus, if she's not home, I'm stuck. This accident immediately reminded me of one of a trait of the 'reload' command: it can be scheduled.

In the case of updating a device remotely, it is as easy as:

router# reload in 2
router# conf t
router(config)# [type in desired commands]
router(config)# exit
router# reload cancel

If the commands are entered in fine, then cancel the reload. If there is a problem, then the router will reboot and resort to the startup config.

Converting GTFS to GraphServer

Published on Monday, June 6, 2011

If you want to use Graphserver to do some analysis with GTFS, you will need to convert GTFS into the database. This is how I did it.

Get an appropriate AMI from Amazon's EC2

I used the following AMI. If you have enough memory, you don't need to do this.

ami-7000f019

Lookup and read the GTFSDB INSTALL.txt document

Prepare system

sudo apt-get install mercurial
hg clone https://gtfsdb.googlecode.com/hg/ gtfsdb
sudo apt-get install python-setuptools
sudo easy_install psycopg2
sudo apt-get install build-essential

Download GTFS database

ubuntu@domU-12-31-39-00-5D-B8:/mnt/gtfsdb$ pwd
/mnt/gtfsdb
sudo python setup.py install
sudo wget http://cdn.kelvinism.com/google_transit.zip
sudo apt-get install python-psycopg2

Prepare configuration file

#default.cfg
[options]
create = True
database = postgresql://nsw:131500@10.128.49.175:5432/nsw
filename = /mnt/google_transit.zip
geospatial = True
#schema = None

Perform import

screen
python gtfsdb/scripts/load.py

Visualizing Transport

Published on Sunday, April 3, 2011

I've had several conversations with neighbors and co-workers about the "lack" of forward thinking, or at least the lack of forward action. Of course, I keep in the back of my mind that we aren't "experts", and the more I learn about transport the more I learn how complex it is. Dr. Sussman's CLIOS process (Complex, Large-
Scale, Integrated, Open Systems) appears more and more true the longer I work in and study transport. There is a plethora of excuses that can be made, but the general conclusion was that the earlier we prepare the better. I can remember working near Zhongshan 7-8 years ago and driving around on huge roads in the middle of empty fields. There weren't even stoplights at every intersection. It was then that I had an epiphany of how smart the planning was to build the infrastructure before the masses arrived.

Sydney is estimated to increase by some 1.7 million people by 2036, and I can tell you, from a transportation (private and public) standpoint, that sort of scares me. When people ask me why transport is so difficult I justify it by with my uneducated guess that the CBD is next to the ocean, so everybody travels in from just 180 degrees instead of 360. Maybe this is why the NSW government created the "City of Cities" strategy. I realized this within the first few weeks: most people live west but work east.

Tonight (a Saturday) I was bored, and should have been studying, but wanted to create a few visualizations first.

The below maps were created using TDX data released from 131500. After converting it to GTFS I imported it into PostGIS using GTFSDB, and then could serve it via GeoServer. Finally, I could access it via WMS in QGIS. I added the stops into a map of Sydney and added some boundaries, and added the Growth Zones. The result was a map with every bus/train/ferry stop. Darker areas have stops that are closer (not necessarily more frequent service).








One of the first things I noticed is that there isn't much physical infrastructure in these areas. There also aren't many transit stops; I suppose this is why the South West Rail Link is going to be so important. I don't know all of the political ramifications, but let's hope the North West Rail Link is built as well?

Removing Unused ContentTypes

Published on Wednesday, January 19, 2011

I've been cleaning up my personal blog a bit, and I noticed that my tagging system recently broke. I've investigated the cause, and it appears to be because I removed some apps but the contenttypes remained. This meant that whenever I tried calling a tag with a TaggedItem that had been deleted, I was getting this error:
'NoneType' object has no attribute '_meta'


The solution is to first list all app_labels for contenttypes, and then delete any not in use.
In [61]: from django.contrib.contenttypes.models import ContentType

In [62]: for ct in ContentType.objects.all(): print ct.app_label
   ....: 
picasaweb
lifestream
readernaut
delicious
mapfeed
comments
...

I could then delete the unused contenttypes.
ct_list = ["delicious", "flickr", "photologue", "twitter"]

for ct_label in ct_list:
    for ct in ContentType.objects.filter(app_label=ct_label):
        ct.delete()
    


And no more errors! For more details take a look at David's article.

Integrate imified into Django

Published on Tuesday, January 18, 2011

I recently had the desire to send small updates to my so called lifestream page via XMPP/GTalk. I played around with Twisted Words and several other Python XMPP clients, but I didn't really want to keep a daemon running if unnecessary. It turns out imified took a lot of the pain out of it. The steps for me were as follows:
Create an account with imified, and create a URL, e.g. /app/api/
We then configure the urls.conf
urlpatterns = patterns('',   
(r'^app/api/$', bot_stream),
)


We then create the necessary views. So, in views.py:
from django.shortcuts import render_to_response
from django.http import HttpResponse
from lifestream.forms import *
from datetime import datetime
from time import time

def bot_stream(request):
if request.method == 'POST':
botkey = request.POST.get('botkey')
username = request.POST.get('user')
msg = request.POST.get('msg')
network = request.POST.get('network')

if username == "username@gmail.com" or network == "debugger":
blob_obj = Blob(id=time(), body=msg, service_name="Mobile",
link="http://www.kelvinism.com/about-me/", published=datetime.now())
blob_obj.save()
resp = "OK"
else:
resp = "Wrong username %s" % username
else:
resp = "No POST data"
return HttpResponse(resp)



To complete this little example, you can see what I used for my models.py
class Blob(models.Model):
id = models.CharField(max_length=255, primary_key=True)
body = models.TextField(max_length = 1024, null = True, blank = True)
service_name = models.CharField(max_length=50, null=True, blank=True)
link = models.URLField(max_length=255, verify_exists=False, null=True, blank=True)
published = models.DateTimeField(null=True, blank=True)

def __unicode__(self):
return self.id

class Meta:
ordering = ['-published']
verbose_name = 'Blob'
verbose_name_plural = 'Blobs'

def get_absolute_url(self):
return "/about-me/"



It maybe isn't super elegant, but it works just fine, and maybe can provide a hint if somebody else is contemplating using a homebuilt xmpp solution, or just pawning it off on IMified.

Stock Android and Postfix

Published on Sunday, January 9, 2011

I was having some issues with my personal mail server (Postfix) and my phone (Android). The logs depicted the below issue:



Jan  9 09:19:53 ip-11-222-23-223 postfix/smtpd[12345]: NOQUEUE: reject: RCPT from 12-13-14-15.abc.com.au[12.13.14.15]: 504 5.5.2 : Helo command rejected: need fully-qualified hostname; from= to= proto=ESMTP helo=


We can see here that the stock Android email client is doing a 'helo localhost'. One part of my main.cf file specifies this:

smtpd_helo_required = yes
smtpd_helo_restrictions =
    permit_mynetworks,
    reject_non_fqdn_helo_hostname,
    reject_invalid_helo_hostname,
    permit_sasl_authenticated,
    permit


To resolve, unfortunately, just change the order to authenticated clients are permitted earlier:

smtpd_helo_required = yes
smtpd_helo_restrictions =
    permit_mynetworks,
    permit_sasl_authenticated,
    reject_non_fqdn_helo_hostname,
    reject_invalid_helo_hostname,
    permit


You also may need to do the same for smtpd_recipient_restrictions and/or smtpd_sender_restrictions (i.e. put permit_sasl_authenticated above the reject lines).