Reduce Bandwidth By Shrinking Images

A friend has a simple WordPress-based website that I set up for him a few year ago - and he likes to post a lot of images. For various reasons I have it hosted on my VPS in Australia, which only comes with 15GB of bandwidth, most likely because I got a crazy good deal on it.

The problem is, that isn’t much bandwidth. Each page of his site was previously using almost 10MB of bandwidth per view - which meant the 15GB was running out. I deduced the high usage in Firebug and noticed it was gasp from large images, some almost 1MB large. The images on the server are contained in subdirectories under a gallery folder - and I wanted to resize them. What I actually wanted to do was:

  1. If the image was over 1024x900, resize it.
  2. If the image size (bytes) was over X, lower the quality.

I ended up using ImageMagick, which is what first obviously came to mind for image manipulation. While there is likely a better way to resize all the images, this is what I came up with. Maybe somebody else will find it useful too.

for dir in /home/vhosts/website.com/html/wp-content/gallery/*
do
cd $dir
for filename in *.jpg
do
echo Filename: "$filename"
width=`identify -ping -format %w "$filename"`
height=`identify -ping -format %h "$filename"`
if [ $width -ge 1024 -a $height -ge 900 ]; then
convert "$filename" -resize "%80>" "$filename"
echo New Dimensions: $width x $height
fi
filesize=`stat -c%s "$filename"`
if [ $filesize -ge 414981 ]; then
convert "$filename" -quality 90 "$filename"
echo Old Filesize: $filesize
echo New Filesize: `stat -c%s "$filename"`
fi
done
done

Save it as “resize.sh” and run it with “bash resize.sh”. The output:

Filename: img_5364.jpg
Old Filesize: 950748
New Filesize: 331426
Filename: img_5366.jpg
Filename: img_5367.jpg
Filename: 1c10_1.jpg
Filename: 1.jpg
Filename: 2752_1.jpg
Filename: 42920014.jpg
New Dimensions: 1544 x 1024
Filename: 42920017.jpg
New Dimensions: 1544 x 1024

French Word Collage

For the last year YS and I have been studying French once a week (4.5hr on Saturday!) A few months ago I wrote some Python code to read the RSS feeds of the most popular French newspaper site, and then kept track of the count of each word. I repeated this every week for a few months.
In the end I had a database of the top ~3000 words used on the site. Today I decided I wanted to play with Processing, so created this little word collage. It is 1680x1050, my monitor’s resolution. Two thumbs up for Processing!

Vortex86DX Instructions from ICOP

Alexandru T. and I have exchanged a few emails, and he sent through a few helpful suggestions that were provided from ICOP. I have included them below. Thanks Alexandru!

  1. Install Debian 5.0 on a normal PC (using a netinst image, for minimal install)
  2. After installation boot normally from the same PC
  3. Then, take the kernel from ftp://icop.com.tw/DIS_info/VDX/operating_system/VDX_Linux/linux-image-2.6.30-vortex86mx_1.0_i386.deb and then issue the following commands :
# dpkg -i  linux-image-2.6.30-vortex86mx_1.0_i386.deb
# update-initramfs -k 2.6.30-vortex86mx -c
# update-grub
# restart
  1. Then take the hard-drive and install it on the Vortex86DX
  2. When GRUB menu appears, press “e” and modify the boot loader as follows :
root        (hd0,0)
kernel        /boot/vmlinuz-2.6.30-vortex86mx root=/dev/hdb1 ro         --> if hdb1 does not work you can try (hda1= Primary Master or hdc1=Secondary Master)
initrd        /boot/initrd.img-2.6.30-vortex86mx

Then press b to boot

  1. After booting, go to /boot/grub/menu.lst and make modifications from above permanently, so you will boot without any intervention ;)

**Edit:**Bob A. has also sent through some additional resources for your eboxing pleasures.

FYI - this Swedish company, http://www.lweb.se/tag/ubuntu/, has a pre-made ISO for Ubuntu 8.04LTS with the correct kernel for the eBox 3300/3310. It even supports the new (1011) IDE controller on the recent models. You can just put the ISO on a thumb drive, stick it in your eBox, and install normally. No need to install first on another machine, and no need to update the kernel after you’re done. If you’re happy with running 8.04 then this is way easier than any other install option that I’ve found
so far.

Lenny on Ebox 3310A

As a preface, I take absolutely no credit for the below instructions. Stefan L kindly sent these through these instructions on installing Debian on the 3310A. I need to send a special thanks to Stefan, as I receive a lot of emails about the 3310 - but I don’t have one, so I can’t really do much:

The only edit I’ve done is change out the links to my files on S3. If you find these helpful, or want to suggest an alteration, please leave a comment.

Download these files first:

EDIT 16-03-2025: I have since removed these files.

The steps to install Lenny to CF in brief is:

  1. Install i386 version of Lenny to CF on another computer
  2. Add the revised kernel deb with dpkg -i *.deb
  3. Change fstab from hda1 to sdb1 (sda1 if there is no micro sd card) - uuids may be better
  4. Change /boot/grub/menu.lst to:
title           Debian GNU/Linux, kernel 2.6.31.5-vortex86-sl3
root            (hd0,0)
kernel          /boot/vmlinuz-2.6.31.5-vortex86-sl3 root=/dev/sdb1 ro verbose
  1. Probably need to change /boot/grub/device.map
(hd1)   /dev/sda
(hd0)   /dev/sdb

With no micro sd it would be:

(hd0) /dev/sda
  1. delete the section below “# PCI device ….” in /etc/udev/rules.d/70-persistent-net.rules (Otherwise the eBox network gets remapped to eth1 and may not appear if only eth0 is specified in the network settings)
  2. Reboot & pray The next one is a revised initrd for the current Ubuntu 9.10: http://staff.washington.edu/lombaard/initrd.img-2.6.31-14-generic-pata_rdc. (EDIT 12-03-2025: Lost my image that was linked from here). The two changes are: blacklist dm_raid45 & add pata-rdc.ko “blacklist dm_raid45” needs to be added to /etc/modprobe.d/blacklist.conf I managed to boot into gnome desktop without any further problems. I have enabled PCI IDE Bus Mastering, plug&play and IDE native mode in the bios. Hope this saves someone else a few hours of frustration.

New Atheros Module and Ubuntu

I’ve been using Atheros for quite some time, and I’ve always liked the madwifi drivers. They allowed really easy switching into monitor mode, and decent levels of packet injection. However, since I’m mostly in an office now, instead of writing web apps in cafes and trying to score free internet, I don’t really need anything fancy. My gentoo stage 1 (3?) days are over. I use Ubuntu, because I’m lazy, and it mostly works.

My new laptop (well, 1.5 year old laptop now, but still new in my eyes) gave me the option between an Intel card and a Atheros wifi card. I chose the Atheros card; then the ath5k module came out, and life has been turbulent ever since.

In summary: the ath5k driver in the 2.6.28 kernel, which is what Ubuntu 9.04 uses, isn’t as up-to-date as the drivers in compat-wireless. Fancy that… This presents me with the option of compiling a new kernel specifically with it, or just installing compat-wireless. I’m lazy, so…

I’ll get a few basic troubleshooting commands out of the way first. After updating the kernel I kept getting disconnected - it appeared I was associate/disassociating frequently.

# dmesg
...
2577.134060] wlan0: associated
[ 2580.984838] wlan0: disassociating by local choice (reason=3)
...
# lspci | grep Atheros
03:00.0 Ethernet controller: Atheros Communications Inc. AR5212 802.11abg NIC (rev 01)
# ping 192.168.1.1
...
64 bytes from 192.168.1.1: icmp_seq=2409 ttl=64 time=1.13 ms
64 bytes from 192.168.1.1: icmp_seq=2410 ttl=64 time=2236.61 ms
64 bytes from 192.168.1.1: icmp_seq=2411 ttl=64 time=4562.40 ms
64 bytes from 192.168.1.1: icmp_seq=2412 ttl=64 time=6521.868 ms
...

The steps to resolve are as follows:

  1. Make sure you have headers for your current kernel.
  2. Make sure you have ability to compile programs.
  3. Download and install compat-wireless
  4. Unload and load the module.

So, first, use Synapitc to get the latest kernel headers and the ‘build-essential’ packages.

Next, download the compat-wireless package. I needed to use one from a few weeks ago because I received the following error:

make -C /lib/modules/2.6.28-15-generic/build M=/usr/src/compat-wireless-2009-09-22 modules
make[1]: Entering directory `/usr/src/linux-headers-2.6.28-15-generic'
CC [M]  /usr/src/compat-wireless-2009-09-22/drivers/net/wireless/b43/main.o
/usr/src/compat-wireless-2009-09-22/drivers/net/wireless/b43/main.c: In function 'b43_do_interrupt':
/usr/src/compat-wireless-2009-09-22/drivers/net/wireless/b43/main.c:1888: error: 'IRQ_WAKE_THREAD' undeclared (first use in this function)
/usr/src/compat-wireless-2009-09-22/drivers/net/wireless/b43/main.c:1888: error: (Each undeclared identifier is reported only once
/usr/src/compat-wireless-2009-09-22/drivers/net/wireless/b43/main.c:1888: error: for each function it appears in.)
/usr/src/compat-wireless-2009-09-22/drivers/net/wireless/b43/main.c: In function 'b43_request_firmware':
/usr/src/compat-wireless-2009-09-22/drivers/net/wireless/b43/main.c:2218: warning: format not a string literal and no format arguments
/usr/src/compat-wireless-2009-09-22/drivers/net/wireless/b43/main.c: In function 'b43_wireless_core_start':
/usr/src/compat-wireless-2009-09-22/drivers/net/wireless/b43/main.c:3867: error: implicit declaration of function 'request_threaded_irq'
make[4]: *** [/usr/src/compat-wireless-2009-09-22/drivers/net/wireless/b43/main.o] Error 1
make[3]: *** [/usr/src/compat-wireless-2009-09-22/drivers/net/wireless/b43] Error 2
make[2]: *** [/usr/src/compat-wireless-2009-09-22/drivers/net/wireless] Error 2
make[1]: *** [_module_/usr/src/compat-wireless-2009-09-22] Error 2
make[1]: Leaving directory `/usr/src/linux-headers-2.6.28-15-generic'
make: *** [modules] Error 2

You can download a working 2009-09-05 set from orbit-lab.org

# tar -xpjf compat-wireless-2009-09-05.tar.bz2
# cd compat-wireless-2009-09-05
# make
# make install
# make unload
# modprobe ath5k

All done. My variable ping times and random disconnections seem to have been mitigated. Thanks wireless guys!

Remove Dead Tags

I’ve noticed my django-tagging install has been giving a lot of empty entries when doing a lookup on a tag. Tonight I finally got around to looking at what was causing this. This is surely not the best way to do this, but at 12:00am on a weekday, well, I shouldn’t be doing it in the first place… I first wanted to see what type of content was generating the error:

for item in TaggedItem.objects.all():
    try:
        print item.object
    except:
        print item.content_type_id

Now that I could see what was causing it (I had removed an app that used django-tagging, but it left the tags with empty pointers). Removing the empty tags was easy enough:

for item in TaggedItem.objects.all():
    try:
        print item.object
    except:
        item.delete()

No more hanging TaggedItems.

Most Frequently Used French Words

Status: ✅

I’m currently studying French, and if you’ve read any of this site, you’ll notice I’m a bit of a techie. Often several of my interests collide, which is what happened today. I was searching for the “most frequent french words,” and while I found some lists, nothing was exactly what I wanted.
My desire was to have a PDF of the top few thousand most used French words. With the English translation next to it. In order. I’ve found some great resources, which I’ll list now

I’ve also found 100s of sites with 50 words or so - not exactly what I wanted. This spawned a question for me: if I were to search a popular French newspaper website, what words would be the most frequent? I would want to learn those first. A few hours later, and I’ve compiled that list. I’ll write the details of how I did it at the end, but just know I collected over 16,000 unique words, and “read” over 80,000 words from a variety of articles. Below is a PDF of the most popular words, ranked in order, with (maybe incorrect) English translations.

The 625 Most Unofficially Frequent French Words

More to come…! I’m going to continue building the database to make sure the ranking is correct, and will make some pretty graphs when I have time. I will also likely modify things to include what type of word it is, and an example in a sentence.

Please feel free to use this list as you see fit in accordance with CC by 4.0

Testing SMS Gateways

For one of my projects I’m testing an SMS gateway, and decided it would be fun to build a useful alarm clock out of it. For those of you who know Python, you may find this funny. /dev/ttyUSB0 is my Arduino with a temperature sensor.

import serial
import urllib2
 
def check_temp():
    ser = serial.Serial('/dev/ttyUSB0', 9600)
    t = ser.readline().strip()
    return float(t)
 
    t = check_temp()
    if int(t) < 8:
        message = "It+is+now+%f+degrees;+chuck+a+sicky." % t
        f = urllib2.urlopen('http://api.clickatell.com/http/sendmsg?user=johnd&password=p@55w0rd&api_id=2132867&from=61433735555&to=61433735555&text=%s' % message)

```bash

And in crontab:

```bash
45 6 * * * python /opt/scripts/temp_alarm.py

Arduino 101

Tonight, I’m proud to say, I’ve returned to childhood. Let me explain.

OK, fun over, back to work.

For the last year I’ve been wanting to purchase an Arduino, for no other reason than to play with. Like a kid. My Arduino arrived two days ago. This is a story of problems, but not the pull-your-hair-out type, more the… like when you run out of gas in the middle of Sydney. Going to your Christmas dinner. In your friend’s car. Long story.

As a kid I never really learned about electronics. I took apart anything broken my parents set downstairs (and put back together the microwave I dismantled that my parent were going to give to my sister as a gift). But no electronics. I had a computer of some sort from when I was a wee kid, but not a single electronics kit, or not one I can remember.

I soon realised I forgot to order a few resistors needed for the various sensors I purchased. This morning before work I stopped by the local electronics store, and like a kid in a candy shop, ordered $1.60 worth of resistors and LEDs.

Just after coming home and eating insanely good Indian with YS, I set out to get an LED blinking. Without too many troubles, success! I graduated myself to the next grade and pulled out the temperature sensor. I looked at the ‘special’ wires I bought to connect to the sensor and the breadboard, and I soon realised I actually purchased the M->F wires instead of the assorted pack. I started looking around for cables I could cut.

Spare USB cable, no. Telephone cable, maybe. I realised I could just slice up one of the cables and see if it worked. The cables were braided instead of solid coper, which sort of made my hopes sink, but it was too late now.

I soon wired everything together as the various forum posts suggest, plugged it in to my laptop, started the Arduino software, and loaded the Dallas Temperature Library, and - “Invalid CRC”. I pulled the wires off, stripped a little more cabling off, plugged it back in, and tried again. It worked! I don’t really know if the temperature is correct or not - we don’t have any other way to measure it - but at this point, until I put in some more robust wires, I don’t really even care.

Overall, I’m happy. I’ve created two simple physical computing missions, and wrote two blog entries, in well under two hours. This has been quite a relaxing break from the other duties in life. I plan to make the little system ‘proper’ (more robust), and link it to my mini server. I’ll then create a simple chart (via Google’s API) to display on this website.
OK, fun over, back to work.

Simple Arduino + LED Tutorial

For this you’ll need: an Arduino, an LED, a USB cable, and the Arduino software.

  1. Get the LED, and plug it in to your Arduino. Plug in the long end to pin 13, plug the short end to the GND.
  2. Plug in the Arduino into your laptop. Go to Tools->Serial Port. Choose the USB port that appears.
  3. Take the code from the BlinkingLED tutorial and paste it in to the arduino like so:

  1. Hit File->Upload to I/O Board.
  2. Hit (physically) the little reset button. Voila! a blinking LED!