"In het verleden behaalde resultaten bieden geen garanties voor de toekomst"

These are the ramblings of Matthijs Kooijman, concerning the software he hacks on, hobbies he has and occasionally his personal life.

Questions? Praise? Blame? Feel free to contact me.

My old blog (pre-2006) is also still available.

Sun Mon Tue Wed Thu Fri Sat

4 5 6 7 8
9 10 11 12 13 14 15
16 17 18 19 20 21 22
23 24 25 26 27 28 29
30 31
Tag Cloud
&
(With plugins: config, extensionless, hide, tagging, Markdown, macros, breadcrumbs, calendar, directorybrowse, entries_index, feedback, flavourdir, include, interpolate_fancy, listplugins, menu, pagetype, preview, seemore, storynum, storytitle, writeback_recent, moreentries)
Valid XHTML 1.0 Strict & CSS
Getting git-buildpackage to ask for my passphrase

Since a while, I've been using git-buildpackage to manage the Debian packaging for OpenTTD. This offers the option to sign the tags created by git-buildpackage, which is of course cool. However, I've found that I always need to load my GPG key into my gpg-agent first to make that work. If I didn't, I'd see the following:

You need a passphrase to unlock the secret key for
user: "Matthijs Kooijman <matthijs@stdin.nl>"
1024-bit DSA key, ID 8A2FAFBC, created 2005-05-27

gpg: cancelled by user
gpg: skipped "Matthijs Kooijman <matthijs@stdin.nl>": bad passphrase
error: gpg failed to sign the tag
error: unable to sign the tag


This also occurs when calling git tag directly, but not when calling gpg --sign.

I've finally done some research to this, and found that this can be solved by setting the GPG_TTY variable in every shell. According to the gpg-agent manual, you should add the following to your ~/.bashrc:

GPG_TTY=$(tty) export GPG_TTY  And whaddayaknow, it works! Sounded pretty silly to me (why not just use $tty directly?), but it actually makes some sense. First of all, $(tty) is not an evironment variable, but an invocation of the tty command. Secondly, it seems of course silly to put this in the environment when the passphrase-asking code could just find out the current tty by itself. However, it seems that it is actually gpg-agent itself that initiates the passphrase dialog. When I unset $GPG_AGENT_INFO in my terminal, git tag asks for a passhprase normally (but just with an "Enter passphrase:" prompt, instead of the full-screen curses-based prompt I get normally).

This probably means that whatever code is calling gpg-agent from within git tag, is sending along the value of the GPG_TTY variable (or perhaps simple all of the environment?). I guess the gpg command automatically sets the GPG_TTY variable internally, which is why using gpg directly just worked.

This also explains why, after entering my passphrase, my key was added to the agent as well.

xauth breaking X11 forwarding over SSH

This morning, I was trying to enable X forwarding, to run applications on my server (where I have GHC available) to my local workstation (where I have an X server running). The standard way to do this, is to use SSH with the -X option. However, this didn't work for me:

mkooijma@ewi1246:~> ssh -X kat
Last login: Wed May 20 13:48:13 2009 from ewi1246.ewi.utwente.nl
matthijs@katherina:~$xclock X11 connection rejected because of wrong authentication.  Running ssh with -vvv showed me another hint: debug2: X11 connection uses different authentication protocol.  It turned out this problem was caused by some weird entries in my .Xauthority file, which contains tokens to authenticate to X servers. The entries in the file can be queried with the xauth command: matthijs@katherina:~$ xauth list
#ffff##:  XDM-AUTHORIZATION-1  00000000000000000000000000000000


(I replaced the actual authentication keys with zeroes here). The last entry is the useful one. It is the proxy key added by ssh when I logged in. That is the one it should send over the ssh forwarded X connection (where ssh will replace it with the actual key, this is called authentication spoofing). However, I found that for some reason X clients were sending the XDM-AUTHORIZATION-1 key instead (hence the "different authentication protocol" message), causing the connection to fail.

I've solved the issue by removing the #ffff## entries from the .Xauthority file (but since I couldn't just run xauth remove #ffff#, I turned it around by readding only the one I wanted:

matthijs@katherina:~$rm ~/.Xauthority matthijs@katherina:~$ xauth add localhost/unix:10  MIT-MAGIC-COOKIE-1  00000000000000000000000000000000


I'm still not sure what these #ffff## entries do or mean (I suspect xdm has added them, since I am running xdm on this machine), but I've made inquiries on the xorg list.

As a last note: If you want to use X forwarding and enable the GLX protocol extensions for OpenGL rendering, you need to disable security checks in the X forwarding, by running ssh -Y instead of ssh -X.

Awesome window manager

Recently, I switched Window manager again. I still haven't found the window manager that really works for me (having run Ratpoison, Enlightenment, ion3 and wmii in the past), perhaps I should write my own. Anyway, a while back a friend of mine recommended XMonad, a Haskell based window manager. Even though having a functionally programmed window manager is cool, I couldn't quite find my way around in it (especially it's DIY statusbar approach didn't really suit me).

While googling around for half-decent statusbar setups for XMonad, I came across the Awesome window manager. It's again a tiling window manager, as are most of the ones I have been using lately, but it just made this cool impression on me (in particular the clock in the statusbar, which is by default just a counter of type time_t, ie number of seconds since the epoch. Totally unusable and I replaced it already, but cool).

Awesome allows you to tag windows, possibly with multiple tags and then show one tag at a time. (which is similar to having workspaces, but slightly more powerful). But, one other feature which looks promising, is that it can show multiple tags at once. So you can quickly peak at your IRC screen, without losing focus of your webbrowser, for example. I'm not yet quite used to this setup, so I'm not sure if it is really handy, but it looks promising. I will need to fix my problems with terminal resizes and screen for this to work first, though....

Embedded Gentoo: Small footprint linux

In the last few weeks I've been playing around with embedded Linux. Bert, a flatmate, brought two old thin clients from his work. They're bot Wyse WT8440XL devices, containing a 400Mhz K6-II processor, some peripherals, and a "DiskOnChip". This last one is flash device that is used instead of a hard disk. It's soldered directly onto the mainboard and as far as I can tell, it interfaces with the system directly into the memory controller, similar to what a BIOS chip does. Anyway, this means I have no hard disk to put a full OS on, but only 96MB of flash storage (And yes, I know I can just plug in some USB storage, which is even what I do for testing, but not using the internal flash chip wouldn't be half as much fun :-p)

See more ...

Using runas with explorer

It feels a bit weird to write about windows, but perhaps other people have some use for this.

At Inter-Actief, we normally log in with non-privileged accounts. Every now and then, we (the sysadmins) want to do something with privileges. For this, we use the runas utility, which is included with Windows XP by default. It runs a given command with different privileges (after entering a password, of course).

One of these things is browsing the file system and copying files. In other words, you want to get an explorer window with privileges. Simply running

runas.exe /user:Administrator explorer


Doesn't work. This will notice there is already a running explorer process and let that existing process open a new window instead (which will not have privileges).

Previously, we misused Internet Explorer for accomplishing this. Internet Explorer is really just another face of explorer (But with a different process name), so if you would start a new instance of Internet Explorer and navigate to "C:" instead of an url, you'd get a perfectly usable explorer, running as Administrator.

For example,

runas.exe /user:Administrator "C:\Program Files\Internet Explorer\iexplore.exe C:\"


gets you a fancy new explorer window, but only when using IE6. Internet Explorer 7 dropped all those explorer-like features (which is a good thing, btw), but now this trick doesn't work anymore.

Some googling around gave me Aaron Margosis' blog who describes some trick with setting a "run explorer windows in separate processes"-flag for the Administrator user.

This is all nice and fancy, but I won't be logging into every client here to set that bit, only to do it again (or more annoying, forget it) whenever we reinstall a machine.

Fortunately, badri commented another trick: Use the /separate flag to explorer. This forces explorer to run as a seperate process, ignoring any already running instances. This gives a command line such as:

runas.exe /user:Administrator "explorer.exe /separate,C:\"


Be careful to include the "," there, between /separate and the path. I'm not sure why it is there, but it doesn't work without.

This ended up as bit longer post than I intended, but at least I'm writing again.

Copying references from one font to another in FontForge

For the live roleplaying game Symbols, I've been learning a bit about fonts and how to modify them. In the game, we use different fonts to represent different languages. These fonts are not "normal" fonts, but fonts with symbols instead of letters. This means that a text in such a font is not readable, unless you have the corresponding key that maps each symbols back to a letter.

For this, a number of fonts was found on the Internet. These fonts had a symbol for each letter, but had an important shortcoming: They had no accents. So when a text contained, for example, an é it would be displayed as an empty box, or not at all. This was previously solved by replacing all accented letters with their normal equivalents. This was a manual process, which makes it time consuming and error-prone.

I decided to explore FontForge, an open source font editing program to add accents to these fonts. The program turned out to be well-suited to the task.

I first, manually, made all accented characters references to their unaccented equivalents (instead of empty glyphs). Then I created a grave, acute, dieresis and circumflex (yay for my Wacom tablet). Then, I copied references to these into all the accented characters (all the other accented characters simply stayed unaccented). After some manual positioning, the new font was done.

Now, for the actual topic of this post. I now had one font that had the proper references to have all accented characters. I planned on copying a big part of the FontForge .sfd file from that font to all the others, so they would automatically define all accented characters as well.

But, it turned out that FontForge uses the glyph number (based on the order in which the glyphs are in the file) instead of the proper (Unicode) character name/number to save references. Since these glyph numbers were not consistent among the fonts I had to modify, this got completely messed up.

So, I tried the scripting approach. Fortunately, FontForge is easily scriptable, in python as well as in some built in language. I opted for the latter and experimented with scripting. By doing something like the following, I could create the â automatically (I had to apply the fix in this post before CopyReference() would work as expected).

Select('a')
CopyReference()
Select('acircumflex')
Paste();
Select('circumflex')
CopyReference()
Select('acircumflex')
PasteInto();


But, I didn't feel like writing hundreds of these lines manually, so I also wrote a Perl script to extract all references from an existing font file (the one I created manually before). In hindsight, it might have been faster to have done it manually, but now I have this pretty script:

#!/usr/bin/perl

# Usage: pipe in a sfd through stdin.
# ./parse.pl < Guardion/Guardion.sfd > fix.ff

while(<STDIN>) {
if ( /^StartChar: (.*)$/ ) {$code = $1; push(@bases,$code);
}
if ( /^Refer: ([0-9]*)/ ) {
push(@{$uses{$1}}, $code); } } while (($key, $value) = each(%uses)) { foreach$other (@{$value}) { print("Select('$other')\n");
print("Clear()\n");
}
}

while (($key,$value) = each(%uses)) {
print("Select('$bases[$key]')\n");
print("CopyReference()\n");
print("SelectNone()\n");
foreach $other (@{$value}) {
print("SelectMore('$other')\n"); } print("PasteInto()\n"); }  You simply give it an sfd file to parse and it outputs a script that will reproduce all the references from a given font in another font (using File -> Execute Script...). It does not do any position of referenced glyphs, so I still needed to do the positioning of accents manually 0 comments -:- permalink -:- 18:00 Defunct subversion post-commit hook I'm using subversion to manage code for Brevidius, the company I work for. In the code, I use doxygen to document the code. Because I do not care to regenerate the documentation after every change I make, I use subversion's post-commit hook to regenerate the documentation on every commit. Because the generation of documentation takes long, I added a & to the call to doxygen-post-commit-hook in my post-commit file. This was supposed to put the doxygen generation in the background, so my svn client wouldn't take so long for every commit. This didn't work at all. The command was properly backgrounded (any commands I put after the call to doxygen-post-commit-hook were executed quickly), but ps aux --forest showed that although doxygen-post-commit-hook was indeed no longer a child process of post-commit, post-commit was now a defunct process. It turns out that just backgrounding the command is not enough, one should also redirect inputs and outputs (perhaps not all three, but that works at least). So, calling doxygen-post-commit-hook is now: /home/data/svn-hook-scripts/doxygen-post-commit-hook "$REPOS" "$REV" "classbased cms/Doxyfile" "/var/www/doxygen/Brevidius" < /dev/null > /dev/null 2> /dev/null &  This issue is also referenced in an subversion mailing list, though not directly. For those who are interested, here is the doxygen-post-commit-hook too. #!/bin/sh # Arguments are REPOS, REV, DOXYFILE, OUTPUT_DIR, in that order. DOXYFILE is # the path inside the repository (no leading /). REPOS=$1
REV=$2 DOXYFILE=$3
OUTPUT_DIR=$4 TMPDIR=/tmp/doxygen EMPTYFILE=$TMPDIR/empty

svn export file://$REPOS -r$REV $TMPDIR echo >$EMPTYFILE

cd "dirname "$TMPDIR/$DOXYFILE""

echo -e \
"CASE_SENSE_NAMES=YES\nOUTPUT_DIRECTORY=$OUTPUT_DIR\nHTML_HEADER=$EMPTYFILE\nHTML_FOOTER=$EMPTYFILE" \ | cat basename "$DOXYFILE" - | doxygen -

rm -r \$TMPDIR


XDebug javascript snippet for Firefox (and whitespace fix for vim)

I've been toying around with XDebug this week, which allows me to debug php scripts interactively, by setting breakpoints, watching variables and stepping through the code from within Vim (it's a bit rudimentary, but it works). Marijn tipped me about this and I used his useful howto to set it up.

I'm still battling with python and vim to properly support spaces in filenames, I'll post my solution once I found it (I found it, see below). If you want to toy around with XDebug and Vim, you should definately check out this comment about other (non-space) special characters in urls.

Anyway, I started out this post because I made something pretty and wanted to share it. To use the XDebug extension, you need to pass XDEBUG_SESSION_START=id with the request you want to debug. Because I kept forgetting how this parameter was called exactly and because it is annoying to insert it at the end of some long url, especially when the url also uses # to jump to an anchor, I created a nifty bookmark.

var loc, tmp, anchor, url, args;

tmp = document.location.href.split('#');
if(tmp[1])
anchor = "#" + tmp[1];
else
anchor = "";
tmp = tmp[0].split('?');
url = tmp[0];
if (tmp[1])
args = "?" + tmp[1] + "&";
else
args = "?";
args += "XDEBUG_SESSION_START=id";
document.location.href = url + args + anchor;


This script splits the current location in url, args and anchors, adds the argument, and puts the location back together. Simple as that. You can create a bookmark from this script by right-clicking this link and pick "Bookmark this link..." or whatever your browser calls it. You can also use the link to test the script (watch the address bar).

I've tested this script on Firefox, but it should work on most browsers, for it doesn't do anything really complex.

Update: I've managed to convince the XDebug vim script to handle spaces in filenames properly. I've put up a fixed version and a diff with the fixes. Both include the above mentioned special characters fix.

OpenTTD r10000 party

Right now, I'm in the train to Enschede on my way back to Enschede. This afternoon, I've been over at Truelight's today, together with Rubidium and boekabart. We had a little get-together, to celebrate the 10,000th revision of OpenTTD, which we reached last week. It was a nice excuse to finally get together and actually see the faces behind the IRC nicks :-)

We've had a fun afternoon and evening, with lunch, dinner, a lot of chit chat and of course OpenTTD talk. We talked about stuff to do, stuff we did, and how to do it. The biggest topic was the 32 bit graphic rendering, which is a long wanted feature, which actually has seen quite some (prototype) implemenation as well.

Boekabart pointed out that the technical implementation for 32 bit rendering was not being te problem: There are plenty of alternatives for all problems encountered. The true bottleneck here is reaching concensus on which of these alternatives to choose, and decide what parts of it we want to support and what we want to drop.

We concluded that we will probably try to keep supporting the 8bit rendering path, which is useful for low end machines (embedded devices perhaps?) and historical reasons. Furthermore, though we will probably keep supporting NewGRF as well, even though that format was designed with 8 bit (and all the associated nastyness of palleting and colour remapping that doesn't work so well when rendering to 32 bit) in mind.

We made some plans for structuring renderers, blitters and video drivers, also keeping an eye on optimization options. Although personally I won't have time to actually code for the 32 bit effort, the project will have a lot more of my attention than it had before.

The best part of the day, though, was the present boekabart brought into the celebration. He had bought a big cake with the OpenTTD titlescreen printed on it. Of course we made pictures, but I'll forward you to the OpenTTD screenshots section to see it, since though it was printed on a cake, it was still a screenshot :-)

USB Suspend breaking USB Mass storage

I've recently been fiddling around with my kernel, configuring most options as modules to enable faster dehibernation (which seems to work). Anyway, afterwards I could not longer mount my USB stick. For some reason, USB Mass storage support got broken. The USB device was properly recognized, but there was no SCSI device generated.

[   62.166775] usb 2-1: new full speed USB device using ohci_hcd and address 2
[   62.455087] scsi0 : SCSI emulation for USB Mass Storage devices
[   62.460703] usb-storage: device found at 2
[   62.460707] usb-storage: waiting for device to settle before scanning
[   65.047735] usb 2-1: reset full speed USB device using ohci_hcd and address 2
[   65.409629] usb 2-1: reset full speed USB device using ohci_hcd and address 2
[   65.769525] usb 2-1: reset full speed USB device using ohci_hcd and address 2
[   66.041867] usb-storage: device scan complete


To cut a long story (and debugging session) short: The problem was a kernel option "USB Selective suspend/resume and wakeup" (CONFIGUSBSUSPEND) that I enabled in the process (it sounded useful). Disabling this option made everything work again. Yay!

[  629.199144] usb 1-1: new full speed USB device using ohci_hcd and address 4
[  629.786207] Initializing USB Mass Storage driver...
[  629.787051] scsi0 : SCSI emulation for USB Mass Storage devices
[  629.789039] usb-storage: device found at 4
[  629.789043] usb-storage: waiting for device to settle before scanning
[  629.789055] usbcore: registered new driver usb-storage
[  629.789058] USB Mass Storage support registered.
[  632.291508]   Vendor: USB-DISK  Model: FreeDik-FlashUsb  Rev: 1.06
[  632.291519]   Type:   Direct-Access                      ANSI SCSI revision: 00
[  632.302503] SCSI device sda: 32736 512-byte hdwr sectors (17 MB)
[  632.305490] sda: Write Protect is off
[  632.305493] sda: Mode Sense: 0b 00 00 08
[  632.305496] sda: assuming drive cache: write through
[  632.312994] SCSI device sda: 32736 512-byte hdwr sectors (17 MB)
[  632.315988] sda: Write Protect is off
[  632.315992] sda: Mode Sense: 0b 00 00 08
[  632.315994] sda: assuming drive cache: write through
[  632.325982] SCSI device sda: 32736 512-byte hdwr sectors (17 MB)
[  632.328979] sda: Write Protect is off
[  632.328982] sda: Mode Sense: 0b 00 00 08
[  632.328984] sda: assuming drive cache: write through
[  632.329014]  sda: unknown partition table
[  632.334028] sd 0:0:0:0: Attached scsi removable disk sda
[  632.335561] usb-storage: device scan complete