In my work as a Debian Maintainer for the OpenTTD and related packages, I occasionally come across platform-specific problems. That is, compiling and running OpenTTD works fine on my own x86 and amd64 systems, but when I my packages to Debian, it turns out there is some problem that only occurs on more obscure platforms like MIPS, S390 or GNU Hurd.

This morning, I saw that my new grfcodec package is not working on a bunch of architectures (it seems all of the failing architectures are big endian). To find out what's wrong, I'll need to have a machine running one of those architectures so I can debug.

In the past, I've requested access to Debian's "porter" machines, which are intended for these kinds of things. But that's always a hassle, which requires other people's time to set up, so I'm using QEMU to set up a virtual machine running the MIPS architecture now.

What follows is essentially an update for this excellent tutorial about running Debian Etch on QEMU/MIPS(EL) by Aurélien Jarno I found. It's probably best to read that tutorial as well, I'll only give the short version, updated for Squeeze. I've also looked at this tutorial on running Squeeze on QEMU/PowerPC by Uwe Hermann.

Finally, note that Aurélien also has pre-built images available for download, for a whole bunch of platforms, including Squeeze on MIPS. I only noticed this after writing this tutorial, might have saved me a bunch of work ;-p

Preparations

You'll need qemu. The version in Debian Squeeze is sufficient, so just install the qemu package:

$ aptitude install qemu

You'll need a virtual disk to install Debian Squeeze on:

$ qemu-img create -f qcow2 debian_mips.qcow2 2G

You'll need a debian-installer kernel and initrd to boot from:

$ wget http://ftp.de.debian.org/debian/dists/squeeze/main/installer-mips/current/images/malta/netboot/initrd.gz
$ wget http://ftp.de.debian.org/debian/dists/squeeze/main/installer-mips/current/images/malta/netboot/vmlinux-2.6.32-5-4kc-malta

Note that in Aurélien's tutorial, he used a "qemu" flavoured installer. It seems this is not longer available in Squeeze, just a few others (malta, r4k-ip22, r5k-ip32, sb1-bcm91250a). I just picked the first one and apparently that one works on QEMU.

Also, note that [Uwe's PowerPC tutorial] suggests downloading a iso cd image and booting from that. I tried that, but QEMU has no BIOS available for MIPS, so this approach didn't work. Instead, you should tell QEMU about the kernel and initrd and let it load them directly.

Booting the installer

You just run QEMU, pointing it at the installer kernel and initrd and passing some extra kernel options to keep it in text mode:

$ qemu-system-mips -hda debian_mips.qcow2 -kernel vmlinux-2.6.32-5-4kc-malta -initrd initrd.gz -append "root=/dev/ram console=ttyS0" -nographic

Now, you get a Debian installer, which you should complete normally.

As Aurélien also noted, you can ignore the error about a missing boot loader, since QEMU will be directly loading the kernel anyway.

After installation is completed and the virtual system is rebooting, terminate QEMU:

$  killall qemu-system-mips

(I haven't found another way of terminating a -nographic QEMU...)

Booting the system

Booting the system is very similar to booting the installer, but we leave out the initrd and point the kernel to the real root filesystem instead.

Note that this boots using the installer kernel. If you later upgrade the kernel inside the system, you'll need to copy the kernel out from /boot in the virtual system into the host system and use that to boot. QEMU will not look inside the virtual disk for a kernel to boot automagically.

$ qemu-system-mips -hda debian_mips.qcow2 -kernel vmlinux-2.6.32-5-4kc-malta -append "root=/dev/sda1 console=ttyS0" -nographic

More features

Be sure to check Aurélien's tutorial for some more features, options and details.

I've been configuring my new laptop (more on that later) and this time I've tried to get the volume hotkeys working properly with Pulseaudio. On a default Debian Squeeze installation, the volume hotkeys are processed by (the media-keys plugin of) gnome-settings-daemon (1). The good news is that Gnome has switched over to using pulseaudio by default (and even removed support for plain ALSA). However, Debian does not want to force users to use pulseaudio. So the bad news is that Debian has disabled this pulseaudio support in gnome-settings-daemon and has a patch to use the ALSA mixer (via GStreamer).

Normally, it shouldn't matter much which mixer you use, as long as they work. However, I'm using two different sound cards on my laptop: The builtin one for on the road and an external USB sound card when I'm at home (to get a S/PDIF output). So I need pulseaudio to route my audio to the right place, and I want my volume controls to control the same card as well. Note that gnome-volume-control, the GUI to control your volums is installed in two flavours by Debian (Pulseaudio and GStreamer), and the right one is started by a wrapper script depending on whether Pulse is running.

Fortunately, the Debian patch is somewhat configurable: You can select a different mixer device through gconf. To get at that configuration, use gconf-editor and browse to /desktop/gnome/sound/default_mixer_device. Set this value in the form of "element:device", where element selects the gstreamer plugin to use, and device sets its "device" property. I initially tried using the "pulsemixer" element (in the form "pulsemixer:alsa_output.usb-0ccd_USB_Audio-00-Aureon51MkII.analog-stereo"), but that only allowed me to specify a specific Pulseaudio sink, not "whatever-is-default").

So, instead, I settled for using the "alsamixer" gstreamer plugin, together with the Pulseaudio ALSA plugin (the same one you use to redirect ALSA applications to Pulseaudio). For this to work, it's important that you redirect ALSA applications to pulse using the following in your /etc/asound.conf or your ~/.asoundrc:

pcm.!default.type pulse
ctl.!default.type pulse

This makes sure that not just audio streams (pcm) but also mixer controls (ctl) are redirected to Pulseaudio. Now, set the /desktop/gnome/sound/default_mixer_device gconf value to the following:

alsamixer:default

This should make sure that your volume keys work with the device selected as default in Pulseaudio (through pavucontrol or gnome-volume-control for example). It seems this behaviour relies on the fact that gnome-settings-daemon only keeps the mixer controls open for a few seconds, allowing the Pulseaudio ALSA plugin to select the right pulseaudio sink to control everytime the mixer is reopened (so it needs a few seconds of not pressing the volume hotkeys after changing the default device).

By the way, it seems that in the next version of Gnome (and/or Debian) this problem wil probably be fixed out of the box, since the 2.93 packages in Debian experimental have Pulseaudio support enabled (haven't tested them, though).

Hopefully this helps someone else out there struggling with the same problem...

(1): You might have noticed that I'm talking about Gnome here. I case you wondered, I've actually started to use parts of Gnome for daily use on my laptop. I'm still using Awesome as my primary window manager and I'm not using gnome-panel, so I haven't suddenly become a GUI addict all of the sudden ;-)

When you use Screen together with Xorg, you'll recognize this: You log in to an X session, start screen and use the terminals within screen to start programs every now and then. Everything works fine so far. Then, you logout and log in again (or X crashes, or whatever). You happily re-attach the still running screen, which allows you to continue whatever you were doing.

But now, whenever you want to start a GUI program, things get wonky. You'll get errors about not being able to find configuration data, connect to gconf or DBUS, or your programs will not start at all, with the ever-informative error message "No protocol specified". You'll also recognize your ssh-agent and gpg-agent to stop working within the screen session...

What is happening here, is that all those programs are using "environment variables" to communicate. In particular, when you log in, various daemons get started (like the DBUS daemon and your ssh-agent). To allow other programs to connect to these daemons, they put their contact info in an environment variable in the login process. Whenever a process starts another process, these environment variables get transfered from the parent process to the child process. Sine these environment variables are set in the X sesssion startup process, which starts everything else, all programs should have access to them.

However, you'll notice that, after logging in a second time, the screen you re-attach to was not started by the current X session. So that means its environment variables still point to the old (no longer runnig) daemons from the previous X session. This includes any shells already running in the screen as well as new shells started within the screen (since the latter inherit the environment variables from the screen process itself).

To fix this, we would like to somehow update the environment of all processes that are already running when we login, to update them with the addresses of the new daemons. Unfortunately, we can't change the environment of other processes (unless we resort to scary stuff like using gdb or poking around in /dev/mem...). So, we'll have to convice those shells to actually update their own environments.

So, this solution has two parts: First, after login, saving the relevant variables from the environment into a file. Then, we'll need to get our shell to load those variables.

The first part is fairly easy: Just run a script after login that writes out the values to a file. I have a script called ~/bin/save-env to do exactly that. It looks like this (full version here):

#!/bin/sh

# Save a bunch of environment variables. This script should be run just
# after login. The saved variables can then be sourced by every bash
# shell, so long running shells (e.g., in screen) or incoming SSH shells
# can also use these services.

# Save the DBUS sessions address on each login
if [ -n "$DBUS_SESSION_BUS_ADDRESS" ]; then
echo export DBUS_SESSION_BUS_ADDRESS="$DBUS_SESSION_BUS_ADDRESS" > ~/.env.d/dbus
fi

if [ -n "$SSH_AUTH_SOCK" ]; then
echo export SSH_AGENT_PID="$SSH_AGENT_PID" > ~/.env.d/ssh
echo export SSH_AUTH_SOCK="$SSH_AUTH_SOCK" >> ~/.env.d/ssh
fi

# Save other variables here

This script fills the directory ~/.env.d with files containg environment variables, separated by application. I could probably have thrown them all into a single file, but it seemed like a good idea to separate them. Anyway, these files are created in such a way that they can be sourced by a running shell to get the new files.

If you download and install this script, don't forget to make it executable and create the ~/.env.d directory. You'll need to make sure it gets run as late as possible after login. I'm running a (stripped down) Gnome session, so I used gnome-session-properties to add it to my list of startup applications. You might call this script from your .xession, KDE's startup program list, or whatever.

For the second part, we need to set our saved variables in all of our shells. This sounds easy, just run for f in ~/.env.d/*; do source "$f"; done in every shell (Don't be tempted to do source ~/.env.d/*, since that sources just the first file with the other files as arguments!). But, of course we don't want to do this manually, but let every shell do it automatically.

For this, we'll use a tool completely unintended, but suitable enough for this job: $PROMPT_COMMAND. Whenever Bash is about to display a prompt, it evals whatever is in the variable $PROMPT_COMMAND. So it ends up evaluating that command all the time, which makes it a prefect place to load the saved variables. By setting the $PROMPT_COMMAND variable in your ~/.bashrc variable, it will become enabled in every shell you start (except for login shells, so you might want to source ~/.bashrc from your ~/.bash_profile):

# Source some variables at every prompt. This is to make stuff like
# ssh agent, dbus, etc. working in long-running shells (e.g., inside
# screen).
PROMPT_COMMAND='for f in ~/.env.d/*; do source "$f"; done'

You might need to be careful where to place this line, in case PROMPT_COMMAND already has some other value, like is default on Debian for example. Here's my full .bashrc file, note the += and starting ; in the second assignment of $PROMPT_COMMAND.

The astute reader will have noticed that this will only work for existing shells when a prompt is displayed, meaning you might need to just press enter at an existing prompt (to force a new one) after logging in the second time to get the values loaded. But that's a small enough burden, right?

So, with these two components, you'll be able to optimally use your long-running screen sessions, even when your X sessions are not so stable ;-)

Additionally, this stuff also allows you to use your faithful daemons when you SSH into the machine. I use this so I can start GUI programs from another machine (in particular, to open up attachments from my email client which runs on a server somewhere). See my recent blogpost about setting that up. However, since running a command through SSH non-interactively never shows a prompt and thus never evaluates $PROMPT_COMMAND, you'll need to manually source the variables at once in your .bashrc directly. I do this at the top of my ~/.bashrc.

Man, I need to learn how to writer shorter posts...

Last week, my Master's thesis was awarded with the ENIAC Thesis Award 2010. This is is an award for the best-written and most scientific Master's thesis for Computer Science (and related masters). The prize is awarded by ENIAC, the alumni club for computer science.

This award came quite unexpected to me. Apparently my thesis (together with Christiaan's thesis, which discussed the same subject) was nominated by our graduation comittee. I had also not actually expected to win, so that was quite cool.

The award consisted of a nice certificate and a cool sculpture by Lei Hannen. As an extra surprise, it turned out there is also a €500 money prize associated wit this award.

According to the jury rapport, our thesis were awarded because of the novelty and the new research area they open up. Additionally, my thesis was commended for the excellent use of English. Cool :-D

After nearly five years of intensive use, the time has finally come to replace my trusty S270 laptop, Xanthe. Overall, I'm surprised that she held out as long as she did. There has been some defects along the way, but in the end I've managed to keep things pretty much working (where the most recent and most serious fix was to remove the laptop speakers and soldering the inverter cable, which got partly severed in the screen hinge).

However, since a few months, there is one issue that I can't seem to fix: The AC adapter plug doesn't seem to properly connect to the socket, making the battery switch between charging and not charging all the time (once per second or so). This had a great number of side effects, like keys on my keyboard getting stuck or making the keyboard stop working completely, making my network connection break, making my screen brightness change eratically and other stuff that makes me utterly and completely annoyed. I've made a detailed post on the s270-linux mailing list about that, if you're interested.

So, after a few months of on-and-off annoyance, I've finally decided to get a new laptop. It's sucky, since apart from this issue, Xanthe is still serving me well. On the other hand, it's utterly cool, since I just ordered myself a Thinkpad X201. It's a very durable and efficient 12" notebook (over 10 hours of battery life, according to Lenovo!). More details when I get the package (should be somewhere this week!).

For a fair amount of years now, I've been using Mutt as my primary email client. It's a very nice text-based email client that is permanently running on my server (named drsnuggles). This allows me to connect to my server from anywhere, connect to the running Screen and always get exactly the same, highly customized, mail interface (some people will say that their webmail interfaces will allow for exactly the same, but in my experience webmail is always clumsy and slow compared to a decent, well-customized text-based client when processing a couple of hundreds of emails per day).

Attachment troubles

So I like my mutt / screen setup. However, there has been one particular issue that didn't work quite as efficient: attachments. Whenever I wanted to open an email attachment, I needed to save the attachment within mutt to some place that was shared through HTTP, make the file world-readable (mutt insists on not making your saved attachments world-readable), browse to some url on the local machine and open the attachment. Not quite efficient at all.

Yesterday evening I was finally fed up with all this stuff and decided to hack up a fix. It took a bit of fiddling to get it right (and I had nearly started to spend the day coding a patch for mutt when the folks in #mutt pointed out an easier, albeit less elegant "solution"), but it works now: I can select an attachment in mutt, press "x" and it gets opened on my laptop. Coolness.

How does it work?

Just in case anyone else is interested in this solution, I'll document how it works. The big picture is as follows: When I press "x", a mutt macro is invoked that copies the attachment to my laptop and opens the attachment there. There's a bunch of different steps involved here, which I'll detail below.

See more ...

Only recently, Adobe has started to (finally) support 64 bit Linux with its Flash plugin. I could finally watch Youtube movies (and more importantly, do some Flash development work for Brevidius).

However, this month Adobe has announced that it drops support for 64 bit Linux again. Apparently they "are making significant architectural changes to the 64-bit Linux Flash Player and additional security enhancements" and they can't do that while keeping the old architecture around for stable releases, apparently.

This is particularly nasty, because the latest 10.0 version (which still has amd64 support) has a couple of dozens (!) of security vulnerabilities which are fixed in a 10.1 version only (which does not have Linux amd64 support anymore).

So Adobe is effectively encouraging people on amd64 Linux to either not use their product, or use a version with critical security flaws. Right.

Recently I've been fixing up some critical defects in my MSI S270 notebook: My display stopped working due to a broken inverter cable and my power adapter switched on and off all the time due to a broken cable as well.

The inverter cable goes from the laptop mainboard to the inverter, which is below the display. On its way it passes the left hinge, which is exactly where it was broken. Apparently I'm not the only one having these kinds of problems with the S270. Some people suggest that the hinges are badly made causing these problems.

I've looked closely at the inverter cable and it even seems the cable breaks are caused by a tiny thread that is wrapped around the inverter cable to keep it together. Three of the six (IIRC) wires has broken and a few more had cuts in the insulation. The breaks were very clean and seemed to be exactly at the spot where the wrapping thread had cut into the insulation...

Anyway, I managed to solder the tiny wires back together again (using some extra wire to fill the gap). The cable did become bigger from this, so I had to remove the speakers and their cable to make a bit room in the hinge to fit the soldered cable in.

It has been working for a few weeks now, but I'm afraid the solder connection will break eventually again. In the meanwhile I've been trying to get a new cable, but it's pretty impossible to get at and MSI Netherlands refuses to sell me a cable. They say the want to use all cables they have for repairs, meaning I have to send my entire laptop to them and pay €80 - €150 for a repair I could have done myself (the part alone is usually €20 - €30...).

I'm still happy with the laptop, while it is working, but I won't be buying another MSI when this one finally breaks down for good...

My power adapter cable had also broken, switching on and off when I wiggled with the cable. The cable used to have two round "blocks" at the start and the end (ferrite rings to prevent interference). Over a year ago the cable broke near one, now it broke near the other one. I've removed both of them now and soldered the stuff back together, making it work again.

Recently, I've been setting up awstats, a webserver log analyzer, to parse my Lighttpd logs. When I'm done, I might post some details on my setup and the glue scripts used, but for now, I just want to comment on the right LogFormat configuration value to use for lighttpd.

When googling around, a lot of people either not mention LogFormat at all, or suggest to use LogFormat=1, which means the Combined Log Format (CLF). However, lighttpd use a different log format!

In fact, the CLF is very similar to Lighttpd's log format, but it differs in the second field. In CLF, the second field is the identd username, which is ignored by awstats. In Lighttpd's format, this is the virtual host of the current request, which is very relevant if you're logging multiple virtual hosts to the same logfile. This similarity is th reason that LogFormat=1 does work for most people, but it's better to use the proper configuration:

LogFormat="%host %virtualname %logname %time1 %methodurl %code %bytesd %refererquot %uaquot"

I've taken this format string from the only correct posting I found online, but the forum of that posting seems to interpret the %ua in the last field as a newline (probably u for unicode and a for 0x0a, which is the ASCII code for a newline...), so it took me a while to realize that it was correct.

I've recently been hacking a bit with Flash (or rather, with Adobe's Flex compiler. This is a freely available commandline compiler, which actually works on Linux as well.

However, out of the box the commandline compiler, mxmlc, is obnoxiously slow. It takes nearly 10 seconds to compile a simple Hello, world! example, and compiling the video player I'm working on takes over 30 seconds. Not quite productive.

This is a documented "flaw" in the mxmlc compiler, caused by the fact that it has to start up a big java program everytime, loading thousands of classes and because it always recompiles the entire source.

The official solution to get faster compile times is to use the Flex Compiler Shell (fcsh), which is included with the Flex SDK. Basically, it's a caching version of the mxmlc compiler, that keeps running in the background and caches compiled files.

fcsh is intended to be used by IDEs. The IDE starts fcsh and then communicates with it through its stdin/stdout. This means fcsh is really a simple thing, without any support for listing on sockets or properly daemonizing.

Using fcsh speeds up the build time from nearly 40 seconds to just a few seconds (depending on how many changes were made). The first compilation run is still slow, of course.

I've been trying to use fcsh with the ant build system, which makes things a bit tricky. Since there is no long-running process (like an IDE) which can keep fcsh running, this needs some way for fcsh to be run in the background, connected to some fifo or network socket so we can start it on the first compilation run and then connect to it in subsequent compilation runs.

A quick google shows that there are already a few fcsh wrappers to do this, some of which intended to work with ant as well. At a quick glance, it seems that the flex-shell-script-daemonizer seems the most useful one. It is created in Java and runs as a daemon (unlike the alternatives I saw, which were Windows-only due to some useless GUI). It has two modes: In server mode it starts fcsh and connects it to a TCP network socket and in client mode it takes a command to run and passes it over the netowrk to the running server.

There is also some stuff to integrate make ant use fcsh for compiling actionscript files. However, this requires changing the build methods and stuff in build.xml, which I don't want to do (I'm trying to minimize my changes to the sources, since I'm modifiying someone else's project).

So, I created a small shell script that can be used as a drop in replacement for mxmlc. It simply takes joins together all its arguments to a single command and passes that to the flex-shell-script-daemonize command. If the fcsh daemon is not yet running, it automatically starts it (and does some half-baked daemonization, since neither fcsh nor flex-shell-script-daemonizer does that...)

While writing the script, I also found out that the fcsh "shell", doesn't have any way to quote spaces in arguments in the command (at least, I couldn't find any and there was no documentation about it). This means that, AFAICS, there is no way to support spaces in paths. How braindead is that... I guess FlexBuilder, the official IDE from Adobe doesn't use fcsh after all and instead just includes the compiler classes directly...

Of course, just as I finished the script, I encountered flexcompile, which basically does the same thing (and includes the network features of flex-shell-script-daemonizer as well). It's written in python. However, it does require the path to flex and the mxmlc part of the path to be passed in as arguments, so it's not a 100% drop-in replacement (which I needed due to the way the build system I was using was defined). Perhaps it might be useful to you.

Anyway, here's my script. Point the JAR variable at the jar generated by flex-shell-script-daemonizer and FCSH to the fcsh binary included with the Flex SDK.

#!/bin/sh

# (Another) wrapper around FlexShellScript.jar to provide a mxmlc-like
# interface (so we can just replace the call to mxmlc with this script).

# For this, we'll have to call FlexShellScript.jar with the "client"
# argument, and then the entire mxmlc command as a single argument
# (whereas we will be called with a bunch of separate arguments)

JAR=/home/matthijs/docs/src/upstream/fcsh-daemonizer/FlexShellScript.jar
FCSH=/usr/local/flex/bin/fcsh
PORT=53000

# Check if fcsh is running
if ! nc -z localhost "$PORT"; then
    echo "fcsh not running, trying to start..."
    # fcsh not running yet? Start it (and do some poor-man's
    # daemonizing, since the jar doesn't do that..)
    java -jar "$JAR" server "$FCSH" "$PORT" > /dev/null 2> /dev/null <
/dev/null &

    # Wait for the server to have started
    while ! nc -z localhost "$PORT"; do
    echo "Waiting for fcsh to start..."
    sleep 1
    done
fi

# Run the client. Note that this does no quoting of parameters, since I
# can't find any documentation that fcsh actually supports that. Seems
# like spaces in path just won't work...
# We use $* instead of $@ for building the command, since that will
# expand to a single word, instead of multiple words.
java -jar "$JAR" client "mxmlc $*" "$PORT"