I've been toying around with XDebug this week, which allows me to debug php scripts interactively, by setting breakpoints, watching variables and stepping through the code from within Vim (it's a bit rudimentary, but it works). Marijn tipped me about this and I used his useful howto to set it up.

I'm still battling with python and vim to properly support spaces in filenames, I'll post my solution once I found it (I found it, see below). If you want to toy around with XDebug and Vim, you should definately check out this comment about other (non-space) special characters in urls.

Anyway, I started out this post because I made something pretty and wanted to share it. To use the XDebug extension, you need to pass XDEBUG_SESSION_START=id with the request you want to debug. Because I kept forgetting how this parameter was called exactly and because it is annoying to insert it at the end of some long url, especially when the url also uses # to jump to an anchor, I created a nifty bookmark.

var loc, tmp, anchor, url, args;

tmp = document.location.href.split('#');
if(tmp[1]) 
    anchor = "#" + tmp[1];
else 
    anchor = "";
tmp = tmp[0].split('?');
url = tmp[0];
if (tmp[1]) 
    args = "?" + tmp[1] + "&";
else 
    args = "?";
args += "XDEBUG_SESSION_START=id";
document.location.href = url + args + anchor;

This script splits the current location in url, args and anchors, adds the argument, and puts the location back together. Simple as that. You can create a bookmark from this script by right-clicking this link and pick "Bookmark this link..." or whatever your browser calls it. You can also use the link to test the script (watch the address bar).

I've tested this script on Firefox, but it should work on most browsers, for it doesn't do anything really complex.

Update: I've managed to convince the XDebug vim script to handle spaces in filenames properly. I've put up a fixed version and a diff with the fixes. Both include the above mentioned special characters fix.

Right now, I'm in the train to Enschede on my way back to Enschede. This afternoon, I've been over at Truelight's today, together with Rubidium and boekabart. We had a little get-together, to celebrate the 10,000th revision of OpenTTD, which we reached last week. It was a nice excuse to finally get together and actually see the faces behind the IRC nicks :-)

We've had a fun afternoon and evening, with lunch, dinner, a lot of chit chat and of course OpenTTD talk. We talked about stuff to do, stuff we did, and how to do it. The biggest topic was the 32 bit graphic rendering, which is a long wanted feature, which actually has seen quite some (prototype) implemenation as well.

Boekabart pointed out that the technical implementation for 32 bit rendering was not being te problem: There are plenty of alternatives for all problems encountered. The true bottleneck here is reaching concensus on which of these alternatives to choose, and decide what parts of it we want to support and what we want to drop.

We concluded that we will probably try to keep supporting the 8bit rendering path, which is useful for low end machines (embedded devices perhaps?) and historical reasons. Furthermore, though we will probably keep supporting NewGRF as well, even though that format was designed with 8 bit (and all the associated nastyness of palleting and colour remapping that doesn't work so well when rendering to 32 bit) in mind.

We made some plans for structuring renderers, blitters and video drivers, also keeping an eye on optimization options. Although personally I won't have time to actually code for the 32 bit effort, the project will have a lot more of my attention than it had before.

The best part of the day, though, was the present boekabart brought into the celebration. He had bought a big cake with the OpenTTD titlescreen printed on it. Of course we made pictures, but I'll forward you to the OpenTTD screenshots section to see it, since though it was printed on a cake, it was still a screenshot :-)

I've recently been fiddling around with my kernel, configuring most options as modules to enable faster dehibernation (which seems to work). Anyway, afterwards I could not longer mount my USB stick. For some reason, USB Mass storage support got broken. The USB device was properly recognized, but there was no SCSI device generated.

[   62.166775] usb 2-1: new full speed USB device using ohci_hcd and address 2
[   62.455087] scsi0 : SCSI emulation for USB Mass Storage devices
[   62.460703] usb-storage: device found at 2
[   62.460707] usb-storage: waiting for device to settle before scanning
[   65.047735] usb 2-1: reset full speed USB device using ohci_hcd and address 2
[   65.409629] usb 2-1: reset full speed USB device using ohci_hcd and address 2
[   65.769525] usb 2-1: reset full speed USB device using ohci_hcd and address 2
[   66.041867] usb-storage: device scan complete

To cut a long story (and debugging session) short: The problem was a kernel option "USB Selective suspend/resume and wakeup" (CONFIGUSBSUSPEND) that I enabled in the process (it sounded useful). Disabling this option made everything work again. Yay!

[  629.199144] usb 1-1: new full speed USB device using ohci_hcd and address 4
[  629.786207] Initializing USB Mass Storage driver...
[  629.787051] scsi0 : SCSI emulation for USB Mass Storage devices
[  629.789039] usb-storage: device found at 4
[  629.789043] usb-storage: waiting for device to settle before scanning
[  629.789055] usbcore: registered new driver usb-storage
[  629.789058] USB Mass Storage support registered.
[  632.291508]   Vendor: USB-DISK  Model: FreeDik-FlashUsb  Rev: 1.06
[  632.291519]   Type:   Direct-Access                      ANSI SCSI revision: 00
[  632.302503] SCSI device sda: 32736 512-byte hdwr sectors (17 MB)
[  632.305490] sda: Write Protect is off
[  632.305493] sda: Mode Sense: 0b 00 00 08
[  632.305496] sda: assuming drive cache: write through
[  632.312994] SCSI device sda: 32736 512-byte hdwr sectors (17 MB)
[  632.315988] sda: Write Protect is off
[  632.315992] sda: Mode Sense: 0b 00 00 08
[  632.315994] sda: assuming drive cache: write through
[  632.325982] SCSI device sda: 32736 512-byte hdwr sectors (17 MB)
[  632.328979] sda: Write Protect is off
[  632.328982] sda: Mode Sense: 0b 00 00 08
[  632.328984] sda: assuming drive cache: write through
[  632.329014]  sda: unknown partition table
[  632.334028] sd 0:0:0:0: Attached scsi removable disk sda
[  632.335561] usb-storage: device scan complete

This afternoon, in an attempt to make Xdmx work, I decided to dist-upgrade Kat and Xanthe, both of my Debian sid boxes. After upgrading 600-ish packages, suddenly I couldn't start any new xterms. Some investigation showed a font problem, and since a font incompatibility was the reason I started upgrading in the first place, I thought it would help to restart my X server.

That turned out to be nasty, since ion, my window manager, crashed on start up with a rather cryptic message.

>> [72] X_CreateWindow (1) 0: BadValue 
    (integer parameter out of range for operation)

Anyone with some debugging experience will concede that this is not a real pleasure to work with (actually, anyone without this experience will run away screaming seeing this...). Still, I need a window manager to work, so debugging it is...

See more ...

This afternoon on IRC people were discussing the assignments of the NIO, a programming contest for high school students I've participated in a few times. One particularly nice one was the "happy" assignment.

In short, the assignment says there are happy numbers and non happy numbers. To see if a number is happy, you take the square of every digit and add these squares together. You do this squaring and summing until you get either the number "1" of the number "4". If you get "1", the number is happy, if you get "4", the number is not. This is a fairly simple assignment to write a program for, but the program could not have more than 1000 characters, which makes it kinda interesting.

As soon as somebody posed that he could to it in four hundred something characters, without leaving out stuff (like whitespace and indentation), people started trying too. Before long, a couple of people were trying to make their programs as short as possible. Later we merged our tries to get even shorter versions. The end result was a very short solution for this problem (chopped into two lines for readability):

m[666],o,x;h(i){for(x=0;i;i/=10)x+=i%10*(i%10);m[x]=m[x]?:h(x);}
main(i){for(m[m[4]=scanf("%d",&i)]=2;i;o+=h(i--)-1);printf("%d\n",o);}

See more ...

While I was editing some wiki pages just now, I kept being annoyed by the stupid little poor-excuse-for-an-editor text-area in Firefox. But using my favourite editor, vim, would mean copypasting all the text into vim and back again, which is rather clumsy since vim runs in an xterm.

At these moments, you suddenly realise that you cannot be the only one that is being annoyed by something like this. A quick search for "vim" at the Firefox extensions page gives no useful results, though. Searching a little further, I come across Mozex, an extension that supports not only editing textareas in an external program, but also intersepts mailto links, ssh links, news links, etc.

So, whenever I wanna edit something in vim now, I just press Meta+V and get my trusty editor in an xterm :-D

Small post about a stupid problem I was having tonight with Samba. As you may know, Inter-Actief uses samba on its fileserver, using active directory for authentication.

We have enabled the idmap_ad module, by putting idmap backend = ad in smb.conf. All our users are assigned a unix uid in the active director, so they can login. This works okay for normal users, they can login, access files, etc. Yet, this fails for software deployment.

We are using standard windows software deployment techniques, using group policies in our AD. Yet, when clients try to install that software, nobody is logged in yet, so the install process runs as the machine account of the machine. But when trying to authenticate with this account at the fileserver, it can't find the unix uid for the account, so the login fails.

Solving this proved easy (though it took me half an evening to think of it): I added the map to guest = Bad Uid option to my smb.conf. This ensures that any failed uid lookup is mapped to my guest user, nobody. Since the share that provides the deployed software is accessible by guest users (guest ok = yes), this allows clients to access deployed software.

My clients are now able to access the files on the fileserver. Now I am up for the next problem, according to samba logs, they stopped trying, while the windows event viewer still says installation failed because the installation source was not available. Gr.... Stupid windows....

I've previously mentioned nested groups not working on our active directory/winbind/FreeBSD setup. For some reason groups in groups were not properly unfolded by winbind. I suspected this to be because the "domain functional level" was "windows 200 mixed", which is more compatible with older DC's. By now, we've moved to a new DC and raised the level to "windows 2003 native". Unfortunately, this didn't help one tiny bit. Still broken.

For example, the user mkooijma is present in various groups on our AD. Most of these groups are also member of the Actievelingen group, so mkooijma should be too. Yet, running id mkooijma, gives

uid=10008(mkooijma) gid=10000(Domain Users) groups=10000(Domain Users),
  11001(Beheer), 11013(KasCo), 11004(BoCie), 11019(NoiZiA), 11029(Webredactie)

So, no Actievelingen group there.

Again motivated to start digging into source code to find out why this is, I ended up finding a nice big FIXME in the code. That part of the code, which was used by the id utility on FreeBSD (and presumably also by the file system), does not support nested groups.

Seeing a fine oppurtunity to do some coding, I've hacked up a couple of lines of code to supported nested groups there. And I'll be damned, but it worked. Running id mkooijma now gives:

uid=10008(mkooijma) gid=10000(Domain Users) groups=10000(Domain Users),
    11001(Beheer), 11013(KasCo), 10001(Actievelingen), 11004(BoCie),
    11019(NoiZiA), 10002(Webmasters.nested), 11029(Webredactie)

Nice job, so I submitted my patch to the samba-technical mailing list yesterday. I also started hanging out in #samba-technical, where I got into a discussion with Wilco (which I happen to know IRL) this morning about my patch. We discussed in what way I could make it not break on groups that are members of themselves (indirectly), or users that would end up in the same group multiple times. A few minutes into this discussion, vl walked in on IRC. Since he apparently has been working on winbind group mappings before, he pointed me on a few more fundamental flaws of my approach.

It turns out that fixing the getgrent NSS interface is not really the way to go, for two reasons.

• Samba's getgrent interface is not really working that well. It does not support everything as it should, such as nested groups. Yet, the code is stable now, and trying to support nested groups, but not accounting for all possible scenarios will make the code unstable. Or more to the point:

  17:01 < blathijs> okay, so the current code doesn't work but doesn't break
  17:01 < blathijs> but my patch makes it break :-)
  

• Using getgrent for finding the groups a user is in is terribly inefficient. In short, getgrent returns a list of members for a group. So, to get the groups a user is on using getgrent the code has to iterate all existing groups and see if the user is in that particular group. Doable for a couple of groups, but it will not scale to hundreds or thousands groups and users.

  Also, since active directory and LDAP store group membership as "memberOf" attributes for the users, creating a list of members for a group probably involves iterating all users to find members (not sure about that though).

It turns out there is a getgrouplist function in FreeBSD and there is also a getgrouplist function in winbind. Yet, when calling getgrouplist, this call is not forwarded to winbind through nsswitch, but implemented by iterating with getgrent (as described above. getgrent is forwarded through nsswitch). So, FreeBSD should just call getgrouplist through nsswitch and be done with it?

Also, these features are supported on linux, so it should be possible. Wilco pointed out than NetBSD does support this and has nicely documented how. Basically, the problem is that the getgrouplist function uses a parameter is input and output paramater, which makes it unsuitable to forward using nss. NetBSD fixes this by making getgrouplist a wrapper around a (new) getgroupmembership function, which has a slightly different interface that is compatible with nss.

Yet, it seems that this getgroupmembership nss function is not implemented by winbind. This probably means NetBSD was changed to support this with other nss backends, but nobody got around to adding the function to winbind yet. But, we know that linux does support this using nss and winbind, so how is that implemented then?

It turns out that linux has implemented something similar, yet it wraps getgrouplist in a nss-compatible function initgroups_dyn. It's interface is practically compatible with getgroupmembership, the only difference is that getgroupmembership needs a preallocated buffer, while initgroups_dyn can allocate and resize the buffer itself when needed (though this seems like a better idea, the function must still be wrapped in getgrouplist, so it might not really matter anyway....).

Anyway, I now have a clear view what has to bo done. I will either:

• Port getgrouplist nss code from NetBSD to FreeBSD and implement getgroupmembership in winbind (for which I can probably borrow code from nss_ldap, which I think supports both linux and NetBSD).
• Port getgrouplist nss code from linux. This requires no winbind changes, since initgroups_dyn is already implemented in winbind.

Just now, I've found I'm not the only one that wants these changes. So I'll do some more exploring about the necessary changes soon, but now it's time for dinner.

Edit: I've looked at both Linux and NetBSD libc, and it seems NetBSD is the way to go. Both FreeBSD and NetBSD have taken their libc implementations from something called 4.4BSD-Lite2, meaning that both are already quite similar in contrast to Linux, whose libc is structured quite differently.

It has been nearly a year since the last release of OpenTTD and it has been about two months since the first time we answered the "When is the next release?" question with "Soon", but it is now finally here: OpenTTD 0.4.5.

I haven't done too much for this release, since I've been busy with all kinds of other stuff, but I'm building the Debian package right now. I'm still looking for a good way to do packaging (combining with svn and a few other tools), but since I do not get so much practice (few releases) stuff is still a little flakey. Ah well, still learning.

Years ago, when I was still in high school, I used to have a bookmarks folder called "daily". I used to open all the bookmarks in there (Userfriendly, /., a few forums, etc.) with opera's "open all" feature, getting my daily dose of news and humour. In my first year at university, I began to value my sleep more and usually had no time for this in the morning (I'd rather sleep some more).

A while back I dug up my daily folder, added some new stuff to it and removed the junk and started using it again. Also, around the same time, I decided that the hype around RSS was sorta over, so it would be safe to use it. I wanted an RSS reader inside my browser, since I have enough programs already. I am now using the Sage extension for Firefox, which works good enough. Still a few missing features, but I'll add them sometime. ;-)

Having a lot of RSS feeds in my reader also meant that most pages from my daily folder were deprecated: They offer RSS reads. So the only thing that remains are a few forums, Userfriendly (since a graphical comic is hard to put into RSS) and de wereld van Pro Deo. That last one is an aggregator for the blogs of Pro Deo members. It works by gathering RSS feeds of all blogs and putting them on one central page. The only thing it's missing, is aggregating all these blogs in a new RSS feed. So, Theo, if you're reading.... ;-)