Small post about a stupid problem I was having tonight with Samba. As you may know, Inter-Actief uses samba on its fileserver, using active directory for authentication.

We have enabled the idmap_ad module, by putting idmap backend = ad in smb.conf. All our users are assigned a unix uid in the active director, so they can login. This works okay for normal users, they can login, access files, etc. Yet, this fails for software deployment.

We are using standard windows software deployment techniques, using group policies in our AD. Yet, when clients try to install that software, nobody is logged in yet, so the install process runs as the machine account of the machine. But when trying to authenticate with this account at the fileserver, it can't find the unix uid for the account, so the login fails.

Solving this proved easy (though it took me half an evening to think of it): I added the map to guest = Bad Uid option to my smb.conf. This ensures that any failed uid lookup is mapped to my guest user, nobody. Since the share that provides the deployed software is accessible by guest users (guest ok = yes), this allows clients to access deployed software.

My clients are now able to access the files on the fileserver. Now I am up for the next problem, according to samba logs, they stopped trying, while the windows event viewer still says installation failed because the installation source was not available. Gr.... Stupid windows....

I've previously mentioned nested groups not working on our active directory/winbind/FreeBSD setup. For some reason groups in groups were not properly unfolded by winbind. I suspected this to be because the "domain functional level" was "windows 200 mixed", which is more compatible with older DC's. By now, we've moved to a new DC and raised the level to "windows 2003 native". Unfortunately, this didn't help one tiny bit. Still broken.

For example, the user mkooijma is present in various groups on our AD. Most of these groups are also member of the Actievelingen group, so mkooijma should be too. Yet, running id mkooijma, gives

uid=10008(mkooijma) gid=10000(Domain Users) groups=10000(Domain Users),
  11001(Beheer), 11013(KasCo), 11004(BoCie), 11019(NoiZiA), 11029(Webredactie)

So, no Actievelingen group there.

Again motivated to start digging into source code to find out why this is, I ended up finding a nice big FIXME in the code. That part of the code, which was used by the id utility on FreeBSD (and presumably also by the file system), does not support nested groups.

Seeing a fine oppurtunity to do some coding, I've hacked up a couple of lines of code to supported nested groups there. And I'll be damned, but it worked. Running id mkooijma now gives:

uid=10008(mkooijma) gid=10000(Domain Users) groups=10000(Domain Users),
    11001(Beheer), 11013(KasCo), 10001(Actievelingen), 11004(BoCie),
    11019(NoiZiA), 10002(Webmasters.nested), 11029(Webredactie)

Nice job, so I submitted my patch to the samba-technical mailing list yesterday. I also started hanging out in #samba-technical, where I got into a discussion with Wilco (which I happen to know IRL) this morning about my patch. We discussed in what way I could make it not break on groups that are members of themselves (indirectly), or users that would end up in the same group multiple times. A few minutes into this discussion, vl walked in on IRC. Since he apparently has been working on winbind group mappings before, he pointed me on a few more fundamental flaws of my approach.

It turns out that fixing the getgrent NSS interface is not really the way to go, for two reasons.

• Samba's getgrent interface is not really working that well. It does not support everything as it should, such as nested groups. Yet, the code is stable now, and trying to support nested groups, but not accounting for all possible scenarios will make the code unstable. Or more to the point:

  17:01 < blathijs> okay, so the current code doesn't work but doesn't break
  17:01 < blathijs> but my patch makes it break :-)
  

• Using getgrent for finding the groups a user is in is terribly inefficient. In short, getgrent returns a list of members for a group. So, to get the groups a user is on using getgrent the code has to iterate all existing groups and see if the user is in that particular group. Doable for a couple of groups, but it will not scale to hundreds or thousands groups and users.

  Also, since active directory and LDAP store group membership as "memberOf" attributes for the users, creating a list of members for a group probably involves iterating all users to find members (not sure about that though).

It turns out there is a getgrouplist function in FreeBSD and there is also a getgrouplist function in winbind. Yet, when calling getgrouplist, this call is not forwarded to winbind through nsswitch, but implemented by iterating with getgrent (as described above. getgrent is forwarded through nsswitch). So, FreeBSD should just call getgrouplist through nsswitch and be done with it?

Also, these features are supported on linux, so it should be possible. Wilco pointed out than NetBSD does support this and has nicely documented how. Basically, the problem is that the getgrouplist function uses a parameter is input and output paramater, which makes it unsuitable to forward using nss. NetBSD fixes this by making getgrouplist a wrapper around a (new) getgroupmembership function, which has a slightly different interface that is compatible with nss.

Yet, it seems that this getgroupmembership nss function is not implemented by winbind. This probably means NetBSD was changed to support this with other nss backends, but nobody got around to adding the function to winbind yet. But, we know that linux does support this using nss and winbind, so how is that implemented then?

It turns out that linux has implemented something similar, yet it wraps getgrouplist in a nss-compatible function initgroups_dyn. It's interface is practically compatible with getgroupmembership, the only difference is that getgroupmembership needs a preallocated buffer, while initgroups_dyn can allocate and resize the buffer itself when needed (though this seems like a better idea, the function must still be wrapped in getgrouplist, so it might not really matter anyway....).

Anyway, I now have a clear view what has to bo done. I will either:

• Port getgrouplist nss code from NetBSD to FreeBSD and implement getgroupmembership in winbind (for which I can probably borrow code from nss_ldap, which I think supports both linux and NetBSD).
• Port getgrouplist nss code from linux. This requires no winbind changes, since initgroups_dyn is already implemented in winbind.

Just now, I've found I'm not the only one that wants these changes. So I'll do some more exploring about the necessary changes soon, but now it's time for dinner.

Edit: I've looked at both Linux and NetBSD libc, and it seems NetBSD is the way to go. Both FreeBSD and NetBSD have taken their libc implementations from something called 4.4BSD-Lite2, meaning that both are already quite similar in contrast to Linux, whose libc is structured quite differently.

This Monday, I've attended my grandfather's funeral (or actually, of the father of Ineke, my father's new wife). It was nice, at least so far as funerals go on the niceness scale. A few family members said touching words in remembrance and also the church minister had a nice story (Even though I am absolutely no christian, these things are why I think the job of a minister is quite respectable and useful).

We spent the first part of the day in church, where people spoke and sang. We continued in the crematorium, where (after having to wait a while) some more people said some words. We then headed back to the church, for drinks and lunch. The end of the day was rather ok, I spent some time talking to family I had not seen in a while. We finished the day visiting my place in Enschede (my grandfather lived near here) for a cup of tea.

Since his death was not too sudden (He has been in a nursing home for terminal patients for two weeks), I was still able to have an intelligible conversation with him a week before and he turned 91 not too long ago, I seem to be coping with this loss pretty well. Sure, it hurts, but this is pretty much how it is supposed to be and how he wanted it to be. And that makes it a lot nicer.

Yet another attempt at a short post, which will most likely fail horribly. Anyway, I'll not even try to describe all the fun stuff since last post (Weekend of sailing at my father's newly opened sailing school, the elf fantasy fair, a sewing weekend for Evolution Events, learning functional programming) and skip ahead to yesterday.

Simonic invasion

Yesterday, I had a date with Simon, with whom I am going the build the LARP administration system thing. He is finishing his studies here in Enschede (he lives in Apeldoorn), so he dropped over after his day's work. We spent the evening fiddling arround with phpPeanuts, a framework for building information systems. I had high expectations of it, while Simon was a little sceptic.

It turns out the framework is probably highly usable, but (as most open source software) has a little lacking documentation (or actually, it's documentation is lacking some useful structure). We will probably use it and now we have a feeling for how it works and what it can do, we can start designing our thing. I am also quite confident that our cooperation will work out properly, since even though I am way more of a linux wizard than he is and have more php experience than he has, we are quite up to par on the conceptual level. Should be fun.

Pling

This morning I had a meeting with part of my design project group, to finally finish our product (Pling) and get our grades. On our todo list are four things: Getting highscores to work, ironing out the last bugs, getting our server to compile to a native binary (it's java) and finishing the technical documentation for the server.

Today, we spent the morning finding bugs and fighting compilers, and concluded this would not really work out. Compiling our java server native seems impossible due to the mysql library we used and debugging in flash is pretty impossible... So, we might not be able to finish this project fully, in that we will probably not deliver a fully working and useable game as we had hoped.

Ir. Niels

This afternoon, a flatmate, Nelis, held his final presentation. He is now officially Ir. and MSc. After his presentation, the usual (but nasty) questions and the drinks afterwards, we (most flatmates went there) went home to order food. Since I, together with Bert, was the last sober person, I ordered us shoarma. We had a fun evening hanging around on the balcony. Even though they were quite wasted, I had fun lauging with and about them. We've just finished with a nice campfire. So, I'll get outside again now, I think the fire should still be burning.