I recently upgraded kemper.freedesktop.org to lenny. Collabora are
nice enough to sponsor some of my sysadmin work for freedesktop and so
making sure we are actually running a supported distribution was a
good start. The actual dist-upgrade went fine, but when I rebooted
with a 2.6.26 kernel, it just hung in the early boot phase. Luckily,
a newer kernel worked fine. However, a newer kernel also breaks the
NFS kernel server in Lenny. A short backport later, NFS was working
fine, except annarchy (which NFS mounts from kemper) didn’t have
nfs-common installed at all, meaning it lacked mount.nfs. Ooops.
Now, bugs was broken. It used an SSH tunnel from annarchy to kemper, but the startup script was nowhere to be found. I replaced it with a trivial stunnel setup which has the added advantage of reconnecting if the tunnel goes down.
The ssh config had to be fixed slightly. We used to use an old and
patched sshd that stored all the keys in a single file. I added
a tiny script to split that again. We also had MkHomeDir in sshd’s
config, now replaced with pam_mkhomedir.
Another interesting thing I learnt is that the iLO ssh daemon chucks
you out if you try to send enviromental options to it. Like, LANG
which is sent by default. Slightly confusing, but easy enough to fix
once I knew what the problem was.
In addition to kemper, I upgraded, but did not reboot fruit (the
admin and LDAP host), due to not having the iLO password. I did not
want to risk sitting there with a non-booting machine I could not
fix. It’s going to be rebooted at some later stage. I also did not
have the iLO password for gabe, which runs mail and some other faff,
so I’ll have to schedule some more downtime in the near future.