Tollef Fog Heen's blog

I recently upgraded kemper.freedesktop.org to lenny. Collabora are nice enough to sponsor some of my sysadmin work for freedesktop and so making sure we are actually running a supported distribution was a good start. The actual dist-upgrade went fine, but when I rebooted with a 2.6.26 kernel, it just hung in the early boot phase. Luckily, a newer kernel worked fine. However, a newer kernel also breaks the NFS kernel server in Lenny. A short backport later, NFS was working fine, except annarchy (which NFS mounts from kemper) didn't have nfs-common installed at all, meaning it lacked mount.nfs. Ooops.

Now, bugs was broken. It used an SSH tunnel from annarchy to kemper, but the startup script was nowhere to be found. I replaced it with a trivial stunnel setup which has the added advantage of reconnecting if the tunnel goes down.

The ssh config had to be fixed slightly. We used to use an old and patched sshd that stored all the keys in a single file. I added a tiny script to split that again. We also had MkHomeDir in sshd's config, now replaced with pam_mkhomedir.

Another interesting thing I learnt is that the iLO ssh daemon chucks you out if you try to send enviromental options to it. Like, LANG which is sent by default. Slightly confusing, but easy enough to fix once I knew what the problem was.

In addition to kemper, I upgraded, but did not reboot fruit (the admin and LDAP host), due to not having the iLO password. I did not want to risk sitting there with a non-booting machine I could not fix. It's going to be rebooted at some later stage. I also did not have the iLO password for gabe, which runs mail and some other faff, so I'll have to schedule some more downtime in the near future.

Lucas asks about how free the N900 is, whether he can download and recompile and reflash. I'll try to answer some of those questions.

No, you can't download all the source. Part of it is just not open. I am not privy to Nokia's decisions on why or why not to open up, but it seems like the user interface bits are only partially open. Hildon itself is open so you can poke at widgets and see how those work. The address book is not open. The telepathy component that talks to the cellular modem is not open.

As for having to accept EULAs, I honestly don't remember accepting one of those, but I'm not going to say there are none. There's at least one which is every time you install a package where you have to check a box saying "Yes, I know this package is third party and will not sue Nokia if it causes my house to burn down, my wife to divorce me or causes somebody to steal the car". It's annoying, but I'm willing to live with it.

The contents of apt's sources.list is:

deb https://downloads.maemo.nokia.com/fremantle/ssu/apps/ ./ 
deb https://downloads.maemo.nokia.com/fremantle/ssu/mr0 ./ 
deb https://downloads.maemo.nokia.com/fremantle/ovi/ ./ 
deb http://repository.maemo.org/extras/ fremantle free non-free
deb http://repository.maemo.org/extras-devel/ fremantle free non-free

(technically, it comes from /etc/apt/sources.list.d/hildon-application-manager.list, not sources.list.)

I believe the built-in applications are generally not free, so rebuilding everything that is free will for instance leave you without any address book UI, the built-in map application or camera. Sadly, the X driver is also proprietary, so you won't be able to see anything either.

I don't think you can usefully install another free distro on the N900. You might be able to, at some point, assuming somebody goes to the effort.

The last question is "- Besides the non-free telephony stack, are there any other “antifeatures” I should be aware of?". The telephony stack is implemented around Telepathy, which is LGPL-ed free software. While it's correct that telepathy-ring (which talks to the cellular modem), the call UI and most of the address book are proprietary, the rest of Telepathy is free. There are SIP and XMPP connection managers that are free, and you can install more connection managers for MSN, IRC and so on.

Also, I think it's important to emphasise that the telephony stack does not contain any antifeatures. The closest thing you would be able to find is probably the restriction to one active and one held call at the same time, but as one of the developers said: "That's to prevent the UI from going mad".

While I like to tout the N900 as a free phone, it is in no way completely free. Large parts of it are free, and almost as importantly: most of the programming interfaces are free and at least somewhat documented, so if somebody wants to replace the built-in camera application with a free one, they can replace the DBus interface that the camera app provides. Ditto for maps applications, the address book and so on.

I've been using an iphone since late 2007 as my primary phone and so I've gotten quite a few contacts and SMS conversations stored on it. Now that Collabora has given me a nice and shiny N900, I wanted to move my contacts and conversations over, but this proved to be a bit more work than expected. Please note that the following procedure worked for me, I have tried to take reasonable steps to prevent anything breaking, but if something breaks, you get to keep both pieces. I am not responsible and this comes with absolutely no warranty. Take backups.

What you need

• the addressbook and SMS SQLite databases. On my phone, they live in /var/mobile/Library/AddressBook and /var/mobile/Library/SMS.

• A copy of my iphone-contacts-convert script. It's written in Perl and should be reasonably easy to understand. Put it in the same directory as AddressBook.sqlitedb.

• A copy of my iphone-export-sms script. It's also written in Perl and should also be reasonably easy to understand. Put it in the same directory as sms.db.

• The smstools program you can get from this thread on talk.maemo.org.

The address book conversion script takes the SQLite database structure and converts that into a VCF file. It should be completely safe to run multiple times (it only does SELECT from the different tables in the contacts database, and you have made backups, haven't you?).

If it dies with an "Unknown property", "Unknown label" or other error, you can poke it and see if you can work out what's wrong or drop me an email and I'll see if I can help you. Assuming it doesn't fall over, it will spit out a series of VCards, which you should store in a file, which you then to the N900 and open in the address book. Assuming you have less than 1000 contacts, they should now all be in your address book. If you have more, you need to split the file.

A couple of known limitations:

• It doesn't handle some of the attributes, like job title, notes, department, display names, prefix and suffix. None of my contacts used those, so I just didn't care. Patches to change this accepted. Also, it doesn't handle custom attributes and birthdays. I intended to handle birthdays, but forgot and I have few enough contacts with birthdays that I just did it by hand.

• When it hits something it doesn't know how to handle, it stops and you need to add the relevant handle to the code. I think it is mostly clear, how to, but again, feel free to contact me with any problems.

• Only tested on firmware version 2.2. Yes, ancient, but it's what my iphone is running.

• If you have contacts that are organisations, they will come up with a blank full name. Just edit them on the N900 (pressing edit and then save immediately works fine) and they'll be automatically fixed.

• No picture support. This looked a bit involved, so I didn't do this bit. Should be possible with a bit of effort.

The procedure for exporting and importing SMS-es is a bit more involved. First, export the sms-es by running the perl script. It spits out a tab-separated file which you should copy to the N900 along with the smsimporter program from the smstools thread. Run ./smsimporter foo.csv and you should get all your SMS-es put into the conversation app. I ended up compiling my own smsimporter based on the 0.2.1 from the thread with the UUID patch too. Read the whole thread and it should be fairly clear.

Collabora was kind enough to buy N900s for all its employees. Yay! I got mine on Friday and has been playing around with it quite a bit. It's very shiny and the user experience is a lot better than the N810. There are a few graphical glitches, it seems it's XDamage damaging a bit of a window and it's just not quick enough to repaint. Not a problem, and it has far fewer instances of just hanging for half a second which my iPhone has. That is, it hasn't had any of those yet.

The screen is good, but resistive. Takes a short while to get used to when you're used to capacative, but it's not a problem at all. The keyboard is good, but I need to map something as the compose key. Having US/UK key caps and using the Norwegian layout is a bit confusing. Not really the fault of the device though.

The web browser is generally quite good. The gestures take a bit of time to get used to, but they're not hard as such. Some of the default "applications" are implemented as just links to the web pages of services like Twitter, which is a bit silly as you don't even get a version that's optimised for the N900. They're not useless, but they are absolutely nowhere near a real application. Also, the "Store" (Ovi Store) application/web page says "coming soon", which is quite odd.

I'm not sure if I can change the selection of applications on the default application list, but modifying the desktop is easy. There seems to be few themes and background images available so far, at least in anything resembling official repositories. Hopefully this will improve over time.

So far, I haven't actually written any code for the N900. I have some applications I want to write, mostly widget-style apps like "when does the next bus home leave from a bus stop close to me and where is the bus stop", but also some other ones.

Battery life is not great. It almost did 48 hours today with a bit of use underway, and I did charge it before it ran completely out, but when I'm used to closer to a week, it's not that good. Camera seems good and is quite fast, I think it took less than five seconds from opening the camera shutter until I had taken a picture. Shutter delay is quite bad at about a third or half a second, but this is a mobile phone (or mobile computer, as Nokia likes to call it) and not a DSLR, so I'm quite happy with it.

As a phone, it seems fine so far. I can make calls and accept calls and there's no noticeable problems with it. It also functions as a modem/DUN over bluetooth, which is quite useful.

Build quality seems good, there's a good feeling when sliding the keyboard in and out, but only time will tell how good it actually is.

So far, I'm happy with it, it's a big step up from my previous UK phone (which is a Nokia E70; my iPhone is a 2G phone so I can't use it here with the provider I'm using). Hopefully I'll post more happy stories about it in the days to come.

In my last post about the ekey, I complained about two things: memory leak in the server and missing reconnects if the client was disconnected for any reason. I've meaning to blog about the follow up for while, but haven't had the time before now.

Quite quickly after my blog post, Simtec engineers got in touch on IRC and we worked together to find out what the memory leak problem was. They also put in the reconnect support I asked for. All this in less than a week, for a device which only cost £36.

To make things even better, they picked up some other small bug fixes/requests from me, such as making ekeyd-egd-linux just Suggest ekeyd and the latest release (1.1.1) seems to have fixed some more problems.

All in all, I'm very happy about it. To make things even better, Ian Molton (of Collabora) has been busy fixing up virtio_rng in the kernel and adding EGD support (including reconnection support) to qemu and thereby KVM. Hopefully all this hits the next stable releases and I can retire my egd-over-stunnel hack.

As 3.0 format packages are now allowed into the archive, I am thinking about what I would like the workflow to look like and hoping one of them fits me.

For new upstream releases, I am imaginging something like:

1. New upstream version is released.
2. git fetch + merge into upstream branch.
3. Import tarballs, preferably in their original format (bz2/gzip), using pristine-tar.
4. Merge upstream to debian branch. Do necessary fixups and adjustments. At this point, the upstream..debian branch delta is what I want to apply to the upstream release. The reason I need to apply this delta is so I get all generated files into the package that's built and uploaded.

5. The source package has two functions at this point: Be a starting point for further hacking; and be the source that buildds use to build the binary Debian packages.

   For the former, I need the git repository itself. It is increasingly my preferred form of modification and so I consider it part of the source.

   For the latter, it might be easiest just to ship the orig.tar.{gz,bz2} and the upstream..debian delta. This does require the upstream..debian delta not to change any generated files, which I think is a fair requirement.

I'm not actually sure which source format can give me this. I think maybe the 3.0 (git) format can, but I haven't played around with it enough to see. I also don't know if any tools actually support this workflow.

Back at the Debian barbeque party at the end of August, I got myself an EntropyKey from the kind folks at Simtec. It has been working so well that I haven't really had a big need to blog about it. Plug it in and watch /proc/sys/kernel/random/entropy_avail never empty.

However, Collabora, where I am a sysadmin also got one. We are using a few virtual machines rather than physical machines as we want the security domains, but don't have any extreme performance needs. Like most VMs they have been starved from entropy. One problem presents itself: how do we get the entropy from the host system where the key is plugged in to the virtual machines?

Kindly enough the ekeyd package also includes ekeyd-egd-linux which speaks EGD, the TCP protocol the Entropy Gathering Daemon defined a long time ago. ekeyd itself can also output in the same protocol, so this should be easy enough, or so you would think.

Our VMs are all bridged together on the same network that is also exposed to the internet and the EGD protocol doesn't support any kind of encryption, so in order to be safe rather than sorry, I decided to encrypt the entropy. Some people think I'm mad for encrypting what is essentially random bits, but that's me for you.

So, I ended up setting up stunnel, telling ekeyd on the host to listen to localhost on a given port, and stunnel to forward connections to that port. On each VM, I set up stunnel to forward connections from a given port on localhost to the port physical machine where stunnel is listening. ekeyd-linux-egd is then told to connect to the port on localhost where stunnel is listening. After a bit of certificate fiddling and such, I can do:

# pv -rb < /dev/random > /dev/null  
17.5kB [4.39kB/s]

which is way, way better than what you will get without a hardware RNG. The hardware itself seems to be delivering about 32kbit/s of entropy.

My only gripes at this point is that the EGD implementation could use a little bit more work. It seems to leak memory in the EGD server implementation. Also, it would be very useful if the client would reconnect if it was disconnected for any reason. Even with those missing bits, I'm happy about the key so far.

Dear whoever runs the Telefonica APs in both Rio de Janeiro and Sao Paulo airports: Your DNS servers are returning SERVFAIL and has been doing so for quite a while. This is not helpful, perhaps you should set up some monitoring of them?

A little while ago, I read robster's post about librest, and it looked quite neat. I have had a plan for visualising GPX files for quite a while, hopefully with something that allows you to look at data for various bits of a track, like speed and the time you where there, but for various reasons, I haven't had the time before.

Last night, I took a little time to glue librest and libchamplain together. libchamplain is a small gem of a library, quite nice to work with and with the 0.3.3 release, it got support for rendering polygons. The result is in this screenshot:

This is about 160 lines of C, all included.

Currently, varnishlog does not support very advanced filtering. If you run it with -o, you can also do a regular expression match on tag

• expression. An example would be varnishlog -o TxStatus 404 to only show log records where the transmitted status is 404 (not found).

While in Brazil, I needed something a bit more expressive. I needed something that would tell me if I had vcl_recv call pass and the URL ended in .jpg.

varnishlog -o -c | perl -ne 'BEGIN { $/ = "";} print if
(/RxURL.*jpg$/m and /VCL_call.*recv pass/);'

fixed this for me.