Freedombone Blog

Freedom in the Cloud

Continuously Integrating

I've now set up a continuous integration (CI) system for making image builds for the Freedombone system. This is something I had been wanting to do for a while, because building images by hand is laborious and not always reliable. If during a build a repo fails or a certificate expires somewhere on the internets then you have to start again. The details of the new system can be found here.

In compressed format Freedombone images are typically just over 3GB in size. Even on a fast multi-core machine builds can take an hour or more, because it's creating a minimal Debian operating system and then installing and configuring many things on top of that. In the past, and even now, I don't have a spare laptop or desktop machine to dedicate to continuous builds which would be powerful enough to make multi-gigabyte images within a reasonable amount of time. But ARM SBCs are getting good enough for this task, with a combination of a fast CPU and also the ability to run from an SSD which gives I/O performance which is an order of magnitude better than EMMC or microSD.

Various CI systems exist, but I thought I'd do the traditional thing which is to make one out of a bash script. In the old days, before the now well known CI systems existed, hackers would just create some scripts to compile overnight and report the results back by email which could be viewed in the morning. My CI system, called BirbCI is not a lot different from that except that the results are reported on a web page. Originally it was 80 lines of bash. It has grown a little, and is now a few hundred lines, but it's not all that complicated and is very general such that any type of build in whatever language you choose could be supported. The web page on which results appear can also be customized.

So in future it should be easier to do releases, since at any point in time there will be a bunch of ready made images. It will also be easier to know if anything I do with the source code breaks the build.

For the Birbs

The first continuous integration (CI) system I saw in use was back in the mid 2000s. Back then I don't think it was even called "continuous integration". It was just a way to test that builds worked, run unit tests and the send alerts if things had broken. For long builds that would usually be run overnight. It also wasn't anything formal, just some arbitrary and completely bespoke scripts on a server.

More recently I was discussing whether it would be a good idea to have continuous integration for the Freedombone project. Since making images involves building a minimal Debian and then installing and configuring other apps this can take quite a while. A couple of hours on a fairly powerful machine. At first I thought there wouldn't be much to be gained by doing this, but on second thoughts maybe it might be useful if I can find or requisition an old desktop machine to do it. At present testing of Freedombone builds is very manual and quite laborious.

I know that Lars Wirzenius has been developing a CI system called ick. As a side note you might also want to check out his Getting Things Done for Hackers. Ick looks ok and I could probably use it, but the code looks rather complicated and I'm not a huge fan of yarns as a software validation methodology, although I can understand why developers might be attracted down the path of making things as similar to natural language as possible. I was thinking that for my purposes CI doesn't need to be this complicated, since there's only a small set of things which it needs to do.

So I spent roughly a day writing a CI system and the result is BirbCI. This is really a single command implemented as a bash script which can set up systemd daemons to run builds and then report the results as a html page. It's about the simplest possible CI system at roughly 330 lines of script including comments. I also blinged it up a bit with a background image and coloured icons and text representing build status, so that it doesn't look too sparse and utilitarian.

This should be good enough to start thinking about making some sort of Freedombone build server. It's not a super-high priority, but maybe something to work on as a background task in the next few weeks or months.