After yet another compile-from-source of libhugetlbfs and
the hassle always entailed by source installs, I decided to take a look
at what is involved making a package for Debian. I have been a long-term
user of Debian but resisted temptations to get involved as there was enough
going on already and the flames on Debian lists were legendary. Since I've
realised there will always too much to do and I can push the D key
on my mailer if too lazy to killfile, I though I would give it a shot to
see what happens. This article is basically a log of what I did and what
I encountered along the way.|
It is not very prominent on the Debian Developer page but there is a Debian New Maintainers' Guide near the bottom of the page. I glanced at the policy documents first but most of the information seemed "obvious" in the context of a free software project with a strong philosophy. I could absorb the documents one at a time or I could punt a package out there and listen for complaints as a form of lint. Being lazy, I choose the latter. If I manage to really tick someone off in the process, err.... sorry, but I'd do it again. This is on my own time and Saturday is not that long.
First on the cards was getting the recommended packages. Plenty of disk space so rather than making careful decisions (time is short)
mel@arnold:~$ sudo apt-get update && sudo apt-get install dpkg-dev file \ gcc g++ libc6-dev make patch perl autoconf automake dh-make \ debhelper devscripts fakeroot gnupg g77 gpc xutils lintian \ linda pbuilder debian-policy developers-referenceBy rights, I should read all the policy and developers reference at this point but I figured at this point it was more useful to get a deb first and worry about maintainer mechanics later.
At this point I double checked libhugetlbfs was not already in Debian. I couldn't find evidence of it in the repositories, the package search tool or on Google. No one appeared to be working on it and I reckoned I met most of the other criteria other than the warning that a new maintainer should not be trying to manage libraries - sod it, how hard can it possibly be. Everything seems to be in place for package creation.
I hear that there are numerous workflows with dealing with Debian and I am sure I will look back on this some day and laugh with how primitive the start was. Anyway, from the guide
mel@arnold:~$ mkdir -p deb-libhugetlbfs mel@arnold:~$ cd deb-libhugetlbfs/ mel@arnold:~/deb-libhugetlbfs$ wget -q \ http://.../sourceforge/libhugetlbfs/libhugetlbfs-1.2.tar.gz mel@arnold:~/deb-libhugetlbfs$ tar -zxf libhugetlbfs-1.2.tar.gz mel@arnold:~/deb-libhugetlbfs$ cd libhugetlbfs-1.2 mel@arnold:~/deb-libhugetlbfs/libhugetlbfs-1.2$ make libs || echo Failure mel@arnold:~/deb-libhugetlbfs/libhugetlbfs-1.2$First note at this point is that building the tests fails in this version of libhugetlbfs. Will deal with this later.
After a quick glance through the dh_make man page, I ran
mel@arnold:~/deb-libhugetlbfs/libhugetlbfs-1.2$ dh_make -e email@example.com \ -f ../libhugetlbfs-1.2.tar.gz \ -c LGPL -l Maintainer name : Mel Gorman Email-Address : firstname.lastname@example.org Date : Sat, 17 Nov 2007 12:46:16 +0000 Package Name : libhugetlbfs Version : 1.2 License : lgpl Type of Package : Library HitThis generates a pile of boiler-plate gunk according to diff. None of it was a major surprise although there were unnecessary items like init scripts.
libhugetlbfs was both awkward and handy in this respect. It is awkward in that it does not use autoconf. It was handy in that the modifications were pretty simple, the installation does not create special symlinks and most of the paths it used already adhered to the Filesystem Hierarchy Standard. Here are the modifications I made;
Note here that the guide makes DESTDIR seem magical until the results file is encountered later. It can be set to BANANA as long as the rules file gets it right.
Writing the Control file
The first step in writing the control file was working out the build dependencies.
#!/bin/bash # Simple script to list debian packages depended on for building # Adapted from the Debian New Maintainers' Guide 1.2.11 # # Copyright © 1998-2002 Josip Rodin. # Copyright © 2005-2007 Osamu Aoki. strace -f -o /tmp/log $@ for x in `dpkg -S $(grep open /tmp/log| \ perl -pe 's!.* open\(\"([^\"]*).*!$1!' | \ grep "^/"| sort | uniq | \ grep -v "^\(/tmp\|/dev\|/proc\)" ) 2>/dev/null| \ cut -f1 -d":" | sort | uniq`; do echo -n "$x (>=" `dpkg -s $x|grep ^Version|cut -f2 -d":"` "), "; doneIt listed the dependencies of the build mechanism as
binutils (>= 2.17cvs20070426-8 ), libc6 (>= 2.6.1-1+b1 ), libc6-dev (>= 2.6.1-1+b1 ), libncurses5 (>= 5.6+20070716-1 ), libselinux1 (>= 2.0.15-2+b1 ), libsepol1 (>= 2.0.3-1+b1 ), linux-libc-dev (>= 2.6.21-6 ), locales (>= 2.6.1-1 )As objdump -p stated that libhugetlbfs.so only depended on libc.so.6 and the packages listed appeared to be already in build-essentials, I made the assumption that there were no unusual build dependencies. The end result was this control file.
Source: libhugetlbfs Priority: extra Maintainer: Mel GormanNote the ordering of the packages, this is subtle. The documentation gets installed in the first binary package mentioned in this list. Wanting the HOWTO to be with the library itself, the libhugetlbfs package had to be mentioned first. From there, the copyright and changelog entries were filled in before moving onto the rules file.
Writing the Rules file
The rules file is a Makefile file that determines what dpkg-buildpackage does. The guide said this would be complicated and it was not lying either. By far, getting the results right took the longest. dh_install had to be uncommented and set to read dh_install --sourcedir=debian/tmp. Secondly, the build command had to be updated to not run the tests.
The most awkward part was writing the debian/libhugetlbfs.installs. debian/libhugetlbfs.dirs and the -dev equivalent of the files. Depending on the architecture, there may be 64 bit libraries built as well. A script called libhugetlbfs_write_dirs_installs was created to check if the 64 bit libraries exist and it writes appropriate files for dh_install to pickup. There is probably a much better way of doing this.
Removing Unnecessary files
dh_make created a number of files that are unnecessary in this package. The list of files removed were README.Debian, cron.d.ex, dirs, emacsen-*, init.d.*, menu.ex libhugetlbfs-default.ex.
Documentation and man pages
A skeleton manual page was written to describe libhugetlbfs using manpage.1.ex as a template. The final version was saved as libhugetlbfs.7.ex and the SGML and XML templates were deleted. There was some confusion between the guide and the man pages here. The guide says section 7 is for macros. The man manual page says section 7 is for miscellaneous. I erred on the side of the manual page. The rules file was then updated so that the dh_installman was called properly.
Building the Actual Package
The package was simply built with
mel@arnold:~/deb-libhugetlbfs/libhugetlbfs-1.2$ dpkg-buildpackage -rfakerootTo credit the guide, it ran close to first time without error. Getting rules right earlier was the only time-consuming part. Once that was sorted out, it complained about some things such as not having a GPG key but nothing catastrophic.
Checking the Packages for Further Errors
Of course the guide could not cover all eventualities and so it requires you to run lintian and linda. These are both lint packages that catch different things - probably because they are maintained with different regularities. The errors it caught were fairly straight forward.
First up was a number of example (.ex) files left behind. The package processing scripts were renamed to drop the .ex extension. postinst and postrm were altered to call ldconfig. Second, the watch.ex file was renamed to watch and the line relevant to sourceforge was uncommented.
The last warnings were related to shlibs. While I would not expect someone to depend on libhugetlbfs, a shlibs file containing just the following was added.
libhugetlbfs 1.2 ( >= 1:1.2)These changes kept linda happy. Lintian still complained a lot but it was not clear if what it was complaining about was important. I decided to wait until a Debian Developer had a change to look at the output and determine if it was relevant or not.
At this point, I should also be checking the package with pbuilder. However, I was running out of time at this point and postponed it on the grounds it was unlikely to suffer build dependency errors.
Verifying the Package Works
The packages installed easily enough with dpkg and what remained now was verification. What I used to verify it worked is called STREAM which is a memory bandwidth benchmark that I had a handy script available for using with libhugetlbfs. If you are reproducing this test, I should point out that I recently found that memory bandwidth is reduced when using STREAM on some systems but have not had the chance to check out why yet. These were the results from an old laptop I use for testing new patches.
Running stream on an array 48MB in size allocated by malloc with small pages I got;
Function Rate (MB/s) Avg time Min time Max time Copy: 264.9805 0.1267 0.1266 0.1268 Scale: 289.1775 0.1161 0.1160 0.1166 Add: 296.7493 0.1704 0.1696 0.1762 Triad: 302.6247 0.1668 0.1663 0.1675Running with the libhugetlbfs library installed via the debian package, I got;
Function Rate (MB/s) Avg time Min time Max time Copy: 333.2745 0.1008 0.1007 0.1009 Scale: 325.6892 0.1031 0.1030 0.1031 Add: 383.6921 0.1314 0.1312 0.1325 Triad: 379.2916 0.1330 0.1327 0.1339All in all, seems to be in order.
At this stage, the package has been built, installed and verified. I guess, but don't know that a Debian Developer would upload it at this point based on their experience. However, for me the next step is to go on the debian-mentors mailing list and post a Request For Sponsor (RFS) with an "ad" for the package. I think I have a sponsor already recruited by the most reliable way possible - talked about it with him in a pub! I strongly suspect that actually getting the package in and dealing with the volume of yet more mailing lists and process rules will be the most painful aspect of this undertaking. Time will tell.
Ok, so at the start I could have just compiled and installed this in about 15 minutes without breaking sweat. The objective was to find out if the packaging process was not quite as easy as pub-discovered-Debian-Developer said it was but it was still pretty handy. The total time from sitting down with the view to reading the documentation to having a working package was about 5 hours. Overall I would describe the experience as reasonably pleasant, largely because the guide was so well written. It really was as easy as advertised.
However, I noted that there was a shedload more documentation that I should read and understand. I am going to assume I can pick it up as I go along to some extent. I suspect the more painful aspect of the experience will be getting the package into the distribution and dealing with yet another mailing list. If it becomes involved enough, you'll see another article.
All in all, just building a package from scratch is not hard. There is not much excuse to avoid it if you find it missing.