wiki:Linux/Ubuntu/Packages/CreatingPbuilderVariations

Creating Pbuilder Variations

Instead of using the various debootstrap options and other pbuilder options on the command line I use an amalgamation of the  Ubuntu Wiki Pbuilder Howto and my own modifications to ~/.pbuilderrc and additional pbuilder hook scripts to ease the package build testing for different distributions and architectures before uploading to PPA.

Read the companion article Automated Build Tests for Multiple Releases for scripts that make extensive use of this functionality.

pbuilderrc modification

  • Use previously built packages as build-dependencies. A --bindmount for /var/cache/pbuilder/${RELEASE}-${ARCH}/results is created. Combined with the pdebuild hook D70results this allows packages to build within the pbuild environment that depend on packages not in the official archives, but that have been built for the same distribution and architecture locally, and are in the local cache.
  • Look for the environment variable BASE and use it to set BASETGZ - the location and name of the pbuilder archive file. This allows the setting to over-ride the enforced value in /etc/pbuilderrc (BASETGZ=/var/cache/pbuilder/base.tgz).

Note: The value assigned to BASE is processed by dirname so the last segment beginning with a forward-slash is stripped off. This means that to specify the build location /home/all/pbuilder it needs BASE=/home/all/pbuilder/base. The final /base will be stripped off before the calculated name of the TGZ is added.

Note The ~/.pbuilderrc script was refined so it only applies custom settings when BUILD_TEST=1 is in the environment. This is to prevent it interfering with other pbuilder functions.

  • Building of packages inside the chroot as a regular user rather than root. This can be important since some packages won't build correctly as root:
BUILDUSERID=pbuilder

Note: this won't create the user for pbuilder base packages that already exist. For those base packages the user will need to be added manually:

sudo BUILD_TEST=1 BASE="/home/all/pbuilder/base" RELEASE=hardy ARCH=amd64 pbuilder login --save-after-login

Once at the chroot prompt issue the commands:

useradd -m pbuilder
exit

The updated base will be saved (These commands should be issued after my ~/.pbuilderrc is installed).

It sets the pbuilder location for external scripts to be ~/.pbuilder-hooks

The file is attached to this article Download and can be directly downloaded.

Simple command launch

By setting the environment variables BASE, DIST and ARCH the options are calculated by ~/.pbuilderrc so, for example, all the following become possible:

# Create an environment for Ubuntu gutsy in the current directory
sudo BUILD_TEST=1 BASE="$(pwd)/base" RELEASE=gutsy pbuilder create

# Create an environment for Debian sid
sudo BUILD_TEST=1 RELEASE=sid pbuilder create

# Create an environment for Ubuntu gutsy with the i386 architecture in a defined directory
sudo BUILD_TEST=1 BASE="/home/all/pbuilder/base" RELEASE=gutsy ARCH=i386 pbuilder create

# Update an environment for Ubuntu gutsy in the current directory
sudo BUILD_TEST=1 BASE="$(pwd)/base" RELEASE=gutsy pbuilder update

# Build a package using Ubuntu gutsy (same architecture as is running)
cd ${PACKAGE}-${VERSION}
BUILD_TEST=1 BASE="$(pwd)/base" RELEASE=gutsy pdebuild

# Build a package using Ubuntu gutsy with i386 architecture in a defined directory
cd ${PACKAGE}-${VERSION}
BUILD_TEST=1 BASE="/home/all/pbuilder/base" RELEASE=gutsy ARCH=i386 linux32 pdebuild

# Build a package specified by a .dsc file using Ubuntu hardy with i386 architecture in a defined directory
BUILD_TEST=1 BASE="/home/all/pbuilder/base" RELEASE=hardy ARCH=i386 linux32 pbuilder build /home/all/SourceCode/kvm-72+dfsg-0ubuntu1~ppa1h.dsc

Dependencies on Local Packages

Install the hook `~/.pbuilder-hooks/D70results` Download. Make it executable:

chmod a+x ~/.pbuilder-hooks/D70results

This combined with the BINDMOUNT in ~/.pbuilderrc allows the pbuilder to build packages for which the build-dependencies can only be satisfied from previously built packages in the local cache. In other words, packages that don't exist in the official archives (maybe the build-depend requires a newer version than is in the repositories) will be found in the local cache.

Here's an example of how handy this is. I was building the latest trac-0.11 for Gutsy and Hardy. I grabbed the latest source from the Ubuntu Intrepid package and updated debian/changelog to create a hardy build. It has a Build-Depends on python-genshi (>= 0.5) but the version in the Hardy repository is 0.4.4. If trying to build the package pdebuild fails, reporting:

The following packages have unmet dependencies:
  pbuilder-satisfydepends-dummy: Depends: python-genshi (>= 0.5) but it is not installable

So I then fetched the latest (Intrepid) source for genshi, updated debian/changelog to create a hardy package, and built it using pbuilder. This resulted in the file /var/cache/pbuilder/hardy-amd64/result/python-genshi_0.5.0-1ubuntu1~ppa1h_amd64.deb.

Building trac-0.11 now shows:

-> Mounting /var/cache/pbuilder/hardy-amd64/result/
 -> policy-rc.d already exists
Obtaining the cached apt archive contents
Installing the build-deps
 ** Packages in archive but missing from override file: **
  kvm kvm-source python-genshi trac

 Wrote 4 entries to output Packages file.
Adding apt source: deb file:///var/cache/pbuilder hardy-amd64/result/
Ign file: hardy-amd64/result/ Release.gpg
Ign file: hardy-amd64/result/ Release
Ign file: hardy-amd64/result/ Packages
Get:1 http://archive.ubuntu.com hardy Release.gpg [191B]

...

Selecting previously deselected package python-genshi.
Unpacking python-genshi (from .../python-genshi_0.5.0-1ubuntu1~ppa1h_amd64.deb) ...

...

Setting up python-genshi (0.5.0-1ubuntu1~ppa1h) ...

and completes successfully.

Cross-compiling architecture problems

If testing a package build in an i386 environment on an amd64 host it is possible that unexpected and hard-to-diagnose build errors will occur. I had issues with gnu/stubs-64.h missing trying to build the kvm package. The reason was qemu/configure does a big/little-endian test and the source-code it compiles causes the stubs-64.h file to be #include-ed (since the host architecture is 64-bit).

Another issue is missing gcc-multilib which, if the redirects of STDERR to /dev/null are removed, will reveal:

/usr/bin/ld: skipping incompatible /usr/lib/gcc/i486-linux-gnu/4.2.3/libgcc.a when searching for -lgcc
/usr/bin/ld: skipping incompatible /usr/lib/gcc/i486-linux-gnu/4.2.3/libgcc.a when searching for -lgcc

/usr/bin/ld: cannot find -lgcc
collect2: ld returned 1 exit status
big/little test failed

These problems are solved by causing pbuilder / pdebuild to run with a 32-bit kernel personality. Install the linux32 package on the 64-bit host:

$ sudo apt-get install linux32

And then start pdebuild from linux32:

BASE="/home/all/pbuilder/base" RELEASE=hardy ARCH=i386 linux32 pdebuild

This will build an i386 package using a 32-bit kernel personality. What this means is that if the configure scripts or Makefiles use "uname -m" they'll get a 32-bit result. E.g.:

$ uname -m
x86_64
$ linux32 uname -m
i686

Depends on sun-java?-*

Install the hook ~/.pbuilder-hooks/ D50sun-java-licences Download. Make it executable:

chmod a+x ~/.pbuilder-hooks/ D50sun-java-licences

This will ensure that any package that Build-Depends on, or whose Build-Depends packages in turn Depend on, the sun-java?-*{bin,jre,jdk} will install successfully in a non-interactive buildd or pbuilder. Usually these Sun packages interactively request acceptance of the Sun Distributors Licence for Java (DLJ) which causes builds to fail since there is no operator interaction.

This isn't so important with Hardy and later since most Java packages depend on openjdk (or its Provides: java2-runtime, java5-runtime, java6-runtime). With Gutsy and prior releases however there are several packages that need to depend on the Sun packages since the alternative, icedtea-java7, has many problems that can cause the resulting Java applications to fail.

Updates

2008-09-29 .pbuilderrc, made build-test code conditional based on BUILD_TEST=1, to prevent the custom code interfering with other tools such as prevu.
2008-08-24 .pbuilderrc and D70results Refactored, ${DIST} renamed to ${RELEASE} to be consistent with other scripts
2008-08-20 .pbuilderrc Added support for Feisty
2008-08-20 .pbuilderrc Fix: use sudo for mkdir -p ${BUILDRESULT}

Attachments