Package: debspawn
DebspawnDebspawn is a tool to build Debian packages in an isolated environment. Unlike similar tools likesbuildorpbuilder,debspawnusessystemd-nspawninstead of plain chroots to manage the isolated environment.
This allows Debspawn to isolate builds from the host system much more via containers. It also allows
for more advanced features to manage builds, for example setting resource limits for individual builds.Please keep in mind that Debspawn isnota security feature! While it provides a lot of isolation from the
host system, you should not run arbitrary untrusted code with it. The usual warnings for all container technology
apply here.Debspawn also allows one to run arbitrary custom commands in its environment. This is used by the Laniakea1Spark workers
to execute a variety of non-package builds and QA actions in the same environment in which we usually build packages.Debspawn was created to be simple to use in automation as well as by humans. It should both be easily usable on large
build farms with good integration with a job runner, as well as on a personal workstation by a human user (to reproduce
builds done elsewhere, or to develop a Debian package).
Due to that, the most common operations are as easily accessible as possible and should require zero configuration
by default. Additionally,debspawnwill always try to do the right thing automatically before resorting to a flag
that the user has to set.
Options which change the build environment are - with one exception - not made available intentionally, so
achieving reproducible builds is easier.
See the FAQ below for more details.UsageInstalling DebspawnVia the Debian packageOn Debian/Ubuntu, simply runsudoaptinstalldebspawnto start using Debspawn.Via PyPI⚠ WARNING: Careful when installing via PyPI!While we do shipdebspawnon PyPI, installing it viapipwill not install certain system services
to automate cache cleanup and temp data cleanup. In addition to that, all manual pages will be missing.
This is due to intentional limitations of Python packages installed via pip.If you want to install Debspawn via PyPI anyway, you can usepip install debspawn.
You can decide to install the system data files manually later by running theinstall-sysdata.pyscript from the
Git repository and adjusting thedebspawnbinary path in the installed systemd units, if you want to.Via the Git repositoryClone the Git repository, install the (build and runtime) dependencies ofdebspawn:sudoaptinstallxsltprocdocbook-xslpython3-setuptoolszstdsystemd-containerdebootstrapYou can the rundebspawn.pydirectly from the Git repository, or choose to install it:sudopythonsetup.pyinstall--root=/Debspawn requires at least Python 3.9 on the host system, and Python 3.5 in the container.
We try to keep the dependency footprint of this tool as small as possible, so it is not planned to raise that
requirement or add any more dependencies anytime soon (especially not for the minimum Python version used in a container).On superuser permissionIfsudois available on the system,debspawnwill automatically request root permission
when it needs it, there is no need to run it as root explicitly.
If it can not obtain privileges,debspawnwill exit with the appropriate error message.Creating a new imageYou can easily create images for any suite that has a script indebootstrap. For Debian Unstable for example:$debspawncreatesidThis will create a Debian Sid (unstable) image for the current system architecture.To create an image for testing Ubuntu builds:$debspawncreate--arch=i386cosmicThis creates ani386image for Ubuntu 18.10. If you want to use a different mirror than set by default, pass it
with the--mirroroption.Refreshing an imageJust rundebspawn updateand give the details of the base image that should be updated:$debspawnupdatesid
$debspawnupdate--arch=i386cosmicThis will update the base image contents and perform other maintenance actions.Building a packageYou can build a package from its source directory, or just by passing a plain.dscfile todebspawn. If the result
should be automatically signed, the--signflag needs to be passed too:$cd~/packages/hello
$debspawnbuildsid--sign

$debspawnbuild--arch=i386cosmic./hello_2.10-1.dscBuild results are by default returned in/var/lib/debspawn/results/.If you need to inject other local packages as build dependencies, placedebfiles in/var/lib/debspawn/injected-pkgs(or other location set in the config file).Building a package - with git-buildpackageYou can use a command like this to build your project with gbp and Debspawn:$gbpbuildpackage--git-builder='debspawn build sid --sign'You might also want to add--results-dir=..to the debspawn arguments to get the resulting artifacts in the directory
to which the package repository was originally exported.Manual interactive-shell actionIf you want to, you can log into the container environment and either play around in
ephemeral mode with no persistent changes, or pass--persistenttodebspawnso all changes are permanently saved:$debspawnloginsid# Attention! This may alter the build environment!$debspawnlogin--persistentsidDeleting a container imageAt some point, you may want to permanently remove a container image again, for example because the
release it was built for went end of life.
This is easily done as well:$debspawndeletesid
$debspawndelete--arch=i386cosmicRunning arbitrary commandsThis is achieved with thedebspawn runcommand and is a bit more involved. Refer to its manual page
and help output for more information on how to use it:man debspawn run.Global configurationDebspawn will read a global configuration file from/etc/debspawn/global.toml, or a configuration file in a location
specified by the--configflag.
If a configuration file is specified on the command line, the global file is ignored completely rather than merged.The config is a TOML file containing any of the following (all optional) keys:OSRootsDir: directory for os images (/var/lib/debspawn/images/)ResultsDir: directory for build artifacts (/var/lib/debspawn/results/)APTCacheDir: directory for debspawn's own package cache (/var/lib/debspawn/aptcache/)InjectedPkgsDir: packages placed in this directory will be available as dependencies for builds (/var/lib/debspawn/injected-pkgs/)TempDir: temporary directory used for running containers (/var/tmp/debspawn/)AllowUnsafePermissions: allow usage of riskier container permissions, such as binding the host/devand/procinto the container (false)FAQWhy use systemd-nspawn? Why not $other_container_manager?Systemd-nspawn is a very lightweight container solution readily available without much (or any) setup on all Linux
systems that are using systemd. It does not need any background daemon and while it does not provide a lot of features,
it fits the relatively simple usecase of building in an isolated environment perfectly.Do I need to set up apt-cacher-ng to use this efficiently?No - whileapt-cacher-ngis generally a useful tool, it is not required for efficient use ofdebspawn.debspawnwill cache downloaded packages between runs fully automatically, so packages only get downloaded when
they have not been retrieved before.Is the build environment the same as sbuild?No, unfortunately. Due to the different technology used, there are subtle differences between sbuild chroots anddebspawncontainers.
The differences should - for the most part - not have any impact on package builds, and any such occurrence is highly
likely a bug in the package's build process. If you think it is not, please file a bug against Debspawn. We try to be
as close to sbuild's default environment as possible.One way the build environment differs from Debian's default sbuild setup intentionally is in its consistent use of unicode.
By default,debspawnwill ensure that unicode is always available and used. If you do not want this behavior, you can pass
the--no-unicodeflag todebspawnto disable unicode in the tool itself and in the build environment.Will this replace sbuild?Not in the foreseeable future on Debian itself.
Sbuild is a proven tool that works well for Debian and supports other OSes than Linux, whiledebspawnis Linux-only,
a thing that will not change due to its use of systemd.
However, Laniakea-using derivatives such as PureOS use the tool for building all packages and for constructing other
build environments to e.g. build disk images.What is the relation of this project with Laniakea?The Laniakea job runner usesdebspawnfor a bunch of tasks and the integration with the Laniakea system is generally
quite tight.
Of course you can usedebspawnwithout Laniakea and integrate it with any tool you want. Debspawn will always be usable
without Laniakea automation.This tool is really fast! What is the secret?Surprisingly, building packages withdebspawnis often a bit faster than usingpbuilderandsbuildwith their
default settings.
The speed gain comes in large part from the internal use of the Zstandard compression algorithm for container base images.
Zstd allows for fast decompression of the tarballs, which is exactly why it was chosen (LZ4 would be even faster,
but Zstd actually is a good compromise between compression ration and speed). This shaves off a few seconds of time
for each build that is used on base image decompression.
Additionally, Debspawn useseatmydatato disable fsync & co. by default in a few places, improving the time it takes
to set up the build environment by quite a bit as well.
If you want, you can configure other tools to make use of the same methods as well and see if they run faster.
There's nothing new or unusually clever here at all!