Distro Packaging
Tags: linux, packaging, blog, opensource
Published: 2024-02-25 14:55:57

Packaging is something we experienced long in the history of Unix. It is part of the process of software distribution. Distributing to end-users is often the goal of software. The idea is that an end-user can then take a package and install it on their local system.

There were common packaging standards used in commercial unixs.  A version of it is seen in the older (historical) BSD unix packaging utilities like add_pkg, which still exist in some form of all modern BSD variants. Newer packaging systems, and even BSD ports, often simply build upon / remain compatible with this in BSD.

Unlike with BSD, In GNU/Linux, every major distro re-invented system packaging in it's own image effectively from scratch, all of which are mutually incompatible both in form (data structures and utilities) and in binary builds. Each distro also has different and unrelated policies for packaging, different scheduling, different release processes, etc.

System packaging is still the most efficient way to distribute binary software optimized for systems, but to do so on your own is a major pain point. For a long time these distros have created the artificial "engineering" position of package maintainer to try solving this. In any other industry, the shipping clerk or warehouse guy was a very minor figure. Linux distros made shipping clerks kings of software.

A long time ago I learned redhat packaging. This was because I was producing a library, and all I wanted to do was create multiple packages for the header and library. At the time there was excellent book called Maximum RPM, which explained how to do so quickly and easily. 

I originally had wanted to use Debian GNU/Linux, but the Debian documentation for packaging something with multiple output packages like this at the time simply said "this is far too complex to do, so we won't even try to explain it to you.", and really it was true.

Although it is actually simpler to package in modern debian systems, especially, with the complexity of Debian, but because packaging and distro policies were often closely related and required attention to lots of detail, the kings of packaging separated from software developers, and we had the idea of upstream and downstream emerged. Upstreams produced simple source tarballs, and downstreams would package them in each distro.

This division of labor might have even worked, but for the human factor.  What things get packaged and why can often be a political debate within a distro. All the software developer wants to do is see the source tarball packaged and used, including in the distro they may use, and they might have to wait even many years before this might happen.

Naturally as a developer you may want to package something for a distro you use and do so by yourself. Some distros are very happy to have arbitrary people come along and package things or host personal archive servers to make it easy to do so, and some treat packagers as a kind of secret and exclusive society instead. All do have policies and rules about how packages are meant to work in the distro, accepting a package, updates, etc.  If you do go this route, congratulations, you may even find you now have a second full time unpaid job!

I had decided to focus exclusively on Alpine Linux, because it hit all the target use cases I cared about from stand-alone fast-boot devices to on-premise appliance servers, and even space optimized container images. I have come to accept I will have to package things for it myself, as much as I do not wish to have yet something else I need to do.

Something like flatpak, snaps, etc, do not really solve the problem of distro packaging.  Instead, they represent in a sense "yet another" way and "distro" you can target and maintain packages for, and do try explicitly pushing the task onto the poor developer, too. These packages will tend to run suboptimal on many distros, but indeed do mean you package once for mediocre performance or poor integration everywhere.

If you do go down this route of packaging for Linux related systems, I suggest only dealing with one distro and abandoning the idea you will package for more, maybe even drop any development related activities on other ones altogether. If you want something to be packaged in another distro X, let someone else involved in distro X figure out how to do that. But because you may not be able to get your software packaged or accepted in any distro at all to start with, you may have to target and learn how to do so for one, even if it's like another full time job, regardless of how you do so.