Getting started with distri on Azure
Tags: linux, packagemanagement, azure, go
Published: 2019-09-04 17:55:33

You might have come across Michael Stapelberg's [Linux package managers are slow](https://michael.stapelberg.ch/posts/2019-08-17-linux-package-managers-are-slow/) write-up earlier this month. His hypothesis is that indexes, hooks/triggers and archive operations are a big source of issues with conventional Linux package managers.

Concurrently, [Michael announced **distri**](https://michael.stapelberg.ch/posts/2019-08-17-introducing-distri/), a Linux distro design to experiment with different approaches in these areas. I've been playing around with [**distri**](https://github.com/distr1/distri) after getting it up and running on Azure. Here's what I learned.

## What's in it?

In its current release, `jackherer`, **distri** ships Linux 5.1.9, systemd, and 170+ other packages including Python 2.7, Python 3, Perl, Golang and Docker, all the way to Xorg and i3.

`distri` is also a command-line tool used to build packages and the distro (images, repos) itself. The [online repo](https://repo.distr1.org/distri/jackherer/) contains said images and binary packages. The binary packages are SquashFS files, with metadata.

## What makes it unique?

In addition to the `distri` tool and the [packaging definitions](https://github.com/distr1/distri/tree/master/pkgs) for 400+ packages, the [GitHub repo](https://github.com/distr1/distri) contains the code (Go) for a FUSE filesystem that assembles images from the local software store as a working, read-only filesystem for the running instance.

So if you inspect the root of the filesystem, you'll notice that `/bin`, `/share` and `/lib` (and their `/usr` counterparts) point to `/ro`. And, you guessed it, `/ro` is handled by this FUSE component.

Each package:version pair ships in a separate directory under `/ro`. [Exchange directories](https://distr1.org/#exchange-directories) are used for places where each package is expected to contribute files, such as `/usr/include`.

The part that makes **distri** unique is that this FUSE component lazily loads the images as needed. The SquashFS images, roughly equivalent to binary packages such as debs and rpms, live in a separate *store* under `/roimg`.

What all of this implies is that package manager operations are very fast, because they involve transporting an SquashFS file (and `textproto` metadata) to the local software store, and then letting the FUSE component work its magic by assembling it in the filesystem "view". Here's [Michael showing how that works](https://asciinema.org/a/cwHaOq7LnY01lFB7kpQbAOVua)

Neither the concept of one folder per package or even the lazy loading with FUSE concepts are new (Nix, [AppFS](http://appfs.rkeene.org/web/index)) but I don't think I have seen both put together in a modern way. Michael says that this method provides [just enough FHS compatibility](https://michael.stapelberg.ch/posts/2019-08-17-introducing-distri/#fhs-compat) for 3rd-party applications to work, such as Chrome or Visual Studio Code.

## Behind the scenes

About 80% of the packages in the **distri** repo use the `cbuilder` build method, but other available and popular methods include `gomodbuilder` and `perlbuilder`. 

One of Michael's topics of interest is [hooks and triggers](https://michael.stapelberg.ch/posts/2019-07-20-hooks-and-triggers/), which he argues makes deserialization, idempotency, upstream standardization, etc., much more harder. Only one package in `distri` has a trigger, and that is `openssh` for host key generation.

The packaging is very streamlined (respective to upstream) with only about 60 patches, 130 manual extra build flags, 115 manually declared runtime dependencies and 295 custom build steps across over 400 packages.

Conceptually similar repos that you might want to compare include [nixpkgs](https://github.com/NixOS/nixpkgs), [Spack](https://github.com/spack/spack/tree/develop/var/spack/repos/builtin/packages), [Homebrew formulae](https://formulae.brew.sh/formula/) or the [Photon](https://github.com/vmware/photon/tree/master/SPECS) specs.

This is me building `nano` in distri. I use the `distri scaffold` tool to create a `textproto` file, declare my two build dependencies and go:

[![asciicast](https://asciinema.org/a/266011.svg)](https://asciinema.org/a/266011)

...this will end with something like:

```
2019/09/04 16:59:04   step 0: 2m31.51560212s (command: [${DISTRI_SOURCEDIR}/configure --host=x86_64-pc-linux-gnu --prefix=${DISTRI_PREFIX} --sysconfdir=/etc --disable-dependency-tracking])
2019/09/04 16:59:04   step 1: 18.112800177s (command: [make -j8 V=1])
2019/09/04 16:59:04   step 2: 2.581871913s (command: [make install DESTDIR=${DISTRI_DESTDIR} PREFIX=${DISTRI_PREFIX}])
[...]
2019/09/04 16:59:05 nano runtime deps: ["file-amd64-5.34-3" "glibc-amd64-2.27-3" "zlib-amd64-1.2.11-3" "ncurses-amd64-6.1-5"]
[...]
2019/09/04 16:59:05 package successfully created in /root/distri/build/distri/pkg/nano-amd64-4.3-1.squashfs
```

You can then place this package in your local store, where the FUSE component will pick it up, making the `nano` command avaiable in your `$PATH`. Neat!

## Beyond packaging

**distri** runs on Azure pretty much out of the box. Images can be converted to the VHD format, and serial console can be enabled optionally, but because it doesn't let `root` SSH into it by default (which is good) you might want to [consider the steps I documented](https://github.com/distr1/distri/issues/29#issuecomment-524669776) which include uploading a disk and creating a VM with the Azure CLI.

I also wrote a basic `/etc/os-release` for **distri**, because one is [currently not included](https://github.com/distr1/distri/issues/33). Here's what that looks like:

```
root@distri0:~# hostnamectl 
   Static hostname: distri0
         Icon name: computer-vm
           Chassis: vm
        Machine ID: 201aa7a551c14f299bb973f1ce206503
           Boot ID: e511d5411c804a7e8f24288025219796
    Virtualization: microsoft
  Operating System: distri (jackherer)
            Kernel: Linux 5.1.9
      Architecture: x86-64
```

More demos and [cool things to try](https://github.com/distr1/distri#cool-things-to-try) are on the GitHub repo. If you're interested in **distri**, I suggest you join the [mailing list](https://www.freelists.org/list/distri) or join the conversation on [GitHub Issues](https://github.com/distr1/distri/issues) for the repo.

If you're in Europe and attending [All Systems Go](https://all-systems-go.io/) I'd love to meet and talk more about Linux and package management. And if you're in the USA and attending [All Things Open](https://allthingsopen.org/) join me for a session on [The Future of Linux Distros in the Cloud](https://allthingsopen.org/talk/2-for-1-the-future-of-linux-distros-in-the-cloud/), October 14th, 2019 in Raleigh.