Repository: systemd/mkosi

Issue:
Unable to create bootable (bios) Debian image
I'm unable to create any qemu-bootable Debian image using `BootProtocols=bios`. 

`mkosi.default` file looks like this:
```ini
[Distribution]
Distribution=debian
Release=testing

[Output]
Format=gpt_ext4
Bootable=yes
BootProtocols=bios

[Partitions]
RootSize=2G
```
Same bug appears for Debian stable/unstable  and the recent Ubuntus (tested the last two, I think). In all cases, the issue is that there is no initrd file in `/boot` which then unsurprisingly fails during boot. The same `mkosi.default` file with Fedora 33 as distribution works just fine.

Host system is Fedora 33, reproducible with:
```
$ sudo mkosi --force
$ qemu-system-x86_64 -machine accel=kvm \
                       -smp 2 -m 1024 \
                       -drive format=raw,file=image.raw \
                       -device virtio-net-pci,netdev=net0 \
                       -netdev user,id=net0,hostfwd=tcp::5555-:22 \
                       -display sdl \
                       -serial file:console.out
```

`git bisect` says this started with 46ceca8ffc8a2edc0539e01eadc9d99357dd247 which (handwavy guess) could explain why the initrd never appears, maybe the partition isn't mounted correctly?


Discussion:
Thanks for report and the bisect, @whot. This is an excellent starting point to find the reason.
I get a kernel panic with this config, can you confirm you get the same error?

```
[    0.254608] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
[    0.256038] CPU: 1 PID: 1 Comm: swapper/0 Not tainted 5.9.0-4-amd64 #1 Debian 5.9.11-1
[    0.257113] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS ArchLinux 1.14.0-1 04/01/2014
[    0.258631] Call Trace:
[    0.258995]  dump_stack+0x6b/0x83
[    0.259462]  panic+0x101/0x2d7
[    0.259891]  mount_block_root+0x304/0x313
[    0.260478]  prepare_namespace+0x136/0x165
[    0.261054]  kernel_init_freeable+0x1ef/0x1fa
[    0.261659]  ? rest_init+0xaa/0xaa
[    0.262135]  kernel_init+0xa/0x106
[    0.262613]  ret_from_fork+0x22/0x30
[    0.263152] Kernel Offset: 0x800000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
[    0.264603] ---[ end Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) ]---
```
Yep, same error. And you probably have better tools, but I used this to check `/boot`:

```
offset=$(fdisk -l image.raw | grep image.raw2 | cut -d ' ' -f 3)
sudo mount -o ro,loop,offset=$((offset * 512)) image.raw loop/
sudo ls loop/boot
```
and weep at the lack of an initrd file :)
I think the git bisect was a false positive, or maybe there are other bugs at play. 925a0090e39b6a6b017f9365c1b4d2906a37ee63 introduced this code:
```diff
+ if not do_run_build_script and args.bootable and args.with_unified_kernel_images:
+        # Disable dracut postinstall script for this apt-get run.
+        env['INITRD'] = 'No'
```
I'm guessing this is a pretty good hint for "don't create an initrd". Removing the `env` line makes dracut generate the initrd with the above example. 

Before:
```
Setting up dracut-core (050+65-1) ...
Setting up dracut (050+65-1) ...
update-initramfs: deferring update (trigger activated)
Setting up linux-image-5.9.0-4-amd64 (5.9.11-1) ...
I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.9.0-4-amd64
I: /vmlinuz is now a symlink to boot/vmlinuz-5.9.0-4-amd64
Setting up linux-image-amd64 (5.9.11-1) ...
Processing triggers for libc-bin (2.31-5) ...
Processing triggers for dracut (050+65-1) ...
```
After:
```
Setting up dracut-core (050+65-1) ...
Setting up dracut (050+65-1) ...
update-initramfs: deferring update (trigger activated)
Setting up linux-image-5.9.0-4-amd64 (5.9.11-1) ...
I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.9.0-4-amd64
I: /initrd.img.old is now a symlink to boot/initrd.img-5.9.0-4-amd64
I: /vmlinuz is now a symlink to boot/vmlinuz-5.9.0-4-amd64
I: /initrd.img is now a symlink to boot/initrd.img-5.9.0-4-amd64
/etc/kernel/postinst.d/dracut:
dracut: Generating /boot/initrd.img-5.9.0-4-amd64
mknod: /var/tmp/dracut.n115A4/initramfs/dev/kmsg: Operation not permitted
mknod: /var/tmp/dracut.n115A4/initramfs/dev/console: Operation not permitted
Setting up linux-image-amd64 (5.9.11-1) ...
Processing triggers for libc-bin (2.31-5) ...
Processing triggers for dracut (050+65-1) ...
dracut: Generating /boot/initrd.img-5.9.0-4-amd64
mknod: /var/tmp/dracut.mHlLVr/initramfs/dev/kmsg: Operation not permitted
mknod: /var/tmp/dracut.mHlLVr/initramfs/dev/console: Operation not permitted

```
I'm guessing enough code moved around that the comment is no longer up-to-date and now just prevents the initrd to be created.
Either way, I've tried this twice with/without the `INITRD` environment variable here and removing it reliably fixes the issue.

So you're not getting an initrd because unified kernel images are enabled. The initrd is baked into the efi image generated by dracut. `--without-unified-kernel-images` should fix the issue.

We might need to enable this by default for bios builds. I don't know if it makes much sense to use unified kernel images when doing a BIOS boot.

Thing is that for Debian, both dracut and the default initramfs generator are at play. We haven't really put in the time to sort out the initramfs story on Debian/Ubuntu unfortunately.
I did try that, but apparently "Sorry, --without-unified-kernel-images is not supported on this distro." :)
I'll make a PR tomorrow to enable it on debian when using bios. I think I didn't enable it for debian because it breaks when using efi.

Resolving patch:
diff --git a/mkosi/__init__.py b/mkosi/__init__.py
--- a/mkosi/__init__.py
+++ b/mkosi/__init__.py
@@ -4924,11 +4924,12 @@ def load_args(args: CommandLineArguments) -> CommandLineArguments:
     if args.bootable and args.distribution in (Distribution.clear, Distribution.photon):
         die("Sorry, --bootable is not supported on this distro")
 
-    if not args.with_unified_kernel_images and args.distribution in (Distribution.debian,
-                                                                     Distribution.ubuntu,
-                                                                     Distribution.mageia,
-                                                                     Distribution.opensuse):
-        die(f"Sorry, --without-unified-kernel-images is not supported on this distro.")
+    if not args.with_unified_kernel_images and "uefi" in args.boot_protocols:
+        if args.distribution in (Distribution.debian,
+                                 Distribution.ubuntu,
+                                 Distribution.mageia,
+                                 Distribution.opensuse):
+            die(f"Sorry, --without-unified-kernel-images is not supported in UEFI mode on this distro.")
 
     if args.verity and not args.with_unified_kernel_images:
         die("Sorry, --verity can only be used with unified kernel images")


Test patch: