Repository: systemd/mkosi

Issue:
FileNotFoundError: [Errno 2] No such file or directory: '~/.cache/mkosi/mkosi-workspacees02trw6/root/etc/machine-id'
Full build log: [build.log](https://github.com/systemd/mkosi/files/14852691/build.log)
Note: /home/xyz has been replaced with ~ and some pkg names replaced with the word HIDDEN.
mkosi v22 and v21 seem to have this issue.

Last few lines from the log
```
‣   Installing Ubuntu
‣ + env APT_CONFIG=/etc/apt.conf DEBIAN_FRONTEND=noninteractive DEBCONF_INTERACTIVE_SEEN=true INITRD=No apt-get -o APT::Architecture=amd64 -o APT::Architectures=amd64 -o APT::Install-Recommends=false -o APT::Immediate-Configure=off -o APT::Get::Assume-Yes=true -o APT::Get::AutomaticRemove=true -o APT::Get::Allow-Change-Held-Packages=true -o APT::Get::Allow-Remove-Essential=true -o APT::Sandbox::User=root -o Dir::Cache=/var/cache/apt -o Dir::State=/var/lib/apt -o Dir::Log=/var/log/apt -o Dir::State::Status=~/.cache/mkosi/mkosi-workspacees02trw6/root/var/lib/dpkg/status -o Dir::Log=~/.cache/mkosi/mkosi-workspacees02trw6 -o Dir::Bin::DPkg=/usr/bin/dpkg -o Debug::NoLocking=true -o DPkg::Options::=--root=~/.cache/mkosi/mkosi-workspacees02trw6/root -o DPkg::Options::=--force-unsafe-io -o DPkg::Options::=--force-architecture -o DPkg::Options::=--force-depends -o DPkg::Options::=--no-debsig -o DPkg::Use-Pty=false -o DPkg::Install::Recursive::Minimum=1000 -o pkgCacheGen::ForceEssential=, -o Acquire::AllowInsecureRepositories=true -o Acquire::AllowDowngradeToInsecureRepositories=true -o APT::Get::AllowUnauthenticated=true -o 'DPkg::Options::=--path-exclude=/usr/share/doc/*' -o 'DPkg::Options::=--path-include=/usr/share/doc/*/copyright' -o 'DPkg::Options::=--path-exclude=/usr/share/man/*' -o 'DPkg::Options::=--path-exclude=/usr/share/groff/*' -o 'DPkg::Options::=--path-exclude=/usr/share/info/*' install -oDebug::pkgDPkgPm=1 '-oDPkg::Pre-Install-Pkgs::=cat >~/.cache/mkosi/mkosi-workspacees02trw6/tmp/tmpj7xy8yqm' '?essential' '?exact-name(usr-is-merged)'
Reading package lists...
Building dependency tree...
Reading state information...
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
‣ + env APT_CONFIG=/etc/apt.conf DEBIAN_FRONTEND=noninteractive DEBCONF_INTERACTIVE_SEEN=true INITRD=No apt-get -o APT::Architecture=amd64 -o APT::Architectures=amd64 -o APT::Install-Recommends=false -o APT::Immediate-Configure=off -o APT::Get::Assume-Yes=true -o APT::Get::AutomaticRemove=true -o APT::Get::Allow-Change-Held-Packages=true -o APT::Get::Allow-Remove-Essential=true -o APT::Sandbox::User=root -o Dir::Cache=/var/cache/apt -o Dir::State=/var/lib/apt -o Dir::Log=/var/log/apt -o Dir::State::Status=~/.cache/mkosi/mkosi-workspacees02trw6/root/var/lib/dpkg/status -o Dir::Log=~/.cache/mkosi/mkosi-workspacees02trw6 -o Dir::Bin::DPkg=/usr/bin/dpkg -o Debug::NoLocking=true -o DPkg::Options::=--root=~/.cache/mkosi/mkosi-workspacees02trw6/root -o DPkg::Options::=--force-unsafe-io -o DPkg::Options::=--force-architecture -o DPkg::Options::=--force-depends -o DPkg::Options::=--no-debsig -o DPkg::Use-Pty=false -o DPkg::Install::Recursive::Minimum=1000 -o pkgCacheGen::ForceEssential=, -o Acquire::AllowInsecureRepositories=true -o Acquire::AllowDowngradeToInsecureRepositories=true -o APT::Get::AllowUnauthenticated=true -o 'DPkg::Options::=--path-exclude=/usr/share/doc/*' -o 'DPkg::Options::=--path-include=/usr/share/doc/*/copyright' -o 'DPkg::Options::=--path-exclude=/usr/share/man/*' -o 'DPkg::Options::=--path-exclude=/usr/share/groff/*' -o 'DPkg::Options::=--path-exclude=/usr/share/info/*' install
Reading package lists...
Building dependency tree...
Reading state information...
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
‣ + rm -rf -- ~/.cache/mkosi/mkosi-workspacees02trw6
‣ + rm -rf -- ~/.cache/mkosi/mkosi-workspacel3jigoji
‣  (Fixing ownership of package manager cache directory)
‣ + chown --recursive 65436:65436 '~/.cache/mkosi/ubuntu~focal~x86-64/cache/apt'
‣ + chown --recursive 65436:65436 '~/.cache/mkosi/ubuntu~focal~x86-64/lib/apt'
Traceback (most recent call last):
  File "~/src/dockerfiles/mkosi/repo/mkosi/mkosi/run.py", line 55, in uncaught_exception_handler
    yield
  File "~/src/dockerfiles/mkosi/repo/mkosi/mkosi/run.py", line 96, in fork_and_wait
    target(*args, **kwargs)
  File "~/src/dockerfiles/mkosi/repo/mkosi/mkosi/__init__.py", line 4082, in run_build
    build_image(Context(args, config, workspace=workspace, resources=resources))
  File "~/src/dockerfiles/mkosi/repo/mkosi/mkosi/__init__.py", line 3438, in build_image
    install_kernel(context, partitions)
  File "~/src/dockerfiles/mkosi/repo/mkosi/mkosi/__init__.py", line 2247, in install_kernel
    install_uki(context, kver, kimg, token, partitions)
  File "~/src/dockerfiles/mkosi/repo/mkosi/mkosi/__init__.py", line 2179, in install_uki
    initrds += context.config.initrds or [build_default_initrd(context)]
                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "~/src/dockerfiles/mkosi/repo/mkosi/mkosi/__init__.py", line 1687, in build_default_initrd
    build_image(
  File "~/src/dockerfiles/mkosi/repo/mkosi/mkosi/__init__.py", line 3383, in build_image
    install_distribution(context)
  File "~/src/dockerfiles/mkosi/repo/mkosi/mkosi/__init__.py", line 144, in install_distribution
    (context.root / "etc/machine-id").write_text("uninitialized\n")
  File "/usr/lib/python3.11/pathlib.py", line 1079, in write_text
    with self.open(mode='w', encoding=encoding, errors=errors, newline=newline) as f:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/pathlib.py", line 1045, in open
    return io.open(self, mode, buffering, encoding, errors, newline)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '~/.cache/mkosi/mkosi-workspacees02trw6/root/etc/machine-id'
~/src/dockerfiles/mkosi

```
Further notes:

I am not exactly sure what the reason for this problem is. There seem to be several things playing together, but I am not exactly sure, otherwise I would provide a minimal setup. But since there seems to be a logic error (workspace gets deleted before /etc/machine-id is written to, I guess the fix could be rather simple).

One thing I thought I noticed: this only occurs if I install our packages (via mkosi.prepare.chroot -> final), BUT since the log revealed a strange order of operations, I think the cause might be something else. Just as a hint if it is not so obvious.

```
File "~/src/dockerfiles/mkosi/repo/mkosi/mkosi/__init__.py", line 144, in install_distribution
    (context.root / "etc/machine-id").write_text("uninitialized\n")
```
fails because the workspace folder it uses got deleted right before this operation: 

```
‣ + rm -rf -- ~/.cache/mkosi/mkosi-workspacees02trw6
```

Further note: since the default tool tree can currently not be used (dependency issues), I used

```
# workaround, since mirrors (ubuntu and debian) do have dependency problems
# status: 02/04/2024
ToolsTree=default
ToolsTreeDistribution=debian
ToolsTreeRelease=sid
# workaround end
```

for this build.


Discussion:
Can you provide the the input that lead to this (even though not minimal)?

Also, have you tried the current git main? We have recently switched to writing just `/usr/lib/os-release`.
[mkosi_project.tar.gz](https://github.com/systemd/mkosi/files/14853450/mkosi_project.tar.gz)

Contains the anonymized version of what we do.

Haven't tried git main branch yet. Will let you know if it changes something.
@danielbisar Too much hidden unfortunately I need to be able to reproduce.

But looking at the output you provide, we try to install all packages marked as essential and that's returning zero hits, which causes the issue. So your custom repo is causing the issue. I think I know what will fix this.

Resolving patch:
diff --git a/mkosi/distributions/debian.py b/mkosi/distributions/debian.py
--- a/mkosi/distributions/debian.py
+++ b/mkosi/distributions/debian.py
@@ -152,7 +152,9 @@ def install(cls, context: Context) -> None:
                 [
                     "-oDebug::pkgDPkgPm=1",
                     f"-oDPkg::Pre-Install-Pkgs::=cat >{f.name}",
-                    "?essential", "?exact-name(usr-is-merged)",
+                    "?essential",
+                    "?exact-name(usr-is-merged)",
+                    "base-files",
                 ],
                 mounts=[Mount(f.name, f.name)],
             )


Test patch: