Repository: spack/spack

Issue:
Add a better error message when `container:os_packages:command` is needed
<!-- Explain, in a clear and concise way, the command you ran and the result you were trying to achieve.
Example: "I ran `spack find` to list all the installed packages and ..." -->
I ran `spack containerize` to generate a Dockerfile with custom build and final  images in order to try it out.

### Steps to reproduce the issue
Here is an exaple of `spack.yaml` file:
```console
spack:
  specs:
  - gromacs+mpi
  - mpich

  container:
    # Select the format of the recipe e.g. docker,
    # singularity or anything else that is currently supported
    format: docker

    # Sets the base images for the stages where Spack builds the
    # software or where the software gets installed after being built..
    images:
      build: custom/cuda-10.1-ubuntu18.04:latest
      final: nvidia/cuda:10.1-base-ubuntu18.04

    # Whether or not to strip binaries
    strip: false

    # Additional system packages that are needed at runtime
    os_packages:
      final:
      - libgomp

    # Extra instructions
    extra_instructions:
      final: |
        RUN echo 'export PS1="\[$(tput bold)\]\[$(tput setaf 1)\][gromacs]\[$(tput setaf 2)\]\u\[$(tput sgr0)\]:\w $ "' >> ~/.bashrc

    # Labels for the image
    labels:
      app: "gromacs"
      mpi: "mpich"

```

The following lines caused the issue:
```console
    os_packages:
      final:
      - libgomp
```
*Checked by commenting them out.* 

However, everything is ok if I use official Spack Images i.e.:
```console
    images:
      os: centos/7
      spack: 0.15.4
```

### Error Message

<!-- If Spack reported an error, provide the error message. If it did not report an error but the output appears incorrect, provide the incorrect output. If there was no error message and no output but the result is incorrect, describe how it does not match what you expect. -->
```console
$ spack containerize
==> [2021-01-23-15:00:45.588195] Imported containerize from built-in commands
==> [2021-01-23-15:00:45.588861] Imported containerize from built-in commands
==> [2021-01-23-15:00:45.693973] Reading config file /opt/spack/etc/spack/defaults/config.yaml
Traceback (most recent call last):
  File "/opt/spack/bin/spack", line 68, in <module>
    sys.exit(spack.main.main())
  File "/opt/spack/lib/spack/spack/main.py", line 762, in main
    return _invoke_command(command, parser, args, unknown)
  File "/opt/spack/lib/spack/spack/main.py", line 490, in _invoke_command
    return_val = command(parser, args)
  File "/opt/spack/lib/spack/spack/cmd/containerize.py", line 24, in containerize
    recipe = spack.container.recipe(config)
  File "/opt/spack/lib/spack/spack/container/writers/__init__.py", line 52, in recipe
    return create(configuration)()
  File "/opt/spack/lib/spack/spack/container/writers/__init__.py", line 198, in __call__
    return t.render(**self.to_dict())
  File "/opt/spack/lib/spack/spack/tengine.py", line 64, in to_dict
    d = [(name, getattr(self, name)) for name in self.context_properties]
  File "/opt/spack/lib/spack/spack/tengine.py", line 64, in <listcomp>
    d = [(name, getattr(self, name)) for name in self.context_properties]
  File "/opt/spack/lib/spack/spack/container/writers/__init__.py", line 135, in os_packages_final
    return self._os_packages_for_stage('final')
  File "/opt/spack/lib/spack/spack/container/writers/__init__.py", line 151, in _os_packages_for_stage
    return self._package_info_from(package_list)
  File "/opt/spack/lib/spack/spack/container/writers/__init__.py", line 173, in _package_info_from
    os_pkg_manager = self.container_config['os_packages']['command']
KeyError: 'command'
```

### Information on your system
```console
$ cat /etc/*-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.5 LTS"
NAME="Ubuntu"
VERSION="18.04.5 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.5 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic

$ uname -a
Linux 8d07bd2ab20a 5.4.0-62-generic #70~18.04.1-Ubuntu SMP Tue Jan 12 17:18:00 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

$ spack debug report
* **Spack:** 0.16.0
* **Python:** 3.6.9
* **Platform:** linux-ubuntu18.04-skylake
* **Concretizer:** original
```
<!-- Please include the output of `spack debug report` -->

<!-- If you have any relevant configuration detail (custom `packages.yaml` or `modules.yaml`, etc.) you can add that here as well. -->

### Additional information


<!-- These boxes can be checked by replacing [ ] with [x] or by clicking them after submitting the issue. -->
- [x] I have run `spack debug report` and reported the version of Spack/Python/Platform
- [x] I have searched the issues of this repo and believe this is not a duplicate
- [x] I have run the failing commands in debug mode and reported the output

<!-- We encourage you to try, as much as possible, to reduce your problem to the minimal example that still reproduces the issue. That would help us a lot in fixing it quickly and effectively!

If you want to ask a question about the tool (how to use it, what it can currently do, etc.), try the `#general` channel on our Slack first. We have a welcoming community and chances are you'll get your reply faster and without opening an issue.

Other than that, thanks for taking the time to contribute to Spack! -->



Discussion:
@ravil-mobile This is a case of a non-informative error message :slightly_frowning_face: What happens is that Spack has a built-in knowledge of which package manager to use for Ubuntu or CentOS, but is not able to guess for `nvidia`. The solution should be just a one line addition to the configuration:
```yaml
    os_packages:
      command: apt
      final:
      - libgomp
```
Let me know if that solves your problem. In any case I'd leave this issue open as a reminder to improve the error message.
Hi, thanks a lot. It works! *command* clause is actually logical. Maybe it will be a good idea to add this case to the documentation?

Btw, as far as I can remember, you implemented this feature i.e., "Custom Images Provided by Users". So, you would like to thank you very much. It is really what I was looking for!
@ravil-mobile Thanks, glad to know that the feature is useful! In case you are interested we have a few extensions planned, which are being discussed here: https://github.com/spack/spack/discussions/20442

Resolving patch:
diff --git a/lib/spack/spack/container/writers/__init__.py b/lib/spack/spack/container/writers/__init__.py
--- a/lib/spack/spack/container/writers/__init__.py
+++ b/lib/spack/spack/container/writers/__init__.py
@@ -10,10 +10,12 @@
 from typing import Optional
 
 import spack.environment as ev
+import spack.error
 import spack.schema.env
 import spack.tengine as tengine
 import spack.util.spack_yaml as syaml
-from spack.container.images import (
+
+from ..images import (
     bootstrap_template_for,
     build_info,
     checkout_command,
@@ -205,12 +207,20 @@ def manifest(self):
     @tengine.context_property
     def os_packages_final(self):
         """Additional system packages that are needed at run-time."""
-        return self._os_packages_for_stage("final")
+        try:
+            return self._os_packages_for_stage("final")
+        except Exception as e:
+            msg = f"an error occurred while rendering the 'final' stage of the image: {e}"
+            raise spack.error.SpackError(msg) from e
 
     @tengine.context_property
     def os_packages_build(self):
         """Additional system packages that are needed at build-time."""
-        return self._os_packages_for_stage("build")
+        try:
+            return self._os_packages_for_stage("build")
+        except Exception as e:
+            msg = f"an error occurred while rendering the 'build' stage of the image: {e}"
+            raise spack.error.SpackError(msg) from e
 
     @tengine.context_property
     def os_package_update(self):
@@ -243,13 +253,24 @@ def _package_info_from(self, package_list):
         if image is None:
             os_pkg_manager = os_package_manager_for(image_config["os"])
         else:
-            os_pkg_manager = self.container_config["os_packages"]["command"]
+            os_pkg_manager = self._os_pkg_manager()
 
         update, install, clean = commands_for(os_pkg_manager)
 
         Packages = collections.namedtuple("Packages", ["update", "install", "list", "clean"])
         return Packages(update=update, install=install, list=package_list, clean=clean)
 
+    def _os_pkg_manager(self):
+        try:
+            os_pkg_manager = self.container_config["os_packages"]["command"]
+        except KeyError:
+            msg = (
+                "cannot determine the OS package manager to use.\n\n\tPlease add an "
+                "appropriate 'os_packages:command' entry to the spack.yaml manifest file\n"
+            )
+            raise spack.error.SpackError(msg)
+        return os_pkg_manager
+
     @tengine.context_property
     def extra_instructions(self):
         Extras = collections.namedtuple("Extra", ["build", "final"])


Test patch: