Repository: spack/spack

Issue:
Cannot set default compiler in packages.yaml
In the [Spack Configuration Documentation](http://software.llnl.gov/spack/configuration.html#concretization-preferences), it shows an example where you can set the default compiler for a package. However, this doesn't seem to work. Libpciaccess and OpenSSL cannot be built with the PGI compilers. I tried adding the following to my `~/.spack/packages.yaml`:

``` yaml
packages:
  libpciaccess:
    compiler: [gcc, intel, nag]
  openssl:
    compiler: [gcc, intel, nag]
```

The behavior I expect is that if I build `curl`, it will build everything with GCC. If I build `curl %intel`, it will build everything with Intel. And if I build `curl %pgi`, it will build curl with PGI but build OpenSSL with GCC. However, I get this behavior:

```
$ spack spec curl%pgi

  curl@7.49.1%pgi@16.5-0 arch=linux-centos6-x86_64
      ^openssl@1.0.2h%pgi@16.5-0 arch=linux-centos6-x86_64
          ^zlib@1.2.8%pgi@16.5-0 arch=linux-centos6-x86_64
```

Even when I set the default compiler to intel:

``` yaml
packages:
  libpciaccess:
    compiler: [intel]
  openssl:
    compiler: [intel]
```

I get:

```
$ spack spec curl

  curl@7.49.1%gcc@6.1.0 arch=linux-centos6-x86_64
      ^openssl@1.0.2h%gcc@6.1.0 arch=linux-centos6-x86_64
          ^zlib@1.2.8%gcc@6.1.0 arch=linux-centos6-x86_64
```

If I specifically try to build OpenSSL, then I get the right behavior:

```
$ spack spec openssl

  openssl@1.0.2h%intel@16.0.3 arch=linux-centos6-x86_64
      ^zlib@1.2.8%intel@16.0.3 arch=linux-centos6-x86_64
```

But this still doesn't help me solve the problem of trying to build OpenMPI and NetCDF with PGI.



Discussion:
@tgamblin @mathstuf could you have a look at this issue? 

I don't see a test for this feature, so I'm not surprised it may have broken with my changes or any other concretization changes. I don't remember anywhere looking for the default compiler based on the package, but it may have changed since I was last in there.

Since this has been broken for more than half a year, should we add a note to the documentation mentioning this? See [http://spack.readthedocs.io/en/latest/build_settings.html#concretization-preferences](url).
@eschnett That's what I was suggesting in #4201. @davydden added a [known issues](https://spack.readthedocs.io/en/latest/known_issues.html) section a while ago that's quite helpful for informing people of longstanding bugs.
fix for this will help to avoid complicated spec dependency specification on command line!
I cannot reproduce this:
```
$ spack compilers
==> Available compilers
-- clang debian8-x86_64 -----------------------------------------
clang@5.0.1

-- gcc debian8-x86_64 -------------------------------------------
gcc@7.2.0  gcc@4.9.2  gcc@4.8
$ cat packages.yaml
packages:                                                                                                                          
  all:                                                                                                                             
     compiler: [gcc@7.2.0] 
[..]
$ spack spec -I curl
Input spec
--------------------------------
     curl

Concretized
--------------------------------
     curl@7.56.0%gcc@7.2.0~nghttp2 arch=linux-debian8-x86_64 
         ^openssl@1.0.2n%gcc@7.2.0 arch=linux-debian8-x86_64 
             ^zlib@1.2.11%gcc@7.2.0+optimize+pic+shared arch=linux-debian8-x86_64 
$ cat packages.yaml
packages:                                                                                                                          
  all:                                                                                                                             
     compiler: [gcc@4.9.2] 
[..]
$ spack spec curl
Input spec
--------------------------------
curl

Concretized
--------------------------------
curl@7.56.0%gcc@4.9.2~nghttp2 arch=linux-debian8-x86_64 
    ^openssl@1.0.2n%gcc@4.9.2 arch=linux-debian8-x86_64 
        ^zlib@1.2.11%gcc@4.9.2+optimize+pic+shared arch=linux-debian8-x86_64 
```
I think you're missing the point...

### Steps to reproduce

1. Set the default compiler for a **specific package** to something different than the default compiler

```console
$ spack compilers
==> Available compilers
-- clang highsierra-x86_64 --------------------------------------
clang@9.0.0-apple

-- gcc highsierra-x86_64 ----------------------------------------
gcc@7.2.0
$ cat ~/.spack/packages.yaml
```
```yaml
packages:
    openssl:
        compiler: [gcc]
```
Note: example is on macOS where default compiler is normally clang.

2. Try to build something **that depends on this package**

### Expected Result

Spack should build `openssl` with `%gcc`, not `%clang`. This is important for packages that cannot be built with certain compilers.

### Actual Result

Spack is using the default compiler instead of the one set for `openssl`.
```console
$ spack spec curl
...
curl@7.56.0%clang@9.0.0-apple+libssh2~nghttp2 arch=darwin-highsierra-x86_64 
    ^libssh2@1.8.0%clang@9.0.0-apple build_type=RelWithDebInfo +shared arch=darwin-highsierra-x86_64 
        ^cmake@3.9.6%clang@9.0.0-apple~doc+ncurses+openssl+ownlibs~qt arch=darwin-highsierra-x86_64 
        ^openssl@1.0.2n%clang@9.0.0-apple+systemcerts arch=darwin-highsierra-x86_64 
            ^zlib@1.2.11%clang@9.0.0-apple+optimize+pic+shared arch=darwin-highsierra-x86_64 
        ^xz@5.2.3%clang@9.0.0-apple arch=darwin-highsierra-x86_64 
```
@adamjstewart is this issue still active?
Issue is still present with the following config in develop:
```yaml
packages:
  all:
    compiler: [apple-clang]
  zlib:
    compiler: [gcc]
```
```console
$ spack spec --fresh curl
Input spec
--------------------------------
curl

Concretized
--------------------------------
curl@7.84.0%gcc@11.3.0~gssapi~ldap~libidn2~librtmp~libssh~libssh2~nghttp2 libs=shared,static patches=9b011c9 tls=secure_transport arch=darwin-monterey-m1
    ^gnuconfig@2021-08-14%gcc@11.3.0 arch=darwin-monterey-m1
    ^pkgconf@1.8.0%gcc@11.3.0 arch=darwin-monterey-m1
    ^zlib@1.2.12%gcc@11.3.0+optimize+pic+shared patches=0d38234 arch=darwin-monterey-m1

$ spack spec --fresh curl%apple-clang
Input spec
--------------------------------
curl%apple-clang

Concretized
--------------------------------
curl@7.84.0%apple-clang@13.1.6~gssapi~ldap~libidn2~librtmp~libssh~libssh2~nghttp2 libs=shared,static patches=9b011c9 tls=secure_transport arch=darwin-monterey-m1
    ^gnuconfig@2021-08-14%apple-clang@13.1.6 arch=darwin-monterey-m1
    ^pkgconf@1.8.0%apple-clang@13.1.6 arch=darwin-monterey-m1
    ^zlib@1.2.12%apple-clang@13.1.6+optimize+pic+shared patches=0d38234 arch=darwin-monterey-m1
```
Let me try with the new requires syntax.
Issue is still present with require, and I think require is actually broken. With:
```yaml
packages:
  all:
    require: "%apple-clang"
  zlib:
    require: "%gcc"
```
I would expect to see zlib build with gcc and everything else build with apple-clang, but I don't see that:
```console
$ spack spec --fresh curl
Input spec
--------------------------------
curl

Concretized
--------------------------------
curl@7.84.0%gcc@11.3.0~gssapi~ldap~libidn2~librtmp~libssh~libssh2~nghttp2 libs=shared,static patches=9b011c9 tls=secure_transport arch=darwin-monterey-m1
    ^gnuconfig@2021-08-14%gcc@11.3.0 arch=darwin-monterey-m1
    ^pkgconf@1.8.0%gcc@11.3.0 arch=darwin-monterey-m1
    ^zlib@1.2.12%gcc@11.3.0+optimize+pic+shared patches=0d38234 arch=darwin-monterey-m1
```
I still have to manually set the default compiler on the command-line:
```console
$ spack spec --fresh curl%apple-clang
Input spec
--------------------------------
curl%apple-clang

Concretized
--------------------------------
curl@7.84.0%apple-clang@13.1.6~gssapi~ldap~libidn2~librtmp~libssh~libssh2~nghttp2 libs=shared,static patches=9b011c9 tls=secure_transport arch=darwin-monterey-m1
    ^gnuconfig@2021-08-14%apple-clang@13.1.6 arch=darwin-monterey-m1
    ^pkgconf@1.8.0%apple-clang@13.1.6 arch=darwin-monterey-m1
    ^zlib@1.2.12%gcc@11.3.0+optimize+pic+shared patches=0d38234 arch=darwin-monterey-m1
```
Currently:
> you can only specify requirements for specific packages: you cannot add requires under all.

see [docs](https://spack.readthedocs.io/en/latest/build_settings.html#package-requirements). There will be work to lift this restriction and also to allow `requires` under virtual packages.

Resolving patch:
diff --git a/lib/spack/spack/solver/asp.py b/lib/spack/spack/solver/asp.py
--- a/lib/spack/spack/solver/asp.py
+++ b/lib/spack/spack/solver/asp.py
@@ -930,7 +930,9 @@ def package_compiler_defaults(self, pkg):
     def package_requirement_rules(self, pkg):
         pkg_name = pkg.name
         config = spack.config.get("packages")
-        requirements = config.get(pkg_name, {}).get("require", [])
+        requirements = config.get(pkg_name, {}).get("require", []) or config.get("all", {}).get(
+            "require", []
+        )
         if isinstance(requirements, string_types):
             rules = [(pkg_name, "one_of", [requirements])]
         else:


Test patch:
diff --git a/lib/spack/spack/test/concretize_requirements.py b/lib/spack/spack/test/concretize_requirements.py
--- a/lib/spack/spack/test/concretize_requirements.py
+++ b/lib/spack/spack/test/concretize_requirements.py
@@ -297,3 +297,55 @@ def test_requirements_are_higher_priority_than_deprecation(concretize_scope, tes
     s1 = Spec("y").concretized()
     assert s1.satisfies("@2.3")
     assert s1.satisfies("%gcc")
+
+
+@pytest.mark.parametrize("spec_str,requirement_str", [("x", "%gcc"), ("x", "%clang")])
+def test_default_requirements_with_all(spec_str, requirement_str, concretize_scope, test_repo):
+    """Test that default requirements are applied to all packages."""
+    if spack.config.get("config:concretizer") == "original":
+        pytest.skip("Original concretizer does not support configuration" " requirements")
+
+    conf_str = """\
+packages:
+  all:
+    require: "{}"
+""".format(
+        requirement_str
+    )
+    update_packages_config(conf_str)
+
+    spec = Spec(spec_str).concretized()
+    for s in spec.traverse():
+        assert s.satisfies(requirement_str)
+
+
+@pytest.mark.parametrize(
+    "requirements,expectations",
+    [
+        (("%gcc", "%clang"), ("%gcc", "%clang")),
+        (("%gcc ~shared", "@1.0"), ("%gcc ~shared", "@1.0 +shared")),
+    ],
+)
+def test_default_and_package_specific_requirements(
+    concretize_scope, requirements, expectations, test_repo
+):
+    """Test that specific package requirements override default package requirements."""
+    if spack.config.get("config:concretizer") == "original":
+        pytest.skip("Original concretizer does not support configuration" " requirements")
+    generic_req, specific_req = requirements
+    generic_exp, specific_exp = expectations
+    conf_str = """\
+packages:
+  all:
+    require: "{}"
+  x:
+    require: "{}"
+""".format(
+        generic_req, specific_req
+    )
+    update_packages_config(conf_str)
+
+    spec = Spec("x").concretized()
+    assert spec.satisfies(specific_exp)
+    for s in spec.traverse(root=False):
+        assert s.satisfies(generic_exp)