Repository: spack/spack

Issue:
Clingo concretizer does not support non-numeric versions
I ran `spack spec` to concretize a package and was unable to concretize using non-numeric compiler or dependent package versions.

### Steps to reproduce the issue

```console
$ spack spec -I hdf5%clang@upstream_gfortran
==> [2020-11-23-10:49:21.285015] The following constraints are unsatisfiable:
  node_compiler_version_hard("hdf5","clang","upstream_gfortran")
  root("hdf5")
```
If the clang version is changed to a fake version (15.7), concretization without MPI succeeds.
But when MPI is added:
```console
$ spack spec -I hdf5+mpi%clang@15.7
==> [2020-11-23-10:50:54.382014] The following constraints are unsatisfiable:
  # a bunch of external_only facts for other MPI providers
  external_only("spectrum-mpi")
  external_version_declared("spectrum-mpi","release",-1,0)
  external_version_declared("spectrum-mpi","release",-2,1)
 # ... other possible_provider clauses for other MPI providers
  node("spectrum-mpi") :- possible_provider("mpi","spectrum-mpi",20).
  root("hdf5")
  single_provider_for("mpi",":") :- real_node("hdf5"),node("hdf5"),variant_value("hdf5","mpi","True").
  variant("hdf5","mpi")
  variant_set("hdf5","mpi","True")
```
Concretization succeeds when the external spectrum-mpi is also converted to use a fake numeric version number.

### Error Message

Above

### Information on your system

* **Spack:** 0.16.0-23-e75b76f43
* **Python:** 3.7.2
* **Platform:** linux-rhel7-power9le
* **Concretizer:** clingo

```yaml
compilers:
- compiler:
    # ...
    spec: clang@upstream_gfortran
```
```yaml
packages:
mpi:
    buildable: false
  spectrum-mpi:
    externals:
      - spec: spectrum-mpi@release%clang@upstream_gfortran
        prefix: /path/to/mpi
```

### Additional information

<!-- These boxes can be checked by replacing [ ] with [x] or by clicking them after submitting the issue. -->
- [x] I have run `spack debug report` and reported the version of Spack/Python/Platform
- [x] I have searched the issues of this repo and believe this is not a duplicate
- [x] I have run the failing commands in debug mode and reported the output

<!-- We encourage you to try, as much as possible, to reduce your problem to the minimal example that still reproduces the issue. That would help us a lot in fixing it quickly and effectively!

If you want to ask a question about the tool (how to use it, what it can currently do, etc.), try the `#general` channel on our Slack first. We have a welcoming community and chances are you'll get your reply faster and without opening an issue.

Other than that, thanks for taking the time to contribute to Spack! -->



Discussion:
Thanks for the report. I can reproduce with a minimal example using this `compilers.yaml`:
```yaml
compilers:
- compiler:
    spec: clang@upstream_fortran
    paths:
      cc: /usr/bin/clang-3.9
      cxx: /usr/bin/clang++-3.9
      f77:
      fc:
    operating_system: ubuntu18.04
    target: x86_64
```
and trying to concretize:
```console
$ spack solve zlib%clang@upstream_fortran
/home/culpo/PycharmProjects/spack/lib/spack/spack/solver/concretize.lp:302:8-59: info: atom does not occur in any rule head:
  compiler_supports_target(Compiler,Version,Target)

==> The following constraints are unsatisfiable:
  root("zlib")
==> Error: Unsatisfiable spec.
```
The issue is that with the custom version the concretizer cannot determine which target microarchitectures are supported by the compiler. 
I'm going to submit a hotfix that will use the generic target for custom versions.
Thanks for looking into this - would specifying the microarchitecture in the spec (something like "clang@upstream_fortran arch=x86_64-haswell") be enough?
> Thanks for looking into this - would specifying the microarchitecture in the spec (something like "clang@upstream_fortran arch=x86_64-haswell") be enough?

I don't think so. I'm checking the code right now, but I'm mostly confident that this stems from Spack using `upstream_gfortran` to lookup this JSON file:

https://github.com/spack/spack/blob/develop/lib/spack/external/archspec/json/cpu/microarchitectures.json

and check which microarchitectures the compiler supports. Of course, being a custom version, the result is none and the solver is basically saying that it can't select which microarchitecture to use for the software because the compiler supports none :slightly_smiling_face: 

Resolving patch:
diff --git a/lib/spack/spack/solver/asp.py b/lib/spack/spack/solver/asp.py
--- a/lib/spack/spack/solver/asp.py
+++ b/lib/spack/spack/solver/asp.py
@@ -31,6 +31,7 @@
 import spack
 import spack.architecture
 import spack.cmd
+import spack.compilers
 import spack.config
 import spack.dependency
 import spack.error
@@ -829,6 +830,18 @@ def compiler_defaults(self):
             f = fn.default_compiler_preference(cspec.name, cspec.version, i)
             self.gen.fact(f)
 
+        # Enumerate target families. This may be redundant, but compilers with
+        # custom versions will be able to concretize properly.
+        for entry in spack.compilers.all_compilers_config():
+            compiler_entry = entry['compiler']
+            cspec = spack.spec.CompilerSpec(compiler_entry['spec'])
+            if not compiler_entry.get('target', None):
+                continue
+
+            self.gen.fact(fn.compiler_supports_target(
+                cspec.name, cspec.version, compiler_entry['target']
+            ))
+
     def compiler_supports_os(self):
         compilers_yaml = spack.compilers.all_compilers_config()
         for entry in compilers_yaml:
@@ -1230,7 +1243,7 @@ def build_version_dict(self, possible_pkgs, specs):
                 if dep.versions.concrete:
                     self.possible_versions[dep.name].add(dep.version)
 
-    def _supported_targets(self, compiler, targets):
+    def _supported_targets(self, compiler_name, compiler_version, targets):
         """Get a list of which targets are supported by the compiler.
 
         Results are ordered most to least recent.
@@ -1239,7 +1252,7 @@ def _supported_targets(self, compiler, targets):
 
         for target in targets:
             try:
-                target.optimization_flags(compiler.name, compiler.version)
+                target.optimization_flags(compiler_name, compiler_version)
                 supported.append(target)
             except archspec.cpu.UnsupportedMicroarchitecture:
                 continue
@@ -1289,7 +1302,22 @@ def target_defaults(self, specs):
         # TODO: investigate this.
         best_targets = set([uarch.family.name])
         for compiler in sorted(compilers):
-            supported = self._supported_targets(compiler, compatible_targets)
+            supported = self._supported_targets(
+                compiler.name, compiler.version, compatible_targets
+            )
+
+            # If we can't find supported targets it may be due to custom
+            # versions in the spec, e.g. gcc@foo. Try to match the
+            # real_version from the compiler object to get more accurate
+            # results.
+            if not supported:
+                compiler_obj = spack.compilers.compilers_for_spec(compiler)
+                compiler_obj = compiler_obj[0]
+                supported = self._supported_targets(
+                    compiler.name,
+                    compiler_obj.real_version,
+                    compatible_targets
+                )
 
             if not supported:
                 continue


Test patch:
diff --git a/lib/spack/spack/test/concretize.py b/lib/spack/spack/test/concretize.py
--- a/lib/spack/spack/test/concretize.py
+++ b/lib/spack/spack/test/concretize.py
@@ -959,3 +959,11 @@ def test_variant_not_default(self):
         # Check that non-default variant values are forced on the dependency
         d = s['dep-with-variants']
         assert '+foo+bar+baz' in d
+
+    @pytest.mark.regression('20055')
+    def test_custom_compiler_version(self):
+        if spack.config.get('config:concretizer') == 'original':
+            pytest.xfail('Known failure of the original concretizer')
+
+        s = Spec('a %gcc@foo os=redhat6').concretized()
+        assert '%gcc@foo' in s
diff --git a/lib/spack/spack/test/data/config/compilers.yaml b/lib/spack/spack/test/data/config/compilers.yaml
--- a/lib/spack/spack/test/data/config/compilers.yaml
+++ b/lib/spack/spack/test/data/config/compilers.yaml
@@ -8,6 +8,7 @@ compilers:
       f77: None
       fc: None
     modules: 'None'
+    target: x86_64
 - compiler:
     spec: gcc@4.5.0
     operating_system: {0.name}{0.version}
@@ -17,6 +18,7 @@ compilers:
       f77: None
       fc: None
     modules: 'None'
+    target: x86_64
 - compiler:
     spec: clang@3.3
     operating_system: CNL
@@ -35,6 +37,7 @@ compilers:
       f77: None
       fc: None
     modules: 'None'
+    target: x86_64
 - compiler:
     spec: clang@3.3
     operating_system: yosemite
@@ -44,6 +47,7 @@ compilers:
       f77: None
       fc: None
     modules: 'None'
+    target: x86_64
 - compiler:
     paths:
       cc: /path/to/gcc
@@ -62,6 +66,7 @@ compilers:
     operating_system: SuSE11
     spec: gcc@4.5.0
     modules: 'None'
+    target: x86_64
 - compiler:
     paths:
       cc: /path/to/gcc
@@ -71,6 +76,7 @@ compilers:
     operating_system: yosemite
     spec: gcc@4.5.0
     modules: 'None'
+    target: x86_64
 - compiler:
     paths:
       cc: /path/to/gcc
@@ -80,6 +86,7 @@ compilers:
     operating_system: elcapitan
     spec: gcc@4.5.0
     modules: 'None'
+    target: x86_64
 - compiler:
     spec: clang@3.3
     operating_system: elcapitan
@@ -89,6 +96,7 @@ compilers:
       f77: None
       fc: None
     modules: 'None'
+    target: x86_64
 - compiler:
     spec: gcc@4.7.2
     operating_system: redhat6
@@ -102,6 +110,7 @@ compilers:
       cxxflags: -O0 -g
       fflags: -O0 -g
     modules: 'None'
+    target: x86_64
 - compiler:
     spec: gcc@4.4.0
     operating_system: redhat6
@@ -123,6 +132,7 @@ compilers:
       cflags: -O3
       cxxflags: -O3
     modules: 'None'
+    target: x86_64
 - compiler:
     spec: clang@8.0.0
     operating_system: redhat7
@@ -135,6 +145,7 @@ compilers:
       cflags: -O3
       cxxflags: -O3
     modules: 'None'
+    target: x86_64
 - compiler:
     spec: apple-clang@9.1.0
     operating_system: elcapitan
@@ -144,6 +155,7 @@ compilers:
       f77: None
       fc: None
     modules: 'None'
+    target: x86_64
 - compiler:
     spec: gcc@foo
     operating_system: redhat6
@@ -153,6 +165,7 @@ compilers:
       f77: /path/to/gfortran
       fc: /path/to/gfortran
     modules: 'None'
+    target: x86_64
 - compiler:
     spec: gcc@4.4.0-special
     operating_system: redhat6
@@ -162,3 +175,4 @@ compilers:
       f77: /path/to/gfortran
       fc: /path/to/gfortran
     modules: 'None'
+    target: x86_64