Repository: spack/spack

Issue:
Misleading concretizer error message: no valid version satisfies
### Steps to reproduce

This issue occurs any time you add a new version to a package, and that version requires new versions of dependencies that aren't in Spack. If you update a lot of recipes like me, that means almost every time.

To reproduce the issue, try something like this:
```diff
diff --git a/var/spack/repos/builtin/packages/python/package.py b/var/spack/repos/builtin/packages/python/package.py
index a579358897..0e93ffefb4 100644
--- a/var/spack/repos/builtin/packages/python/package.py
+++ b/var/spack/repos/builtin/packages/python/package.py
@@ -44,6 +44,7 @@ class Python(Package):
     install_targets = ['install']
     build_targets = []
 
+    version('999', sha256='f3bcc65b1d5f1dc78675c746c98fcee823c038168fc629c5935b044d0911ad28')
     version('3.10.4', sha256='f3bcc65b1d5f1dc78675c746c98fcee823c038168fc629c5935b044d0911ad28')
     version('3.10.3', sha256='5a3b029bad70ba2a019ebff08a65060a8b9b542ffc1a83c697f1449ecca9813b')
     version('3.10.2', sha256='3c0ede893011319f9b0a56b44953a3d52c7abf9657c23fb4bc9ced93b86e9c97')
@@ -206,6 +207,7 @@ class Python(Package):
         depends_on('sqlite@3.7.15:', when='@3.10:+sqlite3')  # https://docs.python.org/3.10/whatsnew/3.10.html#build-changes
         depends_on('gdbm', when='+dbm')  # alternatively ndbm or berkeley-db
         depends_on('libnsl', when='+nis')
+        depends_on('zlib@999', when='@999')
         depends_on('zlib@1.1.3:', when='+zlib')
         depends_on('bzip2', when='+bz2')
         depends_on('xz', when='@3.3:+lzma')
```
Obviously zlib@999 doesn't exist in Spack, so the concretizer should complain if you try to build python@999.

### Error message

I would love to see an error message similar to:
```console
$ spack audit packages python
PKG-DIRECTIVES: 1 issue found
1. python: dependency on zlib@999 cannot be satisfied by known versions of zlib
    happening in /Users/ajstewart/spack/var/spack/repos/builtin/packages/python/package.py
    known versions of zlib are 1.2.12, 1.2.11, 1.2.8, 1.2.3
```
However, I instead see a really misleading error message that leads me to believe that python@999 doesn't exist (it does, zlib@999 is the issue):
```console
$ spack spec python@999
==> Error: No valid version for 'python' satisfies '@999'
```
Full stack trace:
```console
$ spack -d spec python@999
...
==> [2022-05-25-16:00:28.247629] UnsatisfiableSpecError: No valid version for 'python' satisfies '@999'
==> [2022-05-25-16:00:28.247697] Error: No valid version for 'python' satisfies '@999'
Traceback (most recent call last):
  File "/Users/ajstewart/spack/lib/spack/spack/main.py", line 893, in main
    return _main(argv)
  File "/Users/ajstewart/spack/lib/spack/spack/main.py", line 848, in _main
    return finish_parse_and_run(parser, cmd_name, env_format_error)
  File "/Users/ajstewart/spack/lib/spack/spack/main.py", line 876, in finish_parse_and_run
    return _invoke_command(command, parser, args, unknown)
  File "/Users/ajstewart/spack/lib/spack/spack/main.py", line 533, in _invoke_command
    return_val = command(parser, args)
  File "/Users/ajstewart/spack/lib/spack/spack/cmd/spec.py", line 83, in spec
    specs = [(s, s.concretized()) for s in input_specs]
  File "/Users/ajstewart/spack/lib/spack/spack/cmd/spec.py", line 83, in <listcomp>
    specs = [(s, s.concretized()) for s in input_specs]
  File "/Users/ajstewart/spack/lib/spack/spack/spec.py", line 3025, in concretized
    clone.concretize(tests=tests)
  File "/Users/ajstewart/spack/lib/spack/spack/spec.py", line 2929, in concretize
    self._new_concretize(tests)
  File "/Users/ajstewart/spack/lib/spack/spack/spec.py", line 2903, in _new_concretize
    result = solver.solve([self], tests=tests)
  File "/Users/ajstewart/spack/lib/spack/spack/solver/asp.py", line 2280, in solve
    return self.driver.solve(
  File "/Users/ajstewart/spack/lib/spack/spack/solver/asp.py", line 680, in solve
    answers = builder.build_specs(tuples)
  File "/Users/ajstewart/spack/lib/spack/spack/solver/asp.py", line 2137, in build_specs
    action(*args)
  File "/Users/ajstewart/spack/lib/spack/spack/solver/asp.py", line 1963, in error
    raise UnsatisfiableSpecError(msg)
spack.solver.asp.UnsatisfiableSpecError: No valid version for 'python' satisfies '@999'
```

### Information on your system

* **Spack:** 0.18.0.dev0 (14d4203722daf3abd56a2b6c880214a1338e289f)
* **Python:** 3.9.12
* **Platform:** darwin-monterey-m1
* **Concretizer:** clingo


### General information

- [X] I have run `spack debug report` and reported the version of Spack/Python/Platform
- [X] I have searched the issues of this repo and believe this is not a duplicate
- [X] I have run the failing commands in debug mode and reported the output


Discussion:
With this diff:
```diff
diff --git a/var/spack/repos/builtin/packages/hdf5/package.py b/var/spack/repos/builtin/packages/hdf5/package.py
index 45e00c0f6b..be9483f73b 100644
--- a/var/spack/repos/builtin/packages/hdf5/package.py
+++ b/var/spack/repos/builtin/packages/hdf5/package.py
@@ -36,6 +36,9 @@ class Hdf5(CMakePackage):
     version('develop-1.10', branch='hdf5_1_10')
     version('develop-1.8', branch='hdf5_1_8')
 
+    version('20', sha256='051655873105112f7aeccd5f59ab21f35f7f4907f06921ae61aaf1ef1c71fd53')
+    depends_on('zlib@20', when='@20')
+
     # Odd versions are considered experimental releases
     version('1.13.1', sha256='051655873105112f7aeccd5f59ab21f35f7f4907f06921ae61aaf1ef1c71fd53')
     version('1.13.0', sha256='3049faf900f0c52e09ea4cddfb83af057615f2fc1cc80eb5202dd57b09820115')
```
on top of:

* **Spack:** 0.19.0.dev0 (8c58c14c3d77420c86b0ae852bc4a680c2aca472)
* **Python:** 3.8.10
* **Platform:** linux-ubuntu20.04-icelake
* **Concretizer:** clingo

I get:
```console
$ spack spec hdf5@20
==> Error: No valid version for 'zlib' satisfies '@20'
```
which seems correct.
Using this diff instead:
```diff
diff --git a/var/spack/repos/builtin/packages/python/package.py b/var/spack/repos/builtin/packages/python/package.py
index 44c364b652..d82d016cca 100644
--- a/var/spack/repos/builtin/packages/python/package.py
+++ b/var/spack/repos/builtin/packages/python/package.py
@@ -44,6 +44,8 @@ class Python(Package):
     install_targets = ['install']
     build_targets = []  # type: List[str]
 
+    version('999', sha256='18f57182a2de3b0be76dfc39fdcfd28156bb6dd23e5f08696f7492e9e3d0bf2d')
+    depends_on('zlib@999', when='@999')
     version('3.10.5', sha256='18f57182a2de3b0be76dfc39fdcfd28156bb6dd23e5f08696f7492e9e3d0bf2d')
     version('3.10.4', sha256='f3bcc65b1d5f1dc78675c746c98fcee823c038168fc629c5935b044d0911ad28')
     version('3.10.3', sha256='5a3b029bad70ba2a019ebff08a65060a8b9b542ffc1a83c697f1449ecca9813b')
```
(which is exactly what is reported) I can reproduce:
```console
$ spack spec python@999
==> Error: No valid version for 'python' satisfies '@999'
```
Solving the ASP problem gives (showing only errors here):
```console
$ clingo --configuration=tweety --opt-strategy=usc,one concretize.lp python.lp display.lp 
clingo version 5.5.0
Reading from concretize.lp ...
Solving...
Answer: 1
error(1,"No valid version for '{0}' satisfies '@{1}'","python","999") error(1,"No valid version for '{0}' satisfies '@{1}'","openssl",":1") error(1,"No valid version for '{0}' satisfies '@{1}'","gdbm",":1.19") error(1,"No valid version for '{0}' satisfies '@{1}'","pkgconfig","0.9.0:") error(1,"No valid version for '{0}' satisfies '@{1}'","zlib","999") error(1,"No valid version for '{0}' satisfies '@{1}'","xcb-proto","1.14:")
Optimization: 0 6000 0 0 0 4 14 0 14 18 300 0 0 0 146 54 0 0 1830 61 0 0 0 0 0 1 0 0 0 0 0 1 0 0 240
Progression : 0 [1000;6000] (Error: 5)
Answer: 2
error(1,"No valid version for '{0}' satisfies '@{1}'","zlib","999")
Optimization: 0 1000 0 0 1 1 14 0 14 18 300 0 0 0 169 54 0 0 1830 61 0 0 0 0 0 1 0 0 0 0 0 1 0 0 240
Answer: 3
error(1,"No valid version for '{0}' satisfies '@{1}'","zlib","999")
Optimization: 0 1000 0 0 0 1 14 0 14 18 300 0 0 0 148 54 0 0 1830 61 0 0 0 0 0 1 0 0 0 0 0 1 0 0 240
Answer: 4
error(1,"No valid version for '{0}' satisfies '@{1}'","python","999")
Optimization: 0 1000 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 2 0 0 0 0 4 2 0 0 600
OPTIMUM FOUND

Models       : 4
  Optimum    : yes
Optimization : 0 1000 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 2 0 0 0 0 4 2 0 0 600
Calls        : 1
Time         : 0.721s (Solving: 0.05s 1st Model: 0.02s Unsat: 0.01s)
CPU Time     : 0.721s
```
[python.lp.txt](https://github.com/spack/spack/files/8908618/python.lp.txt)

So to explain this issue:
1. To get better error messages, we allow for valid solutions with errors - and optimize towards having 0 errors
2. If there are errors in the final solution, those will be reported by the spec builder on the Python side

To operate like that we have to code some invariants as optimizations. For instance we cannot say that each node in the solution has a single version:
```
1 { version(Package, Version) : node(Package) } 1.
```
but instead we have to give choices to the optimizer to select any version among the possible ones, and say that if there is more than one or if there is none those are errors.

The optimizations towards having zero errors are done together with the (lower priority) optimization to get the "optimal" spec. In the case of `python@999` above it seems the optimizer had a better score for the objective function by pretending it couldn't select a version for Python(optimal Answer 4) rather than saying that it couldn't select a version for `zlib` (Answer 2 and Answer 3).

Resolving patch:
diff --git a/var/spack/repos/builtin.mock/packages/non-existing-conditional-dep/package.py b/var/spack/repos/builtin.mock/packages/non-existing-conditional-dep/package.py
new file mode 100644
--- /dev/null
+++ b/var/spack/repos/builtin.mock/packages/non-existing-conditional-dep/package.py
@@ -0,0 +1,16 @@
+# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
+# Spack Project Developers. See the top-level COPYRIGHT file for details.
+#
+# SPDX-License-Identifier: (Apache-2.0 OR MIT)
+from spack.package import *
+
+
+class NonExistingConditionalDep(Package):
+    """Simple package with no source and one dependency"""
+
+    homepage = "http://www.example.com"
+
+    version('2.0')
+    version('1.0')
+
+    depends_on('dep-with-variants@999', when='@2.0')


Test patch:
diff --git a/lib/spack/spack/test/concretize.py b/lib/spack/spack/test/concretize.py
--- a/lib/spack/spack/test/concretize.py
+++ b/lib/spack/spack/test/concretize.py
@@ -1732,3 +1732,24 @@ def test_best_effort_coconcretize_preferences(
             if expected_spec in spec:
                 counter += 1
         assert counter == occurances, concrete_specs
+
+    @pytest.mark.regression('30864')
+    def test_misleading_error_message_on_version(self, mutable_database):
+        # For this bug to be triggered we need a reusable dependency
+        # that is not optimal in terms of optimization scores.
+        # We pick an old version of "b"
+        import spack.solver.asp
+        if spack.config.get('config:concretizer') == 'original':
+            pytest.skip('Original concretizer cannot reuse')
+
+        reusable_specs = [
+            spack.spec.Spec('non-existing-conditional-dep@1.0').concretized()
+        ]
+        root_spec = spack.spec.Spec('non-existing-conditional-dep@2.0')
+
+        with spack.config.override("concretizer:reuse", True):
+            solver = spack.solver.asp.Solver()
+            setup = spack.solver.asp.SpackSolverSetup()
+            with pytest.raises(spack.solver.asp.UnsatisfiableSpecError,
+                               match="'dep-with-variants' satisfies '@999'"):
+                solver.driver.solve(setup, [root_spec], reuse=reusable_specs)