Repository: spack/spack

Issue:
Bug: installing libpressio from cache fails because of existence of another package in the cache that isn't even in the dependency tree for libpressio
### Steps to reproduce

It looks like Spack is getting confused by the presence of some packages in our build cache because they no longer exist in Spack. But what is odd is that the particular package Spack complains about being in the cache -- here it is `netcdf` -- isn't even in the dependency graph for the package I'm trying to install.

I'm using:
* Docker container image `ecpe4s/ubuntu20.04-runner-x86_64:2022-11-01`
* spack@develop (d56380fc0715a7ae8fd264c7f2d8332ebb7f0c24 from `Wed Nov 23 12:39:45 2022 -0500`
* E4S build cache: `https://cache.e4s.io`

CC @robertu94 @wspear 

See:
```
$> spack mirror add E4S https://cache.e4s.io
$> spack buildcache keys -it

$> spack --debug --stacktrace install libpressio
...
lib/spack/spack/util/web.py:392 ==> [2022-11-23-10:43:06.080448] Checking existence of https://cache.e4s.io/build_cache/index.json
lib/spack/spack/binary_distribution.py:476 ==> [2022-11-23-10:43:06.908180] Cached index for https://cache.e4s.io already up to date
lib/spack/spack/util/web.py:392 ==> [2022-11-23-10:43:06.908939] Checking existence of https://mirror.spack.io/build_cache/index.json
lib/spack/spack/util/web.py:441 ==> [2022-11-23-10:43:06.938828] Failure reading URL: Download failed: HTTP Error 404: Not Found
lib/spack/spack/database.py:389 ==> [2022-11-23-10:43:06.940681] DATABASE LOCK TIMEOUT: 3s
lib/spack/spack/database.py:395 ==> [2022-11-23-10:43:06.940935] PACKAGE LOCK TIMEOUT: No timeout
...
lib/spack/spack/config.py:1002 ==> [2022-11-23-10:43:32.962367] Reading config from file /root/.spack/linux/compilers.yaml
lib/spack/spack/cmd/install.py:451 ==> [2022-11-23-10:44:06.489322] UnknownPackageError: Package 'netcdf' not found.
    You may need to run 'spack clean -m'.
lib/spack/spack/main.py:1003 ==> [2022-11-23-10:44:06.489528] UnknownPackageError: Package 'netcdf' not found.
    You may need to run 'spack clean -m'.
lib/spack/spack/error.py:54 ==> [2022-11-23-10:44:06.489740] Error: Package 'netcdf' not found.
You may need to run 'spack clean -m'.
Traceback (most recent call last):
  File "/spack/lib/spack/spack/repo.py", line 1229, in get_pkg_class
    module = importlib.import_module(fullname)
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 973, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'spack.pkg.builtin.netcdf'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/spack/lib/spack/spack/main.py", line 1000, in main
    return _main(argv)
  File "/spack/lib/spack/spack/main.py", line 955, in _main
    return finish_parse_and_run(parser, cmd_name, env_format_error)
  File "/spack/lib/spack/spack/main.py", line 983, in finish_parse_and_run
    return _invoke_command(command, parser, args, unknown)
  File "/spack/lib/spack/spack/main.py", line 632, in _invoke_command
    return_val = command(parser, args)
  File "/spack/lib/spack/spack/cmd/install.py", line 531, in install
    abstract_specs, concrete_specs = specs_from_cli(args, install_kwargs, reporter)
  File "/spack/lib/spack/spack/cmd/install.py", line 447, in specs_from_cli
    concrete_specs = spack.cmd.parse_specs(
  File "/spack/lib/spack/spack/cmd/__init__.py", line 227, in parse_specs
    spec.concretize(tests=tests)  # implies normalize
  File "/spack/lib/spack/spack/spec.py", line 2978, in concretize
    self._new_concretize(tests)
  File "/spack/lib/spack/spack/spec.py", line 2953, in _new_concretize
    result = solver.solve([self], tests=tests)
  File "/spack/lib/spack/spack/solver/asp.py", line 2435, in solve
    result, _, _ = self.driver.solve(setup, specs, reuse=reusable_specs, output=output)
  File "/spack/lib/spack/spack/solver/asp.py", line 625, in solve
    setup.setup(self, specs, reuse=reuse)
  File "/spack/lib/spack/spack/solver/asp.py", line 1989, in setup
    self._facts_from_concrete_spec(reusable_spec, possible)
  File "/spack/lib/spack/spack/solver/asp.py", line 1897, in _facts_from_concrete_spec
    self.impose(h, spec, body=True)
  File "/spack/lib/spack/spack/solver/asp.py", line 1090, in impose
    imposed_constraints = self.spec_clauses(imposed_spec, body=body, required_from=name)
  File "/spack/lib/spack/spack/solver/asp.py", line 1313, in spec_clauses
    clauses = self._spec_clauses(*args, **kwargs)
  File "/spack/lib/spack/spack/solver/asp.py", line 1478, in _spec_clauses
    for virtual in dep.package.virtuals_provided:
  File "/spack/lib/spack/spack/spec.py", line 1638, in package
    self._package = spack.repo.path.get(self)
  File "/spack/lib/spack/spack/repo.py", line 849, in get
    return self.repo_for_pkg(spec).get(spec)
  File "/spack/lib/spack/spack/repo.py", line 1042, in get
    package_class = self.get_pkg_class(spec.name)
  File "/spack/lib/spack/spack/repo.py", line 1231, in get_pkg_class
    raise UnknownPackageError(pkg_name)
spack.repo.UnknownPackageError: Package 'netcdf' not found.
    You may need to run 'spack clean -m'.
```


### Error message

see above

### Information on your system

* **Spack:** 0.20.0.dev0 (d56380fc0715a7ae8fd264c7f2d8332ebb7f0c24)
* **Python:** 3.8.10
* **Platform:** linux-ubuntu20.04-cascadelake
* **Concretizer:** clingo

### General information

- [X] I have run `spack debug report` and reported the version of Spack/Python/Platform
- [X] I have searched the issues of this repo and believe this is not a duplicate
- [X] I have run the failing commands in debug mode and reported the output


Discussion:

Resolving patch:
diff --git a/lib/spack/spack/solver/asp.py b/lib/spack/spack/solver/asp.py
--- a/lib/spack/spack/solver/asp.py
+++ b/lib/spack/spack/solver/asp.py
@@ -1669,9 +1669,34 @@ class Body(object):
                         if concrete_build_deps or dtype != "build":
                             clauses.append(fn.attr("depends_on", spec.name, dep.name, dtype))
 
-                            # Ensure Spack will not coconcretize this with another provider
-                            # for the same virtual
-                            for virtual in dep.package.virtuals_provided:
+                            # TODO: We have to look up info from package.py here, but we'd
+                            # TODO: like to avoid this entirely. We should not need to look
+                            # TODO: up potentially wrong info if we have virtual edge info.
+                            try:
+                                try:
+                                    pkg = dep.package
+
+                                except spack.repo.UnknownNamespaceError:
+                                    # Try to look up the package of the same name and use its
+                                    # providers. This is as good as we can do without edge info.
+                                    pkg_class = spack.repo.path.get_pkg_class(dep.name)
+                                    spec = spack.spec.Spec(f"{dep.name}@{dep.version}")
+                                    pkg = pkg_class(spec)
+
+                                virtuals = pkg.virtuals_provided
+
+                            except spack.repo.UnknownPackageError:
+                                # Skip virtual node constriants for renamed/deleted packages,
+                                # so their binaries can still be installed.
+                                # NOTE: with current specs (which lack edge attributes) this
+                                # can allow concretizations with two providers, but it's unlikely.
+                                continue
+
+                            # Don't concretize with two providers of the same virtual.
+                            # See above for exception for unknown packages.
+                            # TODO: we will eventually record provider information on edges,
+                            # TODO: which avoids the need for the package lookup above.
+                            for virtual in virtuals:
                                 clauses.append(fn.attr("virtual_node", virtual.name))
                                 clauses.append(fn.provider(dep.name, virtual.name))
 
diff --git a/lib/spack/spack/spec.py b/lib/spack/spack/spec.py
--- a/lib/spack/spack/spec.py
+++ b/lib/spack/spack/spec.py
@@ -2789,11 +2789,11 @@ def inject_patches_variant(root):
         # Also record all patches required on dependencies by
         # depends_on(..., patch=...)
         for dspec in root.traverse_edges(deptype=all, cover="edges", root=False):
-            pkg_deps = dspec.parent.package_class.dependencies
-            if dspec.spec.name not in pkg_deps:
+            if dspec.spec.concrete:
                 continue
 
-            if dspec.spec.concrete:
+            pkg_deps = dspec.parent.package_class.dependencies
+            if dspec.spec.name not in pkg_deps:
                 continue
 
             patches = []


Test patch: