Repository: spack/spack

Issue:
buildcache for foreign architecture no longer readable
<!-- Explain, in a clear and concise way, the command you ran and the result you were trying to achieve.
Example: "I ran `spack find` to list all the installed packages and ..." -->
I generated a buildcache with cnl7 (alcf iota), copied the mirror to a cnl6 system (alcf theta) and tried to list packages.
When building the cache with spack v0.14.2, I was able to view the packages, but not install. When building with v0.15.0, I am neither able to view nor install. 

### Steps to reproduce the issue

```console
熊俊傑@thetalogin1:~/spack$ git checkout v0.14.2
HEAD is now at 7a68a4d85... update CHANGELOG.md for 0.14.2
熊俊傑@thetalogin1:~/spack$ spack buildcache list -al arch=cray-cnl7-mic_knl 
==> Finding buildcaches in /home/willmore/mirror/build_cache
==> Finding buildcaches in /tmp/mirror/build_cache
==> Finding buildcaches in /tmp/build_cache
==> Finding buildcaches in /home/willmore/spack-mirror/build_cache
-- cray-cnl7-mic_knl / gcc@7.3.0 --------------------------------
jscdp26 hdf5@1.10.6  ek675oc zlib@1.2.11

-- cray-cnl7-mic_knl / gcc@9.2.0 --------------------------------
bl5unjg hdf5@1.8.16  zjoxq7c kokkos@3.1.01   zmjwtet zlib@1.2.11
4b3p7v2 hdf5@1.10.6  4itl6jz numactl@2.0.12
熊俊傑@thetalogin1:~/spack$ git checkout v0.15.0
Previous HEAD position was 7a68a4d85... update CHANGELOG.md for 0.14.2
HEAD is now at d4bf70d98... changelog for v0.15.0
熊俊傑@thetalogin1:~/spack$ spack buildcache list -al arch=cray-cnl7-mic_knl 
==> Finding buildcaches at file:///home/willmore/mirror/build_cache
==> Error: Can't recreate arch for spec cray-cnl7-mic_knl on current arch cray-cnl6-haswell; spec architecture is too different
```

tgamblin:
it’s complaining about the OS difference so I believe it should work fine after the upgrade but that’s not great — you want it to list and show what’s there regardless of the host OS right? 

willmore:
yes. It looks like the -o option [for buildcache install] came with v0.15.0 so I upgraded, but I could at least list the packages with the v0.14.2.

### Information on your system

<!-- Please include the output of `spack debug report` -->
熊俊傑@thetalogin1:~/spack$ spack debug report
* **Spack:** 0.15.0
* **Python:** 3.4.6
* **Platform:** cray-sles12-haswell

(This is not the correct target architecture,which should reflectr mic_knl instead of haswell; this also changed since v0.14.2 and reported in issue #16707) 

<!-- If you have any relevant configuration detail (custom `packages.yaml` or `modules.yaml`, etc.) you can add that here as well. -->

### Debug output
```
熊俊傑@thetalogin1:~/spack$ spack -d buildcache list -al arch=cray-cnl7-mic_knl 
==> [2020-07-10-22:47:52.140418, 32101] '/bin/bash' '--norc' '--noprofile' '-lc' 'echo $CRAY_CPU_TARGET'
==> [2020-07-10-22:47:52.145034, 32101] default to front-end architecture
==> [2020-07-10-22:47:52.187344, 32101] Imported buildcache from built-in commands
==> [2020-07-10-22:47:52.190937, 32101] Imported buildcache from built-in commands
==> [2020-07-10-22:47:52.191575, 32101] Reading config file /home/willmore/.spack/cray/mirrors.yaml
==> [2020-07-10-22:47:52.195722, 32101] Finding buildcaches at file:///home/willmore/mirror/build_cache
==> [2020-07-10-22:47:52.196035, 32101] Reading config file /gpfs/mira-home/willmore/spack/etc/spack/defaults/config.yaml
==> [2020-07-10-22:47:52.219533, 32101] DATABASE LOCK TIMEOUT: 3s
==> [2020-07-10-22:47:52.219589, 32101] PACKAGE LOCK TIMEOUT: No timeout
==> [2020-07-10-22:47:52.290808, 32101] '/bin/bash' '--norc' '--noprofile' '-lc' 'echo $CRAY_CPU_TARGET'
==> [2020-07-10-22:47:52.294602, 32101] default to front-end architecture
Traceback (most recent call last):
  File "/home/willmore/spack/bin/spack", line 64, in <module>
    sys.exit(spack.main.main())
  File "/gpfs/mira-home/willmore/spack/lib/spack/spack/main.py", line 761, in main
    return _invoke_command(command, parser, args, unknown)
  File "/gpfs/mira-home/willmore/spack/lib/spack/spack/main.py", line 489, in _invoke_command
    return_val = command(parser, args)
  File "/gpfs/mira-home/willmore/spack/lib/spack/spack/cmd/buildcache.py", line 756, in buildcache
    args.func(args)
  File "/gpfs/mira-home/willmore/spack/lib/spack/spack/cmd/buildcache.py", line 491, in listspecs
    specs = bindist.get_specs(args.allarch)
  File "/gpfs/mira-home/willmore/spack/lib/spack/spack/binary_distribution.py", line 888, in get_specs
    spec_arch = architecture.arch_for_spec(indexed_spec.architecture)
  File "/gpfs/mira-home/willmore/spack/lib/spack/spack/architecture.py", line 476, in arch_for_spec
    "spec architecture is too different" % (arch_spec, sys_type()))
ValueError: Can't recreate arch for spec cray-cnl7-mic_knl on current arch cray-cnl6-haswell; spec architecture is too different
```

### Additional information

<!-- These boxes can be checked by replacing [ ] with [x] or by clicking them after submitting the issue. -->
- [x] I have run `spack debug report` and reported the version of Spack/Python/Platform
- [x] I have searched the issues of this repo and believe this is not a duplicate
- [x] I have run the failing commands in debug mode and reported the output

<!-- We encourage you to try, as much as possible, to reduce your problem to the minimal example that still reproduces the issue. That would help us a lot in fixing it quickly and effectively!

If you want to ask a question about the tool (how to use it, what it can currently do, etc.), try the `#general` channel on our Slack first. We have a welcoming community and chances are you'll get your reply faster and without opening an issue.

Other than that, thanks for taking the time to contribute to Spack! -->



Discussion:
This functionality was removed with the switch to inxex.json
https://github.com/spack/spack/pull/15002
@scottwittenburg 

Resolving patch:
diff --git a/lib/spack/spack/architecture.py b/lib/spack/spack/architecture.py
--- a/lib/spack/spack/architecture.py
+++ b/lib/spack/spack/architecture.py
@@ -436,6 +436,12 @@ def to_dict(self):
             ('target', self.target.to_dict_or_value())])
         return syaml_dict([('arch', d)])
 
+    def to_spec(self):
+        """Convert this Arch to an anonymous Spec with architecture defined."""
+        spec = spack.spec.Spec()
+        spec.architecture = spack.spec.ArchSpec(str(self))
+        return spec
+
     @staticmethod
     def from_dict(d):
         spec = spack.spec.ArchSpec.from_dict(d)
@@ -518,6 +524,14 @@ def platform():
 
 
 @memoized
+def default_arch():
+    """Default ``Arch`` object for this machine.
+
+    See ``sys_type()``.
+    """
+    return Arch(platform(), 'default_os', 'default_target')
+
+
 def sys_type():
     """Print out the "default" platform-os-target tuple for this machine.
 
@@ -530,8 +544,7 @@ def sys_type():
     architectures.
 
     """
-    arch = Arch(platform(), 'default_os', 'default_target')
-    return str(arch)
+    return str(default_arch())
 
 
 @memoized
diff --git a/lib/spack/spack/binary_distribution.py b/lib/spack/spack/binary_distribution.py
--- a/lib/spack/spack/binary_distribution.py
+++ b/lib/spack/spack/binary_distribution.py
@@ -36,7 +36,6 @@
 from spack.spec import Spec
 from spack.stage import Stage
 from spack.util.gpg import Gpg
-import spack.architecture as architecture
 
 _build_cache_relative_path = 'build_cache'
 
@@ -856,13 +855,11 @@ def get_spec(spec=None, force=False):
     return try_download_specs(urls=urls, force=force)
 
 
-def get_specs(allarch=False):
+def get_specs():
     """
     Get spec.yaml's for build caches available on mirror
     """
     global _cached_specs
-    arch = architecture.Arch(architecture.platform(),
-                             'default_os', 'default_target')
 
     if not spack.mirror.MirrorCollection():
         tty.debug("No Spack mirrors are currently configured")
@@ -882,8 +879,7 @@ def get_specs(allarch=False):
                 index_url, 'application/json')
             index_object = codecs.getreader('utf-8')(file_stream).read()
         except (URLError, web_util.SpackWebError) as url_err:
-            tty.error('Failed to read index {0}'.format(index_url))
-            tty.debug(url_err)
+            tty.debug('Failed to read index {0}'.format(index_url), url_err, 1)
             # Continue on to the next mirror
             continue
 
@@ -900,9 +896,7 @@ def get_specs(allarch=False):
         spec_list = db.query_local(installed=False)
 
         for indexed_spec in spec_list:
-            spec_arch = architecture.arch_for_spec(indexed_spec.architecture)
-            if (allarch is True or spec_arch == arch):
-                _cached_specs.add(indexed_spec)
+            _cached_specs.add(indexed_spec)
 
     return _cached_specs
 
diff --git a/lib/spack/spack/cmd/buildcache.py b/lib/spack/spack/cmd/buildcache.py
--- a/lib/spack/spack/cmd/buildcache.py
+++ b/lib/spack/spack/cmd/buildcache.py
@@ -8,6 +8,7 @@
 import sys
 
 import llnl.util.tty as tty
+import spack.architecture
 import spack.binary_distribution as bindist
 import spack.cmd
 import spack.cmd.common.arguments as arguments
@@ -25,6 +26,7 @@
 
 from spack.error import SpecError
 from spack.spec import Spec, save_dependency_spec_yamls
+from spack.util.string import plural
 
 from spack.cmd import display_specs
 
@@ -288,8 +290,12 @@ def match_downloaded_specs(pkgs, allow_multiple_matches=False, force=False,
     # List of specs that match expressions given via command line
     specs_from_cli = []
     has_errors = False
-    allarch = other_arch
-    specs = bindist.get_specs(allarch)
+
+    specs = bindist.get_specs()
+    if not other_arch:
+        arch = spack.architecture.default_arch().to_spec()
+        specs = [s for s in specs if s.satisfies(arch)]
+
     for pkg in pkgs:
         matches = []
         tty.msg("buildcache spec(s) matching %s \n" % pkg)
@@ -488,10 +494,20 @@ def install_tarball(spec, args):
 
 def listspecs(args):
     """list binary packages available from mirrors"""
-    specs = bindist.get_specs(args.allarch)
+    specs = bindist.get_specs()
+    if not args.allarch:
+        arch = spack.architecture.default_arch().to_spec()
+        specs = [s for s in specs if s.satisfies(arch)]
+
     if args.specs:
         constraints = set(args.specs)
         specs = [s for s in specs if any(s.satisfies(c) for c in constraints)]
+    if sys.stdout.isatty():
+        builds = len(specs)
+        tty.msg("%s." % plural(builds, 'cached build'))
+        if not builds and not args.allarch:
+            tty.msg("You can query all available architectures with:",
+                    "spack buildcache list --allarch")
     display_specs(specs, args, all_headers=True)
 
 


Test patch:
diff --git a/lib/spack/spack/test/cmd/buildcache.py b/lib/spack/spack/test/cmd/buildcache.py
--- a/lib/spack/spack/test/cmd/buildcache.py
+++ b/lib/spack/spack/test/cmd/buildcache.py
@@ -12,6 +12,7 @@
 import spack.main
 import spack.binary_distribution
 import spack.environment as ev
+import spack.spec
 from spack.spec import Spec
 
 buildcache = spack.main.SpackCommand('buildcache')
@@ -24,7 +25,22 @@
 def mock_get_specs(database, monkeypatch):
     specs = database.query_local()
     monkeypatch.setattr(
-        spack.binary_distribution, 'get_specs', lambda x: specs
+        spack.binary_distribution, 'get_specs', lambda: specs
+    )
+
+
+@pytest.fixture()
+def mock_get_specs_multiarch(database, monkeypatch):
+    specs = [spec.copy() for spec in database.query_local()]
+
+    # make one spec that is NOT the test architecture
+    for spec in specs:
+        if spec.name == "mpileaks":
+            spec.architecture = spack.spec.ArchSpec('linux-rhel7-x86_64')
+            break
+
+    monkeypatch.setattr(
+        spack.binary_distribution, 'get_specs', lambda: specs
     )
 
 
@@ -37,10 +53,6 @@ def test_buildcache_preview_just_runs(database):
     buildcache('preview', 'mpileaks')
 
 
-@pytest.mark.skipif(
-    platform.system().lower() != 'linux',
-    reason='implementation for MacOS still missing'
-)
 @pytest.mark.db
 @pytest.mark.regression('13757')
 def test_buildcache_list_duplicates(mock_get_specs, capsys):
@@ -50,6 +62,20 @@ def test_buildcache_list_duplicates(mock_get_specs, capsys):
     assert output.count('mpileaks') == 3
 
 
+@pytest.mark.db
+@pytest.mark.regression('17827')
+def test_buildcache_list_allarch(database, mock_get_specs_multiarch, capsys):
+    with capsys.disabled():
+        output = buildcache('list', '--allarch')
+
+    assert output.count('mpileaks') == 3
+
+    with capsys.disabled():
+        output = buildcache('list')
+
+    assert output.count('mpileaks') == 2
+
+
 def tests_buildcache_create(
         install_mockery, mock_fetch, monkeypatch, tmpdir):
     """"Ensure that buildcache create creates output files"""