Repository: spack/spack

Issue:
"spack buildcache list" breaks when a mirror lacks "build_cache/index.json"
<!-- Explain, in a clear and concise way, the command you ran and the result you were trying to achieve.
Example: "I ran `spack find` to list all the installed packages and ..." -->

If a mirror lacks "build_cache/index.json", spack reports an error and prevents me from accessing the build_cache of any other mirrors. In particular, the source mirror that is now provided by default in spack v0.15.1 has no index.json and I have to remove that mirror to access any build cache's I've created myself.

### Steps to reproduce the issue

Set up a local mirror with a build cache:

```console
$ rm -rf spack/ mirror/ ~/.spack/
$ git clone https://github.com/spack/spack.git
$ . spack/share/spack/setup-env.sh
$ spack install libsigsegv
$ spack gpg init
$ spack gpg create name name@mail
$ spack buildcache create -d mirror --rebuild-index libsigsegv
$ spack mirror add local mirror
```

But I can't see the build cache in the mirror I've created:
```console
$ spack buildcache list
==> Finding buildcaches at https://spack-llnl-mirror.s3-us-west-2.amazonaws.com/build_cache
==> Error: Failed to read index https://spack-llnl-mirror.s3-us-west-2.amazonaws.com/build_cache/index.json
```

I can work around this by removing the default mirror. (I notice that `spack mirror remove spack-public` doesn't work.)
```console
$ rm spack/etc/spack/defaults/mirrors.yaml 
$ spack buildcache list
==> Finding buildcaches at file:///home/vagrant/tmp/mirror/build_cache
-- linux-opensuse_leap15-sandybridge / gcc@7.5.0 ----------------
libsigsegv@2.12
```

### Error Message

<!-- If Spack reported an error, provide the error message. If it did not report an error but the output appears incorrect, provide the incorrect output. If there was no error message and no output but the result is incorrect, describe how it does not match what you expect. -->
```console
$ spack --debug --stacktrace buildcache list
lib/spack/spack/cmd/__init__.py:122 ==> [2020-07-16-08:07:48.158605, 26001] Imported buildcache from built-in commands
lib/spack/spack/cmd/__init__.py:122 ==> [2020-07-16-08:07:48.162069, 26001] Imported buildcache from built-in commands
lib/spack/spack/config.py:805 ==> [2020-07-16-08:07:48.163214, 26001] Reading config file /home/vagrant/tmp/spack/etc/spack/defaults/mirrors.yaml
lib/spack/spack/config.py:805 ==> [2020-07-16-08:07:48.164971, 26001] Reading config file /home/vagrant/.spack/mirrors.yaml
lib/spack/spack/binary_distribution.py:876 ==> [2020-07-16-08:07:48.166383, 26001] Finding buildcaches at https://spack-llnl-mirror.s3-us-west-2.amazonaws.com/build_cache
lib/spack/spack/config.py:805 ==> [2020-07-16-08:07:48.166920, 26001] Reading config file /home/vagrant/tmp/spack/etc/spack/defaults/config.yaml
lib/spack/spack/binary_distribution.py:885 ==> [2020-07-16-08:07:48.898070, 26001] Error: Failed to read index https://spack-llnl-mirror.s3-us-west-2.amazonaws.com/build_cache/index.json
lib/spack/spack/binary_distribution.py:886 ==> [2020-07-16-08:07:48.898294, 26001] HTTPError: HTTP Error 404: Not Found
```

### Information on your system

<!-- Please include the output of `spack debug report` -->
```console
$ spack debug report
* **Spack:** 0.15.1
* **Python:** 3.6.5
* **Platform:** linux-opensuse_leap15-sandybridge
```

<!-- If you have any relevant configuration detail (custom `packages.yaml` or `modules.yaml`, etc.) you can add that here as well. -->

### Additional information

<!-- These boxes can be checked by replacing [ ] with [x] or by clicking them after submitting the issue. -->
- [x] I have run `spack debug report` and reported the version of Spack/Python/Platform
- [x] I have searched the issues of this repo and believe this is not a duplicate
- [x] I have run the failing commands in debug mode and reported the output

<!-- We encourage you to try, as much as possible, to reduce your problem to the minimal example that still reproduces the issue. That would help us a lot in fixing it quickly and effectively!

If you want to ask a question about the tool (how to use it, what it can currently do, etc.), try the `#general` channel on our Slack first. We have a welcoming community and chances are you'll get your reply faster and without opening an issue.

Other than that, thanks for taking the time to contribute to Spack! -->



Discussion:
I struggled at some point removing mirrors using the cli also, it seems you often have to specify the scope:

```
spack mirror remove --scope defaults spack-public
```

or:

```
spack mirror remove --scope user/linux e4s
```


Resolving patch:
diff --git a/lib/spack/spack/binary_distribution.py b/lib/spack/spack/binary_distribution.py
--- a/lib/spack/spack/binary_distribution.py
+++ b/lib/spack/spack/binary_distribution.py
@@ -884,8 +884,8 @@ def get_specs(allarch=False):
         except (URLError, web_util.SpackWebError) as url_err:
             tty.error('Failed to read index {0}'.format(index_url))
             tty.debug(url_err)
-            # Just return whatever specs we may already have cached
-            return _cached_specs
+            # Continue on to the next mirror
+            continue
 
         tmpdir = tempfile.mkdtemp()
         index_file_path = os.path.join(tmpdir, 'index.json')


Test patch: