Repository: spack/spack

Issue:
spack info errors out with multiple urls 
@alalazo I think the urls extension from #13881 to allow backup urls
for fetching doesn't work.  I'm following the directions from:

https://spack.readthedocs.io/en/latest/packaging_guide.html#mirrors-of-the-main-url

Here's a simple example using bzip2.  I'm using spack 87e69eed1c48
from today, 2020-03-11.

Use bzip2, comment out all but version 1.0.8, and change url to urls
(plural) to a list of two urls.

```
  urls = [
      'https://sourceware.org/pub/bzip2/bzip2-1.0.8.tar.gz',
      'ftp://sourceware.org/pub/bzip2/bzip2-1.0.8.tar.gz',
  ]
  ...
  version('1.0.8', sha256='...')
  # version('1.0.7', sha256='...')
  # version('1.0.6', sha256='...')
```

This case is just one version and two urls, that is, more urls than
versions.  This case mostly works.

```
spack info bzip2

Preferred version:  
    1.0.8    https://sourceware.org/pub/bzip2/bzip2-1.0.8.tar.gz

Safe versions:  
    1.0.8    ftp://sourceware.org/pub/bzip2/bzip2-1.0.8.tar.gz
```

Although this mostly works (and does fetch with https), it's kinda
wonky because 'preferred' says https and 'safe' says ftp for the same
version.

But now, return versions 1.0.7 and 1.0.6, so we have 3 versions and
two urls and I get this error.

```
spack info bzip2

Preferred version:  
    1.0.8    https://sourceware.org/pub/bzip2/bzip2-1.0.8.tar.gz

Safe versions:  
    1.0.8    ftp://sourceware.org/pub/bzip2/bzip2-1.0.8.tar.gz
==> Error: pop from empty list
```

The problem seems to happen whenever there are more versions than
urls.  This suggests that something is pairing the list of versions
with the list of urls.  That is, walking down both lists and
complaining when urls runs out before versions.

Instead, it should be more of a cross product.  That is, any version
should be fetchable from anything on the list of backup urls.




Discussion:
By:

> I think the urls extension from #13881 to allow backup urls for fetching doesn't work. 

do you mean that `spack info` doesn't work (I can reproduce that) or are there other issues with actual fetching?
@alalazo Ok, fair enough.  I got as far as the error from `spack info`
and I didn't try `spack fetch`.  :-(

You're right.  It does fetch, even with the error from spack info.
And it does correctly roll over to the next url when I put a bogus url
in front.

So it's only `spack info` that fails and it's not nearly as serious as
I had thought.  I can go back to adding mirror urls for elfutils,
bzip, etc for #15394.

Feel free to change the subject line, if you wish.

Thanks!
But I guess it's still moderately serious.  I wanted to add backup
urls for elfutils and bzip2 because sourceware.org is often flaky.

I can write a patch and test it, but I can't merge it until this is
fixed first because that would break spack info and hide all the
versions, variants, etc.
I have a fix incoming :slightly_smiling_face: 

Resolving patch:
diff --git a/lib/spack/spack/fetch_strategy.py b/lib/spack/spack/fetch_strategy.py
--- a/lib/spack/spack/fetch_strategy.py
+++ b/lib/spack/spack/fetch_strategy.py
@@ -1261,7 +1261,7 @@ def _from_merged_attrs(fetcher, pkg, version):
         # TODO: refactor this logic into its own method or function
         # TODO: to avoid duplication
         mirrors = [spack.url.substitute_version(u, version)
-                   for u in getattr(pkg, 'urls', [])]
+                   for u in getattr(pkg, 'urls', [])[1:]]
         attrs = {fetcher.url_attr: url, 'mirrors': mirrors}
     else:
         url = getattr(pkg, fetcher.url_attr)
diff --git a/lib/spack/spack/package.py b/lib/spack/spack/package.py
--- a/lib/spack/spack/package.py
+++ b/lib/spack/spack/package.py
@@ -763,7 +763,7 @@ def url_for_version(self, version):
         # If no specific URL, use the default, class-level URL
         url = getattr(self, 'url', None)
         urls = getattr(self, 'urls', [None])
-        default_url = url or urls.pop(0)
+        default_url = url or urls[0]
 
         # if no exact match AND no class-level default, use the nearest URL
         if not default_url:


Test patch:
diff --git a/lib/spack/spack/test/url_fetch.py b/lib/spack/spack/test/url_fetch.py
--- a/lib/spack/spack/test/url_fetch.py
+++ b/lib/spack/spack/test/url_fetch.py
@@ -32,7 +32,7 @@ def pkg_factory():
     def factory(url, urls):
 
         def fn(v):
-            main_url = url or urls.pop(0)
+            main_url = url or urls[0]
             return spack.url.substitute_version(main_url, v)
 
         return Pkg(