Repository: spack/spack

Issue:
Spack's default timeout is too short
Spack occasionally aborts a fetch too early when a url source is too slow.  I ran into this issue with util-macros on a busy night. 


### Steps to reproduce the issue

```console
$ spack install util-macros
```

### Steps that resolved the issue (workaround)

I edited spack/lib/spack/spack/fetch_strategy.py to increase the timeout to 1 minute from 10 seconds.  Ideally this should be configurable.

### Error Message


```console
==> Installing util-macros
==> Searching for binary cache of util-macros
==> No binary for util-macros found: installing from source
==> Fetching https://www.x.org/archive/individual/util/util-macros-1.19.1.tar.bz2

curl: (28) Operation timed out after 10001 milliseconds with 0 out of 0 bytes received
==> Failed to fetch file from URL: https://www.x.org/archive/individual/util/util-macros-1.19.1.tar.bz2
    Curl failed with error 28
==> Fetching from https://www.x.org/archive/individual/util/util-macros-1.19.1.tar.bz2 failed.
==> Error: FetchError: All fetchers failed for spack-stage-util-macros-1.19.1-3khni2zc7qm6ruqeot3pbstb2lb36uhd

/blues/gpfs/home/underwoo/git/spack/lib/spack/spack/package.py:1110, in do_fetch:
       1107                raise FetchError("Will not fetch %s" %
       1108                                 self.spec.format('{name}{@version}'), ck_msg)
       1109
  >>   1110        self.stage.create()
       1111        self.stage.fetch(mirror_only)
       1112        self._fetch_time = time.time() - start_time
       1113
```


### Information on your system

I ran this on ANL's bebop, but also replicated it on Clemson's palmetto using spack's default configuration.

Eigen project is moving
The Eigen project is moving from Bitbucket to GitLab, and its Spack recipe will have to be updated accordingly.  See announcement below.


> Dear all,
> 
> following the decision of Bitbucket.org to delete all mercurial repositories next spring, we decided to migrate to a git repository hosted on gitlab.com. We also decided to take this opportunity to migrate our self hosted bugzilla instance to gitlab.com. Both migrations are scheduled on the 4th of December, starting at 8:00 UTC, and the new project:
> 
> https://gitlab.com/libeigen/eigen
> 
> will be made public once the migration is completed.
> 
> Details on the migration process:
> 
> - Our bugzilla (https://eigen.tuxfamily.org/bz/) will be made read only; all entries, comments and attachments will be transferred to gitlab.com while keeping the same bug IDs.
> 
> - https://bitbucket.org/eigen/eigen will be unavailable and replaced by an empty project pointing to https://gitlab.com/libeigen/eigen
> 
> - The old mercurial repository will still be readable on eigen.tuxfamily.org but with no updates.
> 
> - Existing pull-requests will be archived as static pages hosted on eigen.tuxfamily.org
> 
> - The official git mirror (https://github.com/eigenteam/eigen-git-mirror) will be kept as it is for a few months with no updates until we eventually replace it by an empty project pointing to https://gitlab.com/libeigen/eigen. (it cannot be synchronised because of hash differences)
> 
> Cheers,
> Gael
> 


unifyfs fails to build, unknown type 'MPI_Fint'
Package `unifyfs` fails to build on Ubuntu 18.04, x86_64 with Spack develop (e3b357ad87). The full spack-build-log is attached, but the error is shown below. I've tried with both `mpich` and `openmpi` -- same error either way.

Full build log: [unifyfs-build-out.txt](https://github.com/spack/spack/files/4276893/unifyfs-build-out.txt)

**Error:**
```
make[3]: Entering directory '/tmp/walker/spack-stage/spack-stage-unifyfs-0.9.0-yqx3hsbr465azvv6lwnq7q5vp7r5st3p/spack-src/spack-build/client/src'
  CC       libunifyfs_la-margo_client.lo
  CC       libunifyfs_la-unifyfs.lo
  CC       libunifyfs_la-unifyfs-dirops.lo
  CC       libunifyfs_la-unifyfs-fixed.lo
  CC       libunifyfs_la-unifyfs-stack.lo
  CC       libunifyfs_la-unifyfs-stdio.lo
  CC       libunifyfs_la-unifyfs-sysio.lo
  CC       libunifyfs_la-pmpi_wrappers.lo
In file included from /tmp/walker/spack-stage/spack-stage-unifyfs-0.9.0-yqx3hsbr465azvv6lwnq7q5vp7r5st3p/spack-src/client/src/pmpi_wrappers.c:15:0:
/tmp/walker/spack-stage/spack-stage-unifyfs-0.9.0-yqx3hsbr465azvv6lwnq7q5vp7r5st3p/spack-src/client/src/pmpi_wrappers.h:21:16: error: unknown type name 'MPI_Fint'
 void mpi_init_(MPI_Fint* ierr);
                ^~~~~~~~
/tmp/walker/spack-stage/spack-stage-unifyfs-0.9.0-yqx3hsbr465azvv6lwnq7q5vp7r5st3p/spack-src/client/src/pmpi_wrappers.h:26:20: error: unknown type name 'MPI_Fint'
 void mpi_finalize_(MPI_Fint* ierr);
                    ^~~~~~~~
Makefile:646: recipe for target 'libunifyfs_la-pmpi_wrappers.lo' failed
make[3]: *** [libunifyfs_la-pmpi_wrappers.lo] Error 1
make[3]: Leaving directory '/tmp/walker/spack-stage/spack-stage-unifyfs-0.9.0-yqx3hsbr465azvv6lwnq7q5vp7r5st3p/spack-src/spack-build/client/src'
Makefile:460: recipe for target 'all-recursive' failed
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory '/tmp/walker/spack-stage/spack-stage-unifyfs-0.9.0-yqx3hsbr465azvv6lwnq7q5vp7r5st3p/spack-src/spack-build/client'
Makefile:490: recipe for target 'all-recursive' failed
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory '/tmp/walker/spack-stage/spack-stage-unifyfs-0.9.0-yqx3hsbr465azvv6lwnq7q5vp7r5st3p/spack-src/spack-build'
Makefile:400: recipe for target 'all' failed
make: *** [all] Error 2
```

@CamStan 


Discussion:
I also had this issue with libbsd today. In my case it worked on about the 10th try.
I also had this issue today with libffi and bzip2.
> I also had this issue today with libffi and bzip2.

For the record build tests on `develop` at https://github.com/spack/spack/commit/8d8925c725491cf028ef6c1b93e431dcb200f439 were failing because of that.
I just ran into the same problem and created a way to configure the timeout in PR #15212. Let me know whether this works for you as well.
I am a new user.  I am now encountering this issue as Spack cannot pull from bitbucket as per:

```terminal
==> No binary for eigen found: installing from source
==> Fetching https://bitbucket.org/eigen/eigen/get/3.3.7.tar.bz2

curl: (22) The requested URL returned error: 401 Unauthorized
==> Fetching from https://bitbucket.org/eigen/eigen/get/3.3.7.tar.bz2 failed.
==> Error: FetchError: All fetchers failed for spack-stage-eigen-3.3.7-rnuwo2wk73rcr3eqk7howzsna35i4ech

/opt/spack_toolchain/spack/lib/spack/spack/package.py:1079, in do_fetch:
       1076                raise FetchError("Will not fetch %s" %
       1077                                 self.spec.format('{name}{@version}'), ck_msg)
       1078
  >>   1079        self.stage.create()
       1080        self.stage.fetch(mirror_only)
       1081        self._fetch_time = time.time() - start_time
       1082
```

I have downloaded the tar.bz2 from the GitHub mirror.  Can I ask, how do I issue a spack install command point to a local tar file?
Thanks,
Andy
Can you update to the new URL in the `package.py` file, test that it works for you, and submit it as a PR?  That would be super!

Note the following additional news just came through on the Eigen email list:
```The critical steps of the migration are done. I've thus made public the new repo:

https://gitlab.com/libeigen/eigen

I've also re-opened the deprecated mercurial repo on bitbucket so that people can smoothly migrate their clones, links, etc.

Please report any outdated links and mercurial/bitbucket references you'll find. I'm already aware of some pages on our wiki, like:
- http://eigen.tuxfamily.org/index.php?title=Developer%27s_Corner
- http://eigen.tuxfamily.org/index.php?title=Mercurial

Any help on updating those old documentations will be greatly appreciated!
```

The fact they have re-opened fixed that problem.  I can confirm it also bust CANTERA as that has Eigen as a dependancy.  I'm about 36 hours new to spack, happy to help, where is that package.py downloaded onto my machine?  Is it only downloaded once I issue `spack install eigen` or is it always there?


Resolving patch:
diff --git a/var/spack/repos/builtin/packages/jblob/package.py b/var/spack/repos/builtin/packages/jblob/package.py
new file mode 100644
--- /dev/null
+++ b/var/spack/repos/builtin/packages/jblob/package.py
@@ -0,0 +1,35 @@
+# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other
+# Spack Project Developers. See the top-level COPYRIGHT file for details.
+#
+# SPDX-License-Identifier: (Apache-2.0 OR MIT)
+
+from spack import *
+
+
+class Jblob(Package):
+    """
+       The German Climate Computing Center
+       (DKRZ: Deutsches Klimarechenzentrum GmbH)
+       provides a Long Term Archiving Service for large research
+       data sets which are relevant for climate or Earth system research.
+    """
+
+    homepage = "https://cera-www.dkrz.de/WDCC/ui/cerasearch"
+    url      = "https://cera-www.dkrz.de/jblob/jblob-3.0.zip"
+
+    maintainers = ['ajkotobi']
+
+    version('3.0', sha256='576b5956358386a8832c6d1d13c410705e54888354a10cfd4f094513458067e4')
+
+    depends_on('java@8:', type='run')
+
+    def setup_run_environment(self, env):
+        env.set('JAVA_HOME', self.spec['java'].prefix)
+
+    def install(self, spec, prefix):
+        filter_file('/opt/jblob-' + self.version, prefix, 'jblob')
+
+        mkdir(prefix.bin)
+        install('jblob', prefix.bin)
+        install_tree('lib', prefix.lib)
+        install_tree('docs', prefix.docs)


Test patch: