Repository: spack/spack

Issue:
Cannot build HDF5 @1.10.0-patch1 +mpi
I am trying to build HDF5 on a Linux workstation with the `+mpi` option. This build fails in the configure stage with this error:

```
configure:4528: checking whether the C compiler works
configure:4550: /xfs1/eschnetter/src/spack/opt/spack/linux-x86_64/gcc-6.1.0-spack/openmpi-1.10.2-thgejjepeloy5b3wgat2ax6t2d47poon/bin/mpicc       conftest.c  >&5
/xfs1/eschnetter/src/spack/opt/spack/linux-x86_64/gcc-6.1.0-spack/openmpi-1.10.2-thgejjepeloy5b3wgat2ax6t2d47poon/lib/libmpi.so: undefined reference to `orte_proc_is_bound'
/xfs1/eschnetter/src/spack/opt/spack/linux-x86_64/gcc-6.1.0-spack/openmpi-1.10.2-thgejjepeloy5b3wgat2ax6t2d47poon/lib/libmpi.so: undefined reference to `orte_rml_send_callback'
/xfs1/eschnetter/src/spack/opt/spack/linux-x86_64/gcc-6.1.0-spack/openmpi-1.10.2-thgejjepeloy5b3wgat2ax6t2d47poon/lib/libmpi.so: undefined reference to `orte_grpcomm_base'
/xfs1/eschnetter/src/spack/opt/spack/linux-x86_64/gcc-6.1.0-spack/openmpi-1.10.2-thgejjepeloy5b3wgat2ax6t2d47poon/lib/libmpi.so: undefined reference to `orte_rml_recv_callback'
/xfs1/eschnetter/src/spack/opt/spack/linux-x86_64/gcc-6.1.0-spack/openmpi-1.10.2-thgejjepeloy5b3wgat2ax6t2d47poon/lib/libmpi.so: undefined reference to `orte_state_base_framework'
/xfs1/eschnetter/src/spack/opt/spack/linux-x86_64/gcc-6.1.0-spack/openmpi-1.10.2-thgejjepeloy5b3wgat2ax6t2d47poon/lib/libmpi.so: undefined reference to `orte_rml_recv_cb_t_class'
/xfs1/eschnetter/src/spack/opt/spack/linux-x86_64/gcc-6.1.0-spack/openmpi-1.10.2-thgejjepeloy5b3wgat2ax6t2d47poon/lib/libmpi.so: undefined reference to `orte_proc_state_to_str'
/xfs1/eschnetter/src/spack/opt/spack/linux-x86_64/gcc-6.1.0-spack/openmpi-1.10.2-thgejjepeloy5b3wgat2ax6t2d47poon/lib/libmpi.so: undefined reference to `orte_grpcomm_collective_t_class'
/xfs1/eschnetter/src/spack/opt/spack/linux-x86_64/gcc-6.1.0-spack/openmpi-1.10.2-thgejjepeloy5b3wgat2ax6t2d47poon/lib/libmpi.so: undefined reference to `orte_rmaps_base_set_mapping_policy'
/xfs1/eschnetter/src/spack/opt/spack/linux-x86_64/gcc-6.1.0-spack/openmpi-1.10.2-thgejjepeloy5b3wgat2ax6t2d47poon/lib/libmpi.so: undefined reference to `orte_state'
/xfs1/eschnetter/src/spack/opt/spack/linux-x86_64/gcc-6.1.0-spack/openmpi-1.10.2-thgejjepeloy5b3wgat2ax6t2d47poon/lib/libmpi.so: undefined reference to `orte_info_close_components'
/xfs1/eschnetter/src/spack/opt/spack/linux-x86_64/gcc-6.1.0-spack/openmpi-1.10.2-thgejjepeloy5b3wgat2ax6t2d47poon/lib/libmpi.so: undefined reference to `orte_rmaps_base_set_ranking_policy'
/xfs1/eschnetter/src/spack/opt/spack/linux-x86_64/gcc-6.1.0-spack/openmpi-1.10.2-thgejjepeloy5b3wgat2ax6t2d47poon/lib/libmpi.so: undefined reference to `orte_info_show_orte_version'
/xfs1/eschnetter/src/spack/opt/spack/linux-x86_64/gcc-6.1.0-spack/openmpi-1.10.2-thgejjepeloy5b3wgat2ax6t2d47poon/lib/libmpi.so: undefined reference to `orte_grpcomm_base_store_modex'
/xfs1/eschnetter/src/spack/opt/spack/linux-x86_64/gcc-6.1.0-spack/openmpi-1.10.2-thgejjepeloy5b3wgat2ax6t2d47poon/lib/libmpi.so: undefined reference to `orte_plm_base_fork_hnp'
/xfs1/eschnetter/src/spack/opt/spack/linux-x86_64/gcc-6.1.0-spack/openmpi-1.10.2-thgejjepeloy5b3wgat2ax6t2d47poon/lib/libmpi.so: undefined reference to `orte_info_register_framework_params'
collect2: error: ld returned 1 exit status
configure:4554: $? = 1
configure:4592: result: no
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "HDF5"
| #define PACKAGE_TARNAME "hdf5"
| #define PACKAGE_VERSION "1.10.0"
| #define PACKAGE_STRING "HDF5 1.10.0"
| #define PACKAGE_BUGREPORT "help@hdfgroup.org"
| #define PACKAGE_URL ""
| #define PACKAGE "hdf5"
| #define VERSION "1.10.0"
| /* end confdefs.h.  */
| 
| int
| main ()
| {
| 
|   ;
|   return 0;
| }
configure:4597: error: in `/tmp/eschnetter/spack-stage/spack-stage-XI7f3H/hdf5-1.10.0':
configure:4599: error: C compiler cannot create executables
```

This used to work a few weeks ago. Any pointers would be appreciated.



Discussion:
@eschnett I'll try to look into the issue. Is : 

```
spack install hdf5@1.10.0-patch1+mpi
```

the exact command you are giving?

I first installed gcc via Spack, added it as a compiler, and then gave this command:

```
spack install hdf5@1.10.0-patch1+mpi %gcc@6.1.0-spack ^openmpi@1.10.2 %gcc@6.1.0-spack
```

That is, I'm building against the most recent version of OpenMPI.

I was not able to reproduce your error, but encountered another:

```
$ spack compilers
==> Available compilers
-- gcc ----------------------------------------------------------
gcc@6.1.0  gcc@4.8

$ spack install hdf5@1.10.0-patch1+mpi %gcc@6.1.0 ^openmpi@1.10.2 %gcc@6.1.0
==> Installing hdf5
==> zlib is already installed in /home/mculpo/PycharmProjects/spack/opt/spack/linux-x86_64/gcc-6.1.0/zlib-1.2.8-xfi4wikeyyjiaqlpze45nx7yc3jife6r
==> openmpi is already installed in /home/mculpo/PycharmProjects/spack/opt/spack/linux-x86_64/gcc-6.1.0/openmpi-1.10.2-fa3jbsybc4lyvss6raftwihfvp7opclp
==> Trying to fetch from file:///home/mculpo/production/spack-mirror/hdf5/hdf5-1.10.0-patch1.tar.gz
######################################################################## 100,0%
==> Staging archive: /home/mculpo/PycharmProjects/spack/var/spack/stage/hdf5-1.10.0-patch1-lokldyhek5mswvwowz3soycsgxtpcnuf/hdf5-1.10.0-patch1.tar.gz
==> Created stage in /home/mculpo/PycharmProjects/spack/var/spack/stage/hdf5-1.10.0-patch1-lokldyhek5mswvwowz3soycsgxtpcnuf
==> No patches needed for hdf5
==> Building hdf5
Traceback (most recent call last):
  File "/home/mculpo/PycharmProjects/spack/lib/spack/spack/build_environment.py", line 406, in fork
    function()
  File "/home/mculpo/PycharmProjects/spack/lib/spack/spack/package.py", line 950, in build_process
    self.install(self.spec, self.prefix)
  File "/home/mculpo/PycharmProjects/spack/var/spack/repos/builtin/packages/hdf5/package.py", line 145, in install
    self.check_install(spec)
  File "/home/mculpo/PycharmProjects/spack/var/spack/repos/builtin/packages/hdf5/package.py", line 195, in check_install
    raise RuntimeError("HDF5 install check failed")
RuntimeError: HDF5 install check failed
==> Error: Installation process had nonzero exit code.
```

I'll have a look if this is a false positive. If it matters I am running on Ubuntu 14.04.

I think have a pull request for this; the version check fails since it doesn't handle the `patch1` in the version number -- this is part of Spack's version number, but not part of HDF5's version number. (They release patches without increasing the version number reported at run time.)

I can also reproduce this with the simpler command

```
spack install hdf5 +mpi %gcc@5.3.0 ^openmpi
```

which doesn't require a Spack-built gcc, and uses HDF5 1.8 instead of 1.10.

Sorry, still not able to reproduce on a fresh `LLNL/develop` checkout:

```
$ spack install hdf5 +mpi %gcc@4.8 ^openmpi
==> Installing hdf5
==> Installing zlib
==> Trying to fetch from file:///home/massimiliano/production/spack/mirror/zlib/zlib-1.2.8.tar.gz
######################################################################## 100,0%
==> Staging archive: /home/massimiliano/PycharmProjects/spack/var/spack/stage/zlib-1.2.8-d6pdl6xvnvap6ihrqcqtgvweghbszmix/zlib-1.2.8.tar.gz
==> Created stage in /home/massimiliano/PycharmProjects/spack/var/spack/stage/zlib-1.2.8-d6pdl6xvnvap6ihrqcqtgvweghbszmix
==> No patches needed for zlib
==> Building zlib
==> Successfully installed zlib
  Fetch: 0.05s.  Build: 9.26s.  Total: 9.32s.
[+] /home/massimiliano/PycharmProjects/spack/opt/spack/linux-x86_64/gcc-4.8/zlib-1.2.8-d6pdl6xvnvap6ihrqcqtgvweghbszmix
==> Installing openmpi
==> Installing hwloc
==> Installing libpciaccess
==> Installing libtool
==> Installing m4
==> Installing libsigsegv
==> Trying to fetch from file:///home/massimiliano/production/spack/mirror/libsigsegv/libsigsegv-2.10.tar.gz
curl: (37) Couldn't open file /home/massimiliano/production/spack/mirror/libsigsegv/libsigsegv-2.10.tar.gz
==> Fetching from file:///home/massimiliano/production/spack/mirror/libsigsegv/libsigsegv-2.10.tar.gz failed.
==> Trying to fetch from ftp://ftp.gnu.org/gnu/libsigsegv/libsigsegv-2.10.tar.gz
######################################################################## 100,0%
==> Staging archive: /home/massimiliano/PycharmProjects/spack/var/spack/stage/libsigsegv-2.10-ev7o3dbqvwzojhesg3bvv5rv4oy27nzk/libsigsegv-2.10.tar.gz
==> Created stage in /home/massimiliano/PycharmProjects/spack/var/spack/stage/libsigsegv-2.10-ev7o3dbqvwzojhesg3bvv5rv4oy27nzk
==> No patches needed for libsigsegv
==> Building libsigsegv
==> Successfully installed libsigsegv
  Fetch: 6.78s.  Build: 10.66s.  Total: 17.44s.
[+] /home/massimiliano/PycharmProjects/spack/opt/spack/linux-x86_64/gcc-4.8/libsigsegv-2.10-ev7o3dbqvwzojhesg3bvv5rv4oy27nzk
==> Trying to fetch from file:///home/massimiliano/production/spack/mirror/m4/m4-1.4.17.tar.gz
curl: (37) Couldn't open file /home/massimiliano/production/spack/mirror/m4/m4-1.4.17.tar.gz
==> Fetching from file:///home/massimiliano/production/spack/mirror/m4/m4-1.4.17.tar.gz failed.
==> Trying to fetch from ftp://ftp.gnu.org/gnu/m4/m4-1.4.17.tar.gz
######################################################################## 100,0%
==> Staging archive: /home/massimiliano/PycharmProjects/spack/var/spack/stage/m4-1.4.17-iubkayqswvtjdvnzc2xgs7lphsuybetv/m4-1.4.17.tar.gz
==> Created stage in /home/massimiliano/PycharmProjects/spack/var/spack/stage/m4-1.4.17-iubkayqswvtjdvnzc2xgs7lphsuybetv
==> Applied patch pgi.patch
==> Building m4
==> Successfully installed m4
  Fetch: 11.87s.  Build: 1m 0.64s.  Total: 1m 12.51s.
[+] /home/massimiliano/PycharmProjects/spack/opt/spack/linux-x86_64/gcc-4.8/m4-1.4.17-iubkayqswvtjdvnzc2xgs7lphsuybetv
==> Trying to fetch from file:///home/massimiliano/production/spack/mirror/libtool/libtool-2.4.6.tar.gz
######################################################################## 100,0%
==> Staging archive: /home/massimiliano/PycharmProjects/spack/var/spack/stage/libtool-2.4.6-duvelz7ywvzbcrag7q4clr4pu3scr7kv/libtool-2.4.6.tar.gz
==> Created stage in /home/massimiliano/PycharmProjects/spack/var/spack/stage/libtool-2.4.6-duvelz7ywvzbcrag7q4clr4pu3scr7kv
==> No patches needed for libtool
==> Building libtool
==> Successfully installed libtool
  Fetch: 0.10s.  Build: 20.51s.  Total: 20.61s.
[+] /home/massimiliano/PycharmProjects/spack/opt/spack/linux-x86_64/gcc-4.8/libtool-2.4.6-duvelz7ywvzbcrag7q4clr4pu3scr7kv
==> Trying to fetch from file:///home/massimiliano/production/spack/mirror/libpciaccess/libpciaccess-0.13.4.tar.bz2
######################################################################## 100,0%
==> Staging archive: /home/massimiliano/PycharmProjects/spack/var/spack/stage/libpciaccess-0.13.4-hs4bbwph7geugp33d5x62c7vryzqgfw7/libpciaccess-0.13.4.tar.bz2
==> Created stage in /home/massimiliano/PycharmProjects/spack/var/spack/stage/libpciaccess-0.13.4-hs4bbwph7geugp33d5x62c7vryzqgfw7
==> No patches needed for libpciaccess
==> Building libpciaccess
==> Successfully installed libpciaccess
  Fetch: 0.04s.  Build: 13.03s.  Total: 13.07s.
[+] /home/massimiliano/PycharmProjects/spack/opt/spack/linux-x86_64/gcc-4.8/libpciaccess-0.13.4-hs4bbwph7geugp33d5x62c7vryzqgfw7
==> Trying to fetch from file:///home/massimiliano/production/spack/mirror/hwloc/hwloc-1.11.3.tar.gz
curl: (37) Couldn't open file /home/massimiliano/production/spack/mirror/hwloc/hwloc-1.11.3.tar.gz
==> Fetching from file:///home/massimiliano/production/spack/mirror/hwloc/hwloc-1.11.3.tar.gz failed.
==> Trying to fetch from http://www.open-mpi.org/software/hwloc/v1.11/downloads/hwloc-1.11.3.tar.gz
######################################################################## 100,0%
==> Staging archive: /home/massimiliano/PycharmProjects/spack/var/spack/stage/hwloc-1.11.3-irtf3rl4jyqu7qlojubjrf5swyf7ddrh/hwloc-1.11.3.tar.gz
==> Created stage in /home/massimiliano/PycharmProjects/spack/var/spack/stage/hwloc-1.11.3-irtf3rl4jyqu7qlojubjrf5swyf7ddrh
==> No patches needed for hwloc
==> Building hwloc
==> Successfully installed hwloc
  Fetch: 19.83s.  Build: 1m 3.37s.  Total: 1m 23.19s.
[+] /home/massimiliano/PycharmProjects/spack/opt/spack/linux-x86_64/gcc-4.8/hwloc-1.11.3-irtf3rl4jyqu7qlojubjrf5swyf7ddrh
==> Trying to fetch from file:///home/massimiliano/production/spack/mirror/openmpi/openmpi-1.10.2.tar.bz2
######################################################################## 100,0%
==> Staging archive: /home/massimiliano/PycharmProjects/spack/var/spack/stage/openmpi-1.10.2-5ie6o3s3fnbfwnseoh5v33pr6ks7tcly/openmpi-1.10.2.tar.bz2
==> Created stage in /home/massimiliano/PycharmProjects/spack/var/spack/stage/openmpi-1.10.2-5ie6o3s3fnbfwnseoh5v33pr6ks7tcly
==> Building openmpi
==> Successfully installed openmpi
  Fetch: 0.18s.  Build: 19m 52.10s.  Total: 19m 52.28s.
[+] /home/massimiliano/PycharmProjects/spack/opt/spack/linux-x86_64/gcc-4.8/openmpi-1.10.2-5ie6o3s3fnbfwnseoh5v33pr6ks7tcly
==> Trying to fetch from file:///home/massimiliano/production/spack/mirror/hdf5/hdf5-1.8.16.tar.gz
######################################################################## 100,0%
==> Staging archive: /home/massimiliano/PycharmProjects/spack/var/spack/stage/hdf5-1.8.16-qjvk7idxsgjgns6aaxukmoggchqj5mem/hdf5-1.8.16.tar.gz
==> Created stage in /home/massimiliano/PycharmProjects/spack/var/spack/stage/hdf5-1.8.16-qjvk7idxsgjgns6aaxukmoggchqj5mem
==> No patches needed for hdf5
==> Building hdf5
==> Successfully installed hdf5
  Fetch: 0.15s.  Build: 8m 24.39s.  Total: 8m 24.54s.
[+] /home/massimiliano/PycharmProjects/spack/opt/spack/linux-x86_64/gcc-4.8/hdf5-1.8.16-qjvk7idxsgjgns6aaxukmoggchqj5mem
```

I find the option `-L/usr/lib` in OpenMPI's `libmpi.la` file. This will not do; this adds `/usr/lib` as a library search directory early into the search path, so that the system libraries are found before Spack's libraries. Hence HDF5 fails to configure -- it picks up a strange combination of the system OpenMPI and Spack's OpenMPI.

I don't know yet why `-L/usr/lib` is there, but it shouldn't. This might be a bug in OpenMPI, or in Spack. In other circumstances, I have to explicitly filter directories such as `/usr/lib` from search paths; maybe that's all that's missing.

The `/usr/lib` are produced by OpenMPI. I'm going to open a bug report upstream.


Resolving patch:
diff --git a/var/spack/repos/builtin/packages/openmpi/package.py b/var/spack/repos/builtin/packages/openmpi/package.py
--- a/var/spack/repos/builtin/packages/openmpi/package.py
+++ b/var/spack/repos/builtin/packages/openmpi/package.py
@@ -142,7 +142,7 @@ def install(self, spec, prefix):
         ])
         if '+verbs' in spec:
             path = _verbs_dir()
-            if path is not None:
+            if path is not None and path not in ('/usr', '/usr/local'):
                 config_args.append('--with-%s=%s' % (self.verbs, path))
             else:
                 config_args.append('--with-%s' % self.verbs)


Test patch: