Repository: spack/spack

Issue:
Installation issue: cray-mpich not setting up correct compiler wrappers, fails hdf5 build
<!-- Thanks for taking the time to report this build failure. To proceed with the report please:

1. Title the issue "Installation issue: <name-of-the-package>".
2. Provide the information required below.

We encourage you to try, as much as possible, to reduce your problem to the minimal example that still reproduces the issue. That would help us a lot in fixing it quickly and effectively! -->

### Steps to reproduce the issue

I don't know who maintains mpich. The recent change from @alalazo seems to be causing this, but I could be wrong!

<!-- Fill in the exact spec you are trying to build and the relevant part of the error message -->
```console
$ spack install hdf5@1.10.2%intel@19.0.4.243 +mpi+hl
...
```

### Information on your system

<!-- Please include the output of `spack debug report` -->

<!-- If you have any relevant configuration detail (custom `packages.yaml` or `modules.yaml`, etc.) you can add that here as well. -->

### Additional information

Relevant packages.yaml to use cray-mpich module:
```yaml
packages:
  mpich:
    version: ['7.7.6']
    externals:
    - spec: mpich@7.7.6
      modules:
      - cray-mpich/7.7.6
    buildable: False
```

compilers.yaml for intel/19.0.4:
```yaml
compilers:
  - compiler:
      environment: {}
      extra_rpaths:
        - /opt/intel/2019.4-068/compilers_and_libraries_2019/linux/lib/intel64
      flags: {}
      modules:
        - PrgEnv-intel
        - intel/19.0.4
      operating_system: cnl6
      paths:
        cc: cc
        cxx: CC
        f77: ftn
        fc: ftn
      spec: intel@19.0.4.243
      target: any
```
<!-- Please upload the following files. They should be present in the stage directory of the failing build. Also upload any config.log or similar file if one exists. -->
Error output from Spack:
```
2 errors found in build log:
     28    checking for config ./config/site-specific/host-nid00091... no
     29    checking build mode... production
     30    checking for gcc... /opt/cray/pe/mpt/7.7.6/gni/mpich-intel/16.0/bin/
           mpicc
     31    checking whether the C compiler works... no
  >> 32    configure: error: in `/tmp/stam/spack-stage/spack-stage-hdf5-1.10.2-
           f3rktogdajbkilmejexzafbuiv4hpoec/spack-src':
  >> 33    configure: error: C compiler cannot create executables
     34    See `config.log' for more details
```

Error output from hdf5:
```
configure:4498: checking for C compiler version
configure:4507: /opt/cray/pe/mpt/7.7.6/gni/mpich-intel/16.0/bin/mpicc --version >&5
/tmp/stam/spack-stage/spack-stage-hdf5-1.10.2-f3rktogdajbkilmejexzafbuiv4hpoec/spack-src/configure: line 4509: /opt/cray/pe/mpt/7.7.6/gni/mpich-intel/16.0/bin/mpicc: No such file or directory
configure:4518: $? = 127
configure:4507: /opt/cray/pe/mpt/7.7.6/gni/mpich-intel/16.0/bin/mpicc -v >&5
/tmp/stam/spack-stage/spack-stage-hdf5-1.10.2-f3rktogdajbkilmejexzafbuiv4hpoec/spack-src/configure: line 4509: /opt/cray/pe/mpt/7.7.6/gni/mpich-intel/16.0/bin/mpicc: No such file or directory
configure:4518: $? = 127
configure:4507: /opt/cray/pe/mpt/7.7.6/gni/mpich-intel/16.0/bin/mpicc -V >&5
/tmp/stam/spack-stage/spack-stage-hdf5-1.10.2-f3rktogdajbkilmejexzafbuiv4hpoec/spack-src/configure: line 4509: /opt/cray/pe/mpt/7.7.6/gni/mpich-intel/16.0/bin/mpicc: No such file or directory
configure:4518: $? = 127
configure:4507: /opt/cray/pe/mpt/7.7.6/gni/mpich-intel/16.0/bin/mpicc -qversion >&5
/tmp/stam/spack-stage/spack-stage-hdf5-1.10.2-f3rktogdajbkilmejexzafbuiv4hpoec/spack-src/configure: line 4509: /opt/cray/pe/mpt/7.7.6/gni/mpich-intel/16.0/bin/mpicc: No such file or directory
configure:4518: $? = 127
configure:4538: checking whether the C compiler works
configure:4560: /opt/cray/pe/mpt/7.7.6/gni/mpich-intel/16.0/bin/mpicc  -fPIC     conftest.c  >&5
/tmp/stam/spack-stage/spack-stage-hdf5-1.10.2-f3rktogdajbkilmejexzafbuiv4hpoec/spack-src/configure: line 4562: /opt/cray/pe/mpt/7.7.6/gni/mpich-intel/16.0/bin/mpicc: No such file or directory
configure:4564: $? = 127
configure:4602: result: no
```

Looking at the recent changes to the mpich package, it looks like `if self.spec.external_modules and 'cray' in self.spec.external_modules:` isn't working as intended. I added some print statements in mpich/package.py to see where it's going wrong:

```python
    def setup_run_environment(self, env):
        # Because MPI implementations provide compilers, they have to add to
        # their run environments the code to make the compilers available.
        # For Cray MPIs, the regular compiler wrappers *are* the MPI wrappers.
        # Cray MPIs always have cray in the module name, e.g. "cray-mpich"
        if self.spec.external_modules and 'cray' in self.spec.external_modules:
            print("I'm in setup_run_environment")
            env.set('MPICC', spack_cc)
            env.set('MPICXX', spack_cxx)
            env.set('MPIF77', spack_fc)
            env.set('MPIF90', spack_fc)
        else:
            print("BROKEN: I'm in setup_run_environment")
            env.set('MPICC', join_path(self.prefix.bin, 'mpicc'))
            env.set('MPICXX', join_path(self.prefix.bin, 'mpic++'))
            env.set('MPIF77', join_path(self.prefix.bin, 'mpif77'))
            env.set('MPIF90', join_path(self.prefix.bin, 'mpif90'))
```

And I get:
```
==> Installing hdf5
==> No binary for hdf5 found: installing from source
==> Warning: Missing a source id for mpich@7.7.6
BROKEN: I'm in setup_run_environment
==> hdf5: Executing phase: 'autoreconf'
==> hdf5: Executing phase: 'configure'
```

<!-- Some packages have maintainers who have volunteered to debug build failures. Run `spack maintainers <name-of-the-package>` and @mention them here if they exist. -->

### General information

<!-- These boxes can be checked by replacing [ ] with [x] or by clicking them after submitting the issue. -->
- [ ] I have run `spack debug report` and reported the version of Spack/Python/Platform
- [ ] I have run `spack maintainers <name-of-the-package>` and @mentioned any maintainers
- [ ] I have uploaded the build log and environment files
- [ ] I have searched the issues of this repo and believe this is not a duplicate



Discussion:

Resolving patch:
diff --git a/var/spack/repos/builtin/packages/mpich/package.py b/var/spack/repos/builtin/packages/mpich/package.py
--- a/var/spack/repos/builtin/packages/mpich/package.py
+++ b/var/spack/repos/builtin/packages/mpich/package.py
@@ -187,7 +187,8 @@ def setup_run_environment(self, env):
         # their run environments the code to make the compilers available.
         # For Cray MPIs, the regular compiler wrappers *are* the MPI wrappers.
         # Cray MPIs always have cray in the module name, e.g. "cray-mpich"
-        if self.spec.external_modules and 'cray' in self.spec.external_modules:
+        external_modules = self.spec.external_modules
+        if external_modules and 'cray' in external_modules[0]:
             env.set('MPICC', spack_cc)
             env.set('MPICXX', spack_cxx)
             env.set('MPIF77', spack_fc)
@@ -210,7 +211,8 @@ def setup_dependent_build_environment(self, env, dependent_spec):
     def setup_dependent_package(self, module, dependent_spec):
         # For Cray MPIs, the regular compiler wrappers *are* the MPI wrappers.
         # Cray MPIs always have cray in the module name, e.g. "cray-mpich"
-        if self.spec.external_modules and 'cray' in self.spec.external_modules:
+        external_modules = self.spec.external_modules
+        if external_modules and 'cray' in external_modules[0]:
             self.spec.mpicc = spack_cc
             self.spec.mpicxx = spack_cxx
             self.spec.mpifc = spack_fc
diff --git a/var/spack/repos/builtin/packages/mvapich2/package.py b/var/spack/repos/builtin/packages/mvapich2/package.py
--- a/var/spack/repos/builtin/packages/mvapich2/package.py
+++ b/var/spack/repos/builtin/packages/mvapich2/package.py
@@ -240,7 +240,8 @@ def setup_dependent_build_environment(self, env, dependent_spec):
     def setup_compiler_environment(self, env):
         # For Cray MPIs, the regular compiler wrappers *are* the MPI wrappers.
         # Cray MPIs always have cray in the module name, e.g. "cray-mvapich"
-        if self.spec.external_modules and 'cray' in self.spec.external_modules:
+        external_modules = self.spec.external_modules
+        if external_modules and 'cray' in external_modules[0]:
             env.set('MPICC',  spack_cc)
             env.set('MPICXX', spack_cxx)
             env.set('MPIF77', spack_fc)
@@ -254,7 +255,8 @@ def setup_compiler_environment(self, env):
     def setup_dependent_package(self, module, dependent_spec):
         # For Cray MPIs, the regular compiler wrappers *are* the MPI wrappers.
         # Cray MPIs always have cray in the module name, e.g. "cray-mvapich"
-        if self.spec.external_modules and 'cray' in self.spec.external_modules:
+        external_modules = self.spec.external_modules
+        if external_modules and 'cray' in external_modules[0]:
             self.spec.mpicc = spack_cc
             self.spec.mpicxx = spack_cxx
             self.spec.mpifc = spack_fc


Test patch: