Repository: spack/spack

Issue:
openmpi fails with intel@17.0.1
```
$ spack install -j1 openmpi@2.1.1%intel@17.0.1
==> xz is already installed in /home/certik/repos/spack/opt/spack/linux-fedora25-x86_64/intel-17.0.1/xz-5.2.3-ajkczqbkmo76meocrthfrwnbup3aclzk
==> zlib is already installed in /home/certik/repos/spack/opt/spack/linux-fedora25-x86_64/intel-17.0.1/zlib-1.2.11-27anjzuze64nyci6djr45njj4t2wjdz3
==> pkg-config is already installed in /home/certik/repos/spack/opt/spack/linux-fedora25-x86_64/intel-17.0.1/pkg-config-0.29.2-iln5fu3hz6i5lrnmqerngwzsjz4a4zwu
==> Installing libxml2
==> Using cached archive: /home/certik/repos/spack/var/spack/cache/libxml2/libxml2-2.9.4.tar.gz
==> Staging archive: /home/certik/repos/spack/var/spack/stage/libxml2-2.9.4-7x7qw2zw4yzxue3dbpbp34tajpzsxl7m/libxml2-2.9.4.tar.gz
==> Created stage in /home/certik/repos/spack/var/spack/stage/libxml2-2.9.4-7x7qw2zw4yzxue3dbpbp34tajpzsxl7m
==> No patches needed for libxml2
==> Building libxml2 [AutotoolsPackage]
==> Executing phase: 'autoreconf'
==> Executing phase: 'configure'
==> Executing phase: 'build'
==> Error: ProcessError: Command exited with status 2:
    'make'

4 errors found in build log:
     [ ... ]
     1147  icc: command line warning #10006: ignoring unknown option '-Wredundant-decls'
     1148  ld: warning: libimf.so, needed by ./.libs/libxml2.so, not found (try using -rpath or -rpath-link)
     1149  ld: warning: libsvml.so, needed by ./.libs/libxml2.so, not found (try using -rpath or -rpath-link)
     1150  ld: warning: libirng.so, needed by ./.libs/libxml2.so, not found (try using -rpath or -rpath-link)
     1151  ld: warning: libintlc.so.5, needed by ./.libs/libxml2.so, not found (try using -rpath or -rpath-link)
     1152  ld: .libs/xmllint: hidden symbol `__intel_cpu_features_init_x' in /opt/intel/compilers_and_libraries_2017.1.132/linux/compiler/lib/intel64_lin/libirc.a(cpu_feature_disp.o) is referenced by DSO
  >> 1153  ld: final link failed: Bad value
     1154  Makefile:1117: recipe for target 'xmllint' failed
  >> 1155  make[2]: *** [xmllint] Error 1
     1156  make[2]: Leaving directory '/tmp/certik/spack-stage/spack-stage-YzPtwr/libxml2-2.9.4'
     1157  Makefile:1440: recipe for target 'all-recursive' failed
  >> 1158  make[1]: *** [all-recursive] Error 1
     1159  make[1]: Leaving directory '/tmp/certik/spack-stage/spack-stage-YzPtwr/libxml2-2.9.4'
     1160  Makefile:859: recipe for target 'all' failed
  >> 1161  make: *** [all] Error 2

See build log for details:
```
The relevant part of the log is at the end:
```
  CCLD     testdso.la
  CC       xmllint.o
icc: command line warning #10006: ignoring unknown option '-Wno-format-extra-args'
icc: command line warning #10006: ignoring unknown option '-Wcast-align'
icc: command line warning #10006: ignoring unknown option '-Waggregate-return'
icc: command line warning #10006: ignoring unknown option '-Wnested-externs'
icc: command line warning #10006: ignoring unknown option '-Wredundant-decls'
  CCLD     xmllint
icc: command line warning #10006: ignoring unknown option '-Wno-format-extra-args'
icc: command line warning #10006: ignoring unknown option '-Wcast-align'
icc: command line warning #10006: ignoring unknown option '-Waggregate-return'
icc: command line warning #10006: ignoring unknown option '-Wnested-externs'
icc: command line warning #10006: ignoring unknown option '-Wredundant-decls'
ld: warning: libimf.so, needed by ./.libs/libxml2.so, not found (try using -rpath or -rpath-link)
ld: warning: libsvml.so, needed by ./.libs/libxml2.so, not found (try using -rpath or -rpath-link)
ld: warning: libirng.so, needed by ./.libs/libxml2.so, not found (try using -rpath or -rpath-link)
ld: warning: libintlc.so.5, needed by ./.libs/libxml2.so, not found (try using -rpath or -rpath-link)
ld: .libs/xmllint: hidden symbol `__intel_cpu_features_init_x' in /opt/intel/compilers_and_libraries_2017.1.132/linux/compiler/lib/intel64_lin/libirc.a(cpu_feature_disp.o) is referenced by DSO
ld: final link failed: Bad value
Makefile:1117: recipe for target 'xmllint' failed
make[2]: *** [xmllint] Error 1
make[2]: Leaving directory '/tmp/certik/spack-stage/spack-stage-YzPtwr/libxml2-2.9.4'
Makefile:1440: recipe for target 'all-recursive' failed
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory '/tmp/certik/spack-stage/spack-stage-YzPtwr/libxml2-2.9.4'
Makefile:859: recipe for target 'all' failed
make: *** [all] Error 2
```


Discussion:
Note that you can't specify to build libxml2 using gcc due to another bug #5707, but here is how you can workaround it in the meantime:
```
spack install openmpi@2.1.1%intel@17.0.1 ^hwloc%gcc@6.4.1
```
Unfortunately, the workaround also failed:
```
$ spack install openmpi@2.1.1%intel@17.0.1 ^hwloc%gcc@6.4.1==> hwloc is already installed in /home/certik/repos/spack/opt/spack/linux-fedora25-x86_64/gcc-6.4.1/hwloc-1.11.7-y3uqv3lpku25cg5ic62tddvymlqhcijz
==> Installing openmpi
==> Using cached archive: /home/certik/repos/spack/var/spack/cache/openmpi/openmpi-2.1.1.tar.bz2
==> Staging archive: /home/certik/repos/spack/var/spack/stage/openmpi-2.1.1-2zz3g6to6wx65dznjwikml5fra56axmi/openmpi-2.1.1.tar.bz2
==> Created stage in /home/certik/repos/spack/var/spack/stage/openmpi-2.1.1-2zz3g6to6wx65dznjwikml5fra56axmi
==> No patches needed for openmpi
==> Building openmpi [AutotoolsPackage]
==> Executing phase: 'autoreconf'
==> Executing phase: 'configure'
==> Executing phase: 'build'
==> Error: ProcessError: Command exited with status 2:
    'make' '-j28'

4 errors found in build log:
     [ ... ]
     9334    CCLD     ompi_info
     9335  ld: warning: libimf.so, needed by ../../../ompi/.libs/libmpi.so, not found (try using -rpath or -rpath-link)
     9336  ld: warning: libsvml.so, needed by ../../../ompi/.libs/libmpi.so, not found (try using -rpath or -rpath-link)
     9337  ld: warning: libirng.so, needed by ../../../ompi/.libs/libmpi.so, not found (try using -rpath or -rpath-link)
     9338  ld: warning: libintlc.so.5, needed by ../../../ompi/.libs/libmpi.so, not found (try using -rpath or -rpath-link)
     9339  ld: .libs/ompi_info: hidden symbol `__intel_cpu_features_init_x' in /opt/intel/compilers_and_libraries_2017.1.132/linux/compiler/lib/intel64_lin/libirc.a(cpu_feature_disp.o) is referenced by DSO
  >> 9340  ld: final link failed: Bad value
     9341  Makefile:1785: recipe for target 'ompi_info' failed
  >> 9342  make[2]: *** [ompi_info] Error 1
     9343  make[2]: Leaving directory '/tmp/certik/spack-stage/spack-stage-X1v1zo/openmpi-2.1.1/ompi/tools/ompi_info'
     9344  Makefile:3353: recipe for target 'all-recursive' failed
  >> 9345  make[1]: *** [all-recursive] Error 1
     9346  make[1]: Leaving directory '/tmp/certik/spack-stage/spack-stage-X1v1zo/openmpi-2.1.1/ompi'
     9347  Makefile:1806: recipe for target 'all-recursive' failed
  >> 9348  make: *** [all-recursive] Error 1

See build log for details:
  /home/certik/repos/spack/var/spack/stage/openmpi-2.1.1-2zz3g6to6wx65dznjwikml5fra56axmi/openmpi-2.1.1/spack-build.out
```
Regarding the failure to build libxml2 in https://github.com/LLNL/spack/issues/5705#issue-264747532: FWIW I can build `libxml2` with `intel@17.0.2`. While Spack should help you build it with `gcc` I want to make sure something else isn't going on here if you have the time to provide some extra info. A few requests:

* Could you post the full output of `spack spec` here (e.g. I'm curious what OS you are building for)? 
* Do you have any modules defined for the `intel@17.0.2` compiler in `compilers.yaml`?
* Do you have external packages defined in `packages.yaml` (with associated modules)?

Thanks!
Isn't that the LANL-specific #6 again? 
> Isn't that the LANL-specific #6 again?

@junghans are you referring to my comment https://github.com/LLNL/spack/issues/5705#issuecomment-335974196 or this issue? At this point I wasn't sure whether modules were being used so I wanted to rule out a possible issue there with my questions. 
I loaded the intel compilers using module load. It sets the LD_LIBRARY_PATH. The system is Fedora 25 I believe. I used the latest master of spack. Should I try with the development branch?I will send detailed info once I get to a computer.

Sent from my mobile phone.
Also, do I need to keep the intel compilers loaded when using spack? I thought once spack knows about them, then I don't need to have it loaded. I didn't have them loaded.What is the recommend way to use Intel compilers with spack?

Sent from my mobile phone.
> Should I try with the development branch?

Yes please. The master branch has not been updated in a long time.

> I loaded the intel compilers using module load. It sets the LD_LIBRARY_PATH.

Module loads which occur before the spack invocation will not affect LD_LIBRARY_PATH. It is possible to specify that a module is associated with a compiler and should be loaded when building with that compiler. The documentation section http://spack.readthedocs.io/en/latest/getting_started.html#compilers-requiring-modules contains an example of adding a module file to a compiler
@scheibelp this was very useful! I modified my `.spack/linux/compilers.yaml` to load the `intel/17.0.1` module for the Intel compiler. Now libxml2 builds, and it fails much later:
```
$ spack install openmpi@2.1.1%intel@17.0.1
...
==> Created stage in /home/certik/repos/spack/var/spack/stage/m4-1.4.18-o44horsk3u6gwzgy7hy7nlmsvtdopji6
==> Applied patch gnulib-pgi.patch
==> Building m4 [AutotoolsPackage]
==> Executing phase: 'autoreconf'
==> Executing phase: 'configure'
==> Executing phase: 'build'
==> Error: ProcessError: Command exited with status 2:
    'make' '-j28'

8 errors found in build log:
     [ ... ]
     635     CC       output.o
     636     CC       macro.o
     637     CC       path.o
     638     CC       symtab.o
     639     CCLD     m4
     640   symtab.o: In function `symtab_init':
  >> 641   symtab.c:(.text+0x41b): undefined reference to `__builtin_mul_overflow'
     642   ../lib/libm4.a(clean-temp.o): In function `create_temp_dir':
  >> 643   clean-temp.c:(.text+0x31d): undefined reference to `__builtin_mul_overflow'
     644   ../lib/libm4.a(fatal-signal.o): In function `at_fatal_signal':
  >> 645   fatal-signal.c:(.text+0x297): undefined reference to `__builtin_mul_overflow'
     646   ../lib/libm4.a(xmalloc.o): In function `xcalloc':
  >> 647   xmalloc.c:(.text+0x102): undefined reference to `__builtin_mul_overflow'
     648   ../lib/libm4.a(xmalloc.o): In function `xnmalloc':
  >> 649   xmalloc.c:(.text.xnmalloc[xnmalloc]+0x12): undefined reference to `__builtin_mul_overflow'
     650   ../lib/libm4.a(xmalloc.o):xmalloc.c:(.text.xnrealloc[xnrealloc]+0x20): more undefined references to `__builtin_mul_overflow' follow
     651   Makefile:1554: recipe for target 'm4' failed
  >> 652   make[2]: *** [m4] Error 1
     653   make[2]: Leaving directory '/tmp/certik/spack-stage/spack-stage-ig6fpd/m4-1.4.18/spack-build/src'
     654   Makefile:1572: recipe for target 'all-recursive' failed
  >> 655   make[1]: *** [all-recursive] Error 1
     656   make[1]: Leaving directory '/tmp/certik/spack-stage/spack-stage-ig6fpd/m4-1.4.18/spack-build'
     657   Makefile:1528: recipe for target 'all' failed
  >> 658   make: *** [all] Error 2

See build log for details:
  /home/certik/repos/spack/var/spack/stage/m4-1.4.18-o44horsk3u6gwzgy7hy7nlmsvtdopji6/m4-1.4.18/spack-build.out
```

This error also looks fishy, almost as if the intel compiler is still not working properly.
I saw this before: https://software.intel.com/en-us/forums/intel-c-compiler/topic/720757, basically `__builtin_mul_overflow` is a gcc-only function and intel doesn't have it.
I guess injecting `-no-gcc` into `CFLAGS` should do it...
Got m4 working! Thanks @junghans. Here is how:
```
spack install m4%intel@17.0.1 cflags="-no-gcc"
```
I can't progress with openmpi easily though, due to #5707.
Ok. So the problem is that the `cflags="-no-gcc"` fails if it is used for all packages. It should only be used for the `m4` package. As such, how do I build openmpi with this?

If I use the hash of the m4 package, as @tgamblin suggested, it fails:
```
$ spack find -l m4%intel
==> 1 installed packages.
-- linux-fedora25-x86_64 / intel@17.0.1 -------------------------
rva2tqg m4@1.4.18
$ spack spec openmpi%intel@17.0.1 ^/rva2tqg
Input spec
--------------------------------
openmpi%intel@17.0.1
    ^m4@1.4.18%intel@17.0.1 cflags="-no-gcc"  patches=3877ab548f88597ab2327a2230ee048d2d07ace1062efe81fc92e91b7f39cd00 +sigsegv arch=linux-fedora25-x86_64 
        ^libsigsegv@2.11%intel@17.0.1 cflags="-no-gcc"  arch=linux-fedora25-x86_64 

Normalized
--------------------------------
==> Error: Package m4 has no variant set(['patches'])!
```
And if I use the `cflags="-no-gcc"` flag for all packages, it also fails:
```
$ spack install openmpi%intel@17.0.1 cflags="-no-gcc"
==> libsigsegv is already installed in /home/certik/repos/spack/opt/spack/linux-fedora25-x86_64/intel-17.0.1/libsigsegv-2.11-pzp4rtuwsma6tbibxx4fqbcote5siyax
==> m4 is already installed in /home/certik/repos/spack/opt/spack/linux-fedora25-x86_64/intel-17.0.1/m4-1.4.18-rva2tqghknqxhbczrejivwods5cwctzk
==> libtool is already installed in /home/certik/repos/spack/opt/spack/linux-fedora25-x86_64/intel-17.0.1/libtool-2.4.6-wjj73wwo62w7t2blbm5nzv3fbjdmcie5
==> Installing pkg-config
==> Using cached archive: /home/certik/repos/spack/var/spack/cache/pkg-config/pkg-config-0.29.2.tar.gz
==> Staging archive: /home/certik/repos/spack/var/spack/stage/pkg-config-0.29.2-6vw3to5kyxg45f7oqtxxnkh3wl7aksa6/pkg-config-0.29.2.tar.gz
==> Created stage in /home/certik/repos/spack/var/spack/stage/pkg-config-0.29.2-6vw3to5kyxg45f7oqtxxnkh3wl7aksa6
==> No patches needed for pkg-config
==> Building pkg-config [AutotoolsPackage]
==> Executing phase: 'autoreconf'
==> Executing phase: 'configure'
==> Executing phase: 'build'
==> Error: ProcessError: Command exited with status 2:
    'make'

9 errors found in build log:
     [ ... ]
     633   giochannel.c(298): warning #1011: missing return statement at end of non-void function "g_io_error_get_from_g_error"
     634     }
     635     ^
     636   
     637     CC       libglib_2_0_la-gkeyfile.lo
     638     CC       libglib_2_0_la-glib-init.lo
  >> 639   glib-init.c(284): error: #error directive: Your platform/compiler is missing constructor support
     640     # error Your platform/compiler is missing constructor support
     641       ^
     642   
  >> 643   Internal error: null pointer
     644   
     645   compilation aborted for glib-init.c (code 4)
     646   Makefile:1478: recipe for target 'libglib_2_0_la-glib-init.lo' failed
  >> 647   make[6]: *** [libglib_2_0_la-glib-init.lo] Error 1
     648   make[6]: Leaving directory '/tmp/certik/spack-stage/spack-stage-pdbxSx/pkg-config-0.29.2/glib/glib'
     649   Makefile:1934: recipe for target 'all-recursive' failed
  >> 650   make[5]: *** [all-recursive] Error 1
     651   make[5]: Leaving directory '/tmp/certik/spack-stage/spack-stage-pdbxSx/pkg-config-0.29.2/glib/glib'
     652   Makefile:953: recipe for target 'all' failed
  >> 653   make[4]: *** [all] Error 2
     654   make[4]: Leaving directory '/tmp/certik/spack-stage/spack-stage-pdbxSx/pkg-config-0.29.2/glib/glib'
     655   Makefile:1046: recipe for target 'all-recursive' failed
  >> 656   make[3]: *** [all-recursive] Error 1
     657   make[3]: Leaving directory '/tmp/certik/spack-stage/spack-stage-pdbxSx/pkg-config-0.29.2/glib'
     658   Makefile:770: recipe for target 'all' failed
  >> 659   make[2]: *** [all] Error 2
     660   make[2]: Leaving directory '/tmp/certik/spack-stage/spack-stage-pdbxSx/pkg-config-0.29.2/glib'
     661   Makefile:707: recipe for target 'all-recursive' failed
  >> 662   make[1]: *** [all-recursive] Error 1
     663   make[1]: Leaving directory '/tmp/certik/spack-stage/spack-stage-pdbxSx/pkg-config-0.29.2'
     664   Makefile:463: recipe for target 'all' failed
  >> 665   make: *** [all] Error 2

See build log for details:
  /home/certik/repos/spack/var/spack/stage/pkg-config-0.29.2-6vw3to5kyxg45f7oqtxxnkh3wl7aksa6/pkg-config-0.29.2/spack-build.out
```
I tried to put the `cflags` into the `packages.yaml` file, but it gets rejected, that `cflags` is not supported. At this point, given that we know what the fix is, how do you make `spack` do the right thing, i.e. only set `cflags` for `m4`, but leave everything else intact?
> Got m4 working! Thanks @junghans. Here is how: `spack install m4%intel@17.0.1 cflags="-no-gcc"`

I guess we should add that to `m4`'s `package.py`.
@junghans do you know how to do that? I tried:
```diff
--- a/var/spack/repos/builtin/packages/m4/package.py
+++ b/var/spack/repos/builtin/packages/m4/package.py
@@ -51,6 +51,9 @@ class M4(AutotoolsPackage):
         if spec.satisfies('%clang') and not spec.satisfies('platform=darwin'):
             args.append('CFLAGS=-rtlib=compiler-rt')
 
+        if spec.satisfies('%intel'):
+            args.append('CFLAGS="-no-gcc"')
+
         if '+sigsegv' in spec:
             args.append('--with-libsigsegv-prefix={0}'.format(
                 spec['libsigsegv'].prefix))
```
and got:
```
$ spack install m4%intel@17.0.1         
==> libsigsegv is already installed in /home/certik/repos/spack/opt/spack/linux-fedora25-x86_64/intel-17.0.1/libsigsegv-2.11-mqfgjte225fmsksblygp73hr76tzytl2
==> Installing m4
==> Using cached archive: /home/certik/repos/spack/var/spack/cache/m4/m4-1.4.18.tar.gz
==> Already staged m4-1.4.18-o44horsk3u6gwzgy7hy7nlmsvtdopji6 in /home/certik/repos/spack/var/spack/stage/m4-1.4.18-o44horsk3u6gwzgy7hy7nlmsvtdopji6
==> Already patched m4
==> Building m4 [AutotoolsPackage]
==> Executing phase: 'autoreconf'
==> Executing phase: 'configure'
==> Error: ProcessError: Command exited with status 77:
    '/home/certik/repos/spack/var/spack/stage/m4-1.4.18-o44horsk3u6gwzgy7hy7nlmsvtdopji6/m4-1.4.18/configure' '--prefix=/home/certik/repos/spack/opt/spack/linux-fedora25-x86_64/intel-17.0.1/m4-1.4.18-o44horsk3u6gwzgy7hy7nlmsvtdopji6' '--enable-c++' 'CFLAGS="-no-gcc"' '--with-libsigsegv-prefix=/home/certik/repos/spack/opt/spack/linux-fedora25-x86_64/intel-17.0.1/libsigsegv-2.11-mqfgjte225fmsksblygp73hr76tzytl2'

2 errors found in build log:
     [ ... ]
     12    checking for gawk... gawk
     13    checking whether make sets $(MAKE)... yes
     14    checking whether make supports nested variables... yes
     15    checking whether make supports nested variables... (cached) yes
     16    checking for gcc... /opt/intel/compilers_and_libraries_2017.1.132/linux/bin/intel64/icc
     17    checking whether the C compiler works... no
  >> 18    configure: error: in `/tmp/certik/spack-stage/spack-stage-ig6fpd/m4-1.4.18/spack-build':
  >> 19    configure: error: C compiler cannot create executables
     20    See `config.log' for more details

See build log for details:
  /home/certik/repos/spack/var/spack/stage/m4-1.4.18-o44horsk3u6gwzgy7hy7nlmsvtdopji6/m4-1.4.18/spack-build.out
```

Resolving patch:
diff --git a/var/spack/repos/builtin/packages/m4/package.py b/var/spack/repos/builtin/packages/m4/package.py
--- a/var/spack/repos/builtin/packages/m4/package.py
+++ b/var/spack/repos/builtin/packages/m4/package.py
@@ -51,6 +51,9 @@ def configure_args(self):
         if spec.satisfies('%clang') and not spec.satisfies('platform=darwin'):
             args.append('CFLAGS=-rtlib=compiler-rt')
 
+        if spec.satisfies('%intel'):
+            args.append('CFLAGS=-no-gcc')
+
         if '+sigsegv' in spec:
             args.append('--with-libsigsegv-prefix={0}'.format(
                 spec['libsigsegv'].prefix))


Test patch: