Repository: spack/spack

Issue:
broken compiler wrappers?
Looks like https://github.com/spack/spack/pull/8136 broke compile wrappers. 
At least I can not build `cmake` anymore, neither the `3.13.3` that I already had installed nor `3.13.4`. Most likely it's the same for any C++11 package.


### Steps to reproduce the issue

```console
$ spack install cmake
```

### Error Message


```console
==> Installing cmake
==> Searching for binary cache of cmake
==> Warning: No Spack mirrors are currently configured
==> No binary for cmake found: installing from source
==> Warning: A dependency has updated CPATH, this may lead pkg-config to assume that the package is part of the system includes and omit it when invoked with '--cflags'.
==> Using cached archive: /Users/davydden/spack/var/spack/cache/cmake/cmake-3.13.3.tar.gz
==> Staging archive: /Users/davydden/spack/var/spack/stage/cmake-3.13.3-i26zpy4o6pqnllgbe5jj4exhbem2wohw/cmake-3.13.3.tar.gz
==> Created stage in /Users/davydden/spack/var/spack/stage/cmake-3.13.3-i26zpy4o6pqnllgbe5jj4exhbem2wohw
==> Applied patch /Users/davydden/spack/var/spack/repos/builtin/packages/cmake/cmake-macos-add-coreservices.patch
==> Building cmake [Package]
==> Executing phase: 'bootstrap'
==> Error: ProcessError: Command exited with status 7:
    './bootstrap' '--prefix=/Users/davydden/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.0-apple/cmake-3.13.3-i26zpy4o6pqnllgbe5jj4exhbem2wohw' '--parallel=8' '--system-libs' '--no-system-jsoncpp' '--no-qt-gui' '--' '-DCMAKE_USE_OPENSSL=ON'

1 error found in build log:
     1    ==> Executing phase: 'bootstrap'
     2    ==> [2019-02-14-10:34:31.020304] './bootstrap' '--prefix=/Users/davydden/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.0-apple/cmake-3.13.3-i26zpy4o6pqnllgbe5jj4exhbem2wohw' '--parallel=8' '--system-libs' '--no-system-jsoncpp' '--no-qt-gui' '--' '-DCMAKE_USE_
          OPENSSL=ON'
     3    ---------------------------------------------
     4    CMake 3.13.3, Copyright 2000-2018 Kitware, Inc. and Contributors
     5    C compiler on this system is: /Users/davydden/spack/lib/spack/env/clang/clang
     6    ---------------------------------------------
  >> 7    Error when bootstrapping CMake:
     8    Cannot find a C++ compiler that supports both C++11 and the specified C++ flags.
     9    Please specify one using environment variable CXX.
     10   The C++ flags are "".
     11   They can be changed using the environment variable CXXFLAGS.
     12   See cmake_bootstrap.log for compilers attempted.
     13   ---------------------------------------------...
```

[spack-build.out.zip](https://github.com/spack/spack/files/2864322/spack-build.out.zip)

I do not set any flags from Spack configs.

### Information on your system

This includes:

 1. macOS 10.14.3.

```
$ clang --version
Apple LLVM version 10.0.0 (clang-1000.11.45.5)
Target: x86_64-apple-darwin18.2.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
```

@scheibelp @becker33 ping.


Discussion:
Just to confirm I reproduced this on the same system with a slightly different cmake version (3.13.2), and that reverting the commit for #8136 locally got cmake building again.

It did not seem to affect the build of `icu4c`, at least, the appropriate flags were added to the compile commands for C++ files.
Confirmed even on:
```console
$ uname -a
Linux nuvolari 4.15.0-45-generic #48-Ubuntu SMP Tue Jan 29 16:28:13 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
```
With the latest `develop` I can't install `cmake`. The installation stops for me at `libarchive` with the error:
```console
$ spack install cmake
[...]
==> Installing libarchive
==> Searching for binary cache of libarchive
==> Finding buildcaches in /home/mculpo/production/mirror/build_cache
==> No binary for libarchive found: installing from source
==> Warning: A dependency has updated CPATH, this may lead pkg-config to assume that the package is part of the system includes and omit it when invoked with '--cflags'.
==> Using cached archive: /home/mculpo/PycharmProjects/spack/var/spack/cache/libarchive/libarchive-3.3.2.tar.gz
==> Staging archive: /home/mculpo/PycharmProjects/spack/var/spack/stage/libarchive-3.3.2-2ujwczef7jqzs7mcf3ojfyaz5tzlvbjq/libarchive-3.3.2.tar.gz
==> Created stage in /home/mculpo/PycharmProjects/spack/var/spack/stage/libarchive-3.3.2-2ujwczef7jqzs7mcf3ojfyaz5tzlvbjq
==> No patches needed for libarchive
==> Building libarchive [AutotoolsPackage]
==> Executing phase: 'autoreconf'
==> Executing phase: 'configure'
==> Error: ProcessError: Command exited with status 1:
    '/home/mculpo/PycharmProjects/spack/var/spack/stage/libarchive-3.3.2-2ujwczef7jqzs7mcf3ojfyaz5tzlvbjq/libarchive-3.3.2/configure' '--prefix=/home/mculpo/PycharmProjects/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-8.2.0/libarchive-3.3.2-2ujwczef7jqzs7mcf3ojfyaz5tzlvbjq'

2 errors found in build log:
     12    checking build system type... x86_64-pc-linux-gnu
     13    checking host system type... x86_64-pc-linux-gnu
     14    checking for gcc... /home/mculpo/PycharmProjects/spack/lib/spack/env/gcc/gcc
     15    checking whether the C compiler works... yes
     16    checking for C compiler default output file name... a.out
     17    checking for suffix of executables...
  >> 18    checking whether we are cross compiling... configure: error: in `/tmp/pytest-of-mculpo/pytest-1/test_keep_exceptions0/tmp/spack-stage/
           spack-stage-wKdt_e/libarchive-3.3.2':
  >> 19    configure: error: cannot run C compiled programs.
     20    If you meant to cross compile, use `--host'.
     21    See `config.log' for more details

See build log for details:
  /home/mculpo/PycharmProjects/spack/var/spack/stage/libarchive-3.3.2-2ujwczef7jqzs7mcf3ojfyaz5tzlvbjq/libarchive-3.3.2/spack-build.out
```
Bisecting to look for the first bad commit:
```console
[...]
$ git bisect good
8ca384875efffbcee1881db9fdef4399e74add50 is the first bad commit
commit 8ca384875efffbcee1881db9fdef4399e74add50
Author: Peter Scheibel <scheibel1@llnl.gov>
Date:   Wed Feb 13 17:38:14 2019 -0600

    Dynamic library/include paths (#8136)
    
    Fixes #7855
    Closes #8070
    Closes #2645
    
    When searching for library directories (e.g. to add "-L" arguments to
    the compiler wrapper) Spack was only trying the "lib/" and "lib64/"
    directories for each dependency install prefix; this missed cases
    where packages would install libraries to subdirectories and also was
    not customizable. This PR makes use of the ".headers" and ".libs"
    properties for more-advanced location of header/library directories.
    Since packages can override the default behavior of ".headers" and
    ".libs", it also allows package writers to customize.
    
    The following environment variables which used to be set by Spack
    for a package build have been removed:
    
    * Remove SPACK_PREFIX and SPACK_DEPENDENCIES environment variables as
      they are no-longer used
    * Remove SPACK_INSTALL environment variable: it was not used before
      this PR

:040000 040000 cf6a4064ab42132ed573b066fc2b91fa69d30d9e 110daaf31ba81a593fc386f098d1a9994a14d8aa M	lib
:040000 040000 75b0a2785a0de3adaf2444fa726f86a0aa685e3c 394d269b2ee5bed504171a34e5ea0c8a07d6a464 M	var
```

Resolving patch:
diff --git a/lib/spack/spack/build_environment.py b/lib/spack/spack/build_environment.py
--- a/lib/spack/spack/build_environment.py
+++ b/lib/spack/spack/build_environment.py
@@ -287,10 +287,11 @@ def set_build_environment_variables(pkg, env, dirty):
         except spack.spec.NoLibrariesError:
             tty.debug("No libraries found for {0}".format(dep.name))
 
-        try:
-            include_dirs.extend(query.headers.directories)
-        except spack.spec.NoHeadersError:
-            tty.debug("No headers found for {0}".format(dep.name))
+        # TODO: fix the line below, currently the logic is broken for
+        # TODO: packages that uses directories as namespaces e.g.
+        # TODO: #include <boost/xxx.hpp>
+        # include_dirs.extend(query.headers.directories)
+
         if os.path.isdir(dep.prefix.include):
             include_dirs.append(dep.prefix.include)
 


Test patch:
diff --git a/lib/spack/spack/test/build_environment.py b/lib/spack/spack/test/build_environment.py
--- a/lib/spack/spack/test/build_environment.py
+++ b/lib/spack/spack/test/build_environment.py
@@ -243,7 +243,6 @@ def test_set_build_environment_variables(
     dep2_pkg = root['dt-diamond-right'].package
     dep2_pkg.spec.prefix = str(dep2_prefix)
     dep2_inc_paths = ['/test2/path/to/ex1.h', '/test2/path/to/subdir/ex2.h']
-    dep2_inc_dirs = ['/test2/path/to', '/test2/path/to/subdir']
     dep2_includes = HeaderList(dep2_inc_paths)
 
     setattr(dep_pkg, 'libs', dep_libs)
@@ -276,7 +275,7 @@ def normpaths(paths):
         # (regardless of whether it contains any header files)
         assert (
             normpaths(header_dir_var.split(':')) ==
-            normpaths(dep2_inc_dirs + [str(dep2_include)]))
+            normpaths([str(dep2_include)]))
     finally:
         delattr(dep_pkg, 'libs')
         delattr(dep2_pkg, 'headers')