Repository: spack/spack

Issue:
Installation issue: gcc with Xcode 13
### Steps to reproduce

After installing Xcode 13, which was released a few weeks ago, and its command line tools, gcc fails to build. I tried to build with command
```
spack install -j 1 gcc
```
(the `-j 1` option being to ensure ungarbled build output). The build error, from `spack-build-out.txt`, is:
```
echo | /private/var/folders/ps/s7z7d8m50qd1k_4ry_v88_mm0000gn/T/ben/spack-stage/spack-stage-gcc-11.2.0-pqdqj7f4ox2aw3kxfcc4vzfejibrtimz/spack-src/spack-build/./gcc/xgcc -B/private/var/folders/ps/s7z7d8m50qd1k_4ry_v88_mm0000gn/T/ben/spack-stage/spack-stage-gcc-11.2.0-pqdqj7f4ox2aw3kxfcc4vzfejibrtimz/spack-src/spack-build/./gcc/ -E -dM - | \
          sed -n -e 's/^#define \([^_][a-zA-Z0-9_]*\).*/\1/p' \
                 -e 's/^#define \(_[^_A-Z][a-zA-Z0-9_]*\).*/\1/p' | \
          sort -u > tmp-macro_list
dyld: Library not loaded: /usr/local/lib/libzstd.1.dylib
  Referenced from: /private/var/folders/ps/s7z7d8m50qd1k_4ry_v88_mm0000gn/T/ben/spack-stage/spack-stage-gcc-11.2.0-pqdqj7f4ox2aw3kxfcc4vzfejibrtimz/spack-src/spack-build/./gcc/cc1
  Reason: image not found
xgcc: internal compiler error: Abort trap: 6 signal terminated program cc1
```

It seems to be an issue with how the zstd library is linked to the binaries built by gcc. The command that builds `xgcc` is
```
/Users/ben/source/spack/lib/spack/env/clang/clang++ -std=c++11 -no-pie   -g  -DIN_GCC    -fno-strict-aliasing -fno-exceptions -fno-rtti -fasynchronous-unwind-tables -W -Wall -Wno-narrowing -Wwrite-strings -Wcast-qual -Wno-error=format-diag -Wno-format -Wmissing-format-attribute -Woverloaded-virtual -pedantic -Wno-long-long -Wno-variadic-macros -Wno-overlength-strings -fno-common  -DHAVE_CONFIG_H -Wl,-rpath,/Users/ben/opt/spack/darwin-bigsur-skylake/apple-clang-13.0.0/gcc-11.2.0-pqdqj7f4ox2aw3kxfcc4vzfejibrtimz/lib -Wl,-rpath,/Users/ben/opt/spack/darwin-bigsur-skylake/apple-clang-13.0.0/gcc-11.2.0-pqdqj7f4ox2aw3kxfcc4vzfejibrtimz/lib64 -Wl,-rpath,/Users/ben/opt/spack/darwin-bigsur-skylake/apple-clang-13.0.0/gmp-6.2.1-hy4uvsghpne2d6ya7mcwxcvxzh4i3bak/lib -Wl,-rpath,/Users/ben/opt/spack/darwin-bigsur-skylake/apple-clang-13.0.0/libiconv-1.16-fu7tfsrqgljwmgqfaf7zfehjbqhbr2vs/lib -Wl,-rpath,/Users/ben/opt/spack/darwin-bigsur-skylake/apple-clang-13.0.0/mpc-1.1.0-lu23k2uff3v34bahbb4yi2zh3vre2w7a/lib -Wl,-rpath,/Users/ben/opt/spack/darwin-bigsur-skylake/apple-clang-13.0.0/mpfr-4.1.0-f323tgxhee2umcfvobvlj4qi6ztyk7wh/lib -Wl,-rpath,/Users/ben/opt/spack/darwin-bigsur-skylake/apple-clang-13.0.0/zlib-1.2.11-xevvljjwbmyqlp45p465lfqqj7e6kjeg/lib -Wl,-rpath,/Users/ben/opt/spack/darwin-bigsur-skylake/apple-clang-13.0.0/zstd-1.5.0-oujwandath4wbabdvhy762bt3kcvdnyx/lib   -o xgcc gcc.o gcc-main.o ggc-none.o \
	  c/gccspec.o driver-i386.o darwin-driver.o libcommon-target.a \
	   libcommon.a ../libcpp/libcpp.a  -L/Users/ben/opt/spack/darwin-bigsur-skylake/apple-clang-13.0.0/libiconv-1.16-fu7tfsrqgljwmgqfaf7zfehjbqhbr2vs/lib -liconv ../libbacktrace/.libs/libbacktrace.a ../libiberty/libiberty.a ../libdecnumber/libdecnumber.a 
```
The correct directory for the Spack-built libzstd is in xgcc's RPATH (confirmed with `otool -l`), but the library doesn't load. Ostensibly the executable tries to load libzstd through `dlopen` (since libzstd isn't in the `otool -L` output), so it seems there's a loader path that's not correctly populated in the executable.

### Error message

```
==> gcc: Executing phase: 'autoreconf'
==> gcc: Executing phase: 'configure'
==> gcc: Executing phase: 'build'
lib/spack/spack/error.py:54 ==> [2021-10-02-20:49:24.380488] Error: ProcessError: Command exited with status 2:
    'make' 'V=1'

4 errors found in build log:
     545031      Referenced from: /private/var/folders/ps/s7z7d8m50qd1k_4ry_v88
               _mm0000gn/T/ben/spack-stage/spack-stage-gcc-11.2.0-pqdqj7f4ox2aw
               3kxfcc4vzfejibrtimz/spack-src/spack-build/./gcc/cc1
     545032      Reason: image not found
     545033    xgcc: internal compiler error: Abort trap: 6 signal terminated p
               rogram cc1
     545034    Please submit a full bug report,
     545035    with preprocessed source if appropriate.
     545036    See <https://github.com/spack/spack/issues> for instructions.
  >> 545037    make[3]: *** [s-selftest-c] Error 4
     545038    rm gcc.pod
  >> 545039    make[2]: *** [all-stage1-gcc] Error 2
  >> 545040    make[1]: *** [stage1-bubble] Error 2
  >> 545041    make: *** [all] Error 2

See build log for details:
  /var/folders/ps/s7z7d8m50qd1k_4ry_v88_mm0000gn/T/ben/spack-stage/spack-stage-gcc-11.2.0-pqdqj7f4ox2aw3kxfcc4vzfejibrtimz/spack-build-out.txt

Traceback (most recent call last):
  File "/Users/ben/source/spack/lib/spack/spack/build_environment.py", line 1029, in _setup_pkg_and_run
    return_value = function(pkg, kwargs)
  File "/Users/ben/source/spack/lib/spack/spack/installer.py", line 1883, in build_process
    return installer.run()
  File "/Users/ben/source/spack/lib/spack/spack/installer.py", line 1749, in run
    self._real_install()
  File "/Users/ben/source/spack/lib/spack/spack/installer.py", line 1847, in _real_install
    phase(pkg.spec, pkg.prefix)
  File "/Users/ben/source/spack/lib/spack/spack/package.py", line 116, in phase_wrapper
    phase(spec, prefix)
  File "/Users/ben/source/spack/lib/spack/spack/build_systems/autotools.py", line 352, in build
    inspect.getmodule(self).make(*params)
  File "/Users/ben/source/spack/lib/spack/spack/build_environment.py", line 148, in __call__
    return super(MakeExecutable, self).__call__(*args, **kwargs)
  File "/Users/ben/source/spack/lib/spack/spack/util/executable.py", line 225, in __call__
    raise ProcessError('Command exited with status %d:' %
spack.util.executable.ProcessError: Command exited with status 2:
    'make' 'V=1'
lib/spack/spack/installer.py:1337 ==> [2021-10-02-20:50:36.525367] Flagging gcc-11.2.0-pqdqj7f4ox2aw3kxfcc4vzfejibrtimz as failed: ProcessError: Command exited with status 2:
    'make' 'V=1'
lib/spack/spack/main.py:800 ==> [2021-10-02-20:50:36.590056] ChildError: ProcessError: Command exited with status 2:
    'make' 'V=1'
```

### Information on your system

* **Spack:** 0.16.2-4358-76ad001c56
* **Python:** 3.8.9
* **Platform:** darwin-bigsur-skylake
* **Concretizer:** clingo

My `compilers.yaml`:
```
compilers:
- compiler:
    spec: apple-clang@13.0.0
    paths:
      cc: /usr/bin/clang
      cxx: /usr/bin/clang++
      f77: null
      fc: null
    flags: {}
    operating_system: bigsur
    target: x86_64
    modules: []
    environment: {}
    extra_rpaths: []
```

### General information

- [X] I have run `spack debug report` and reported the version of Spack/Python/Platform
- [X] I have searched the issues of this repo and believe this is not a duplicate
- [X] I have run the failing commands in debug mode and reported the output


Discussion:
I verified this fails on my machine too.  Interestingly, I *also* hit an almost identical error while building ROOT today. That makes me think this is a problem with spack itself...

This error kind of reminds me of https://github.com/spack/spack/issues/21870 .
@benc303 Figured it out I think: it's a problem with the `libzstd` that's being built: grepping `otool -l libzstd.dylib` includes:
```
Load command 4
          cmd LC_ID_DYLIB
      cmdsize 56
         name /usr/local/lib/libzstd.1.dylib (offset 24)
   time stamp 1 Wed Dec 31 19:00:01 1969
      current version 1.5.0
compatibility version 1.0.0
```
so it thinks it's being installed to an absolute path that it doesn't exist at. I do see `/usr/local` as a default in the Makefile so there's possibly something wonky going on there. I'll keep digging.
OK @benc303 confirmed, the link command inside `libzstd` make file is explicitly setting `-install_name /usr/local/lib/libzstd.1.dylib` because `PREFIX` isn't passed to the `build` stage. I'll fix this. It does mean that this package *might* not be relocatable on mac, even if rpaths are adjusted.
This is a result of the change from cmake to makefile for zstd in #25610 @haampie . Presumably CMake is smarter about not hardcoding paths with `-install_name`, since it will even update rpaths when installing. I'm submitting a patch.

Resolving patch:
diff --git a/var/spack/repos/builtin/packages/zstd/package.py b/var/spack/repos/builtin/packages/zstd/package.py
--- a/var/spack/repos/builtin/packages/zstd/package.py
+++ b/var/spack/repos/builtin/packages/zstd/package.py
@@ -37,12 +37,17 @@ class Zstd(MakefilePackage):
     depends_on('lzma', when='+programs')
     depends_on('lz4', when='+programs')
 
+    def _make(self, *args):
+        # PREFIX must be defined on macOS even when building the library, since
+        # it gets hardcoded into the library's install_path
+        make('VERBOSE=1', 'PREFIX=' + self.prefix, '-C', *args)
+
     def build(self, spec, prefix):
-        make('-C', 'lib')
+        self._make('lib')
         if spec.variants['programs'].value:
-            make('-C', 'programs')
+            self._make('programs')
 
     def install(self, spec, prefix):
-        make('-C', 'lib', 'install', 'PREFIX={0}'.format(prefix))
+        self._make('lib', 'install')
         if spec.variants['programs'].value:
-            make('-C', 'programs', 'install', 'PREFIX={0}'.format(prefix))
+            self._make('programs', 'install')


Test patch: