Repository: spack/spack

Issue:
`llvm+gold` with external `binutils` build failure because of missing `plugin-api.h` header 
### Steps to reproduce

```
git clone -c feature.manyFiles=true https://github.com/spack/spack.git
spack install llvm@13.0.1
```

### Error message

```
==> Installing llvm-13.0.1-uz4giobhbnsvrlmkq6siezxk6hxedpuw
==> No binary for llvm-13.0.1-uz4giobhbnsvrlmkq6siezxk6hxedpuw found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/09/09c50d558bd975c41157364421820228df66632802a4a6a7c9c17f86a7340802.tar.gz
==> No patches needed for llvm
==> llvm: Executing phase: 'cmake'
==> llvm: Executing phase: 'build'
==> Error: ProcessError: Command exited with status 1:
    'ninja' '-j16' '-v'

2 errors found in build log:
...
  >> 4592    FAILED: tools/gold/CMakeFiles/LLVMgold.dir/gold-plugin.cpp.o
     4593    /home/rutvikc2/spack/lib/spack/env/gcc/g++  -D_GNU_SOURCE -D__STDC_CONSTANT_MACRO
             S -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Itools/gold -I/tmp/rutvikc2/spack
             -stage/spack-stage-llvm-13.0.1-uz4giobhbnsvrlmkq6siezxk6hxedpuw/spack-src/llvm/to
             ols/gold -Iinclude -I/tmp/rutvikc2/spack-stage/spack-stage-llvm-13.0.1-uz4giobhbn
             svrlmkq6siezxk6hxedpuw/spack-src/llvm/include -isystem /home/rutvikc2/spack/opt/s
             pack/linux-ubuntu20.04-skylake_avx512/gcc-9.3.0/zlib-1.2.11-ulsjxh4r33cuzn6puqa37
             zmpj4n54yyb/include -std=c++11 -fPIC -fno-semantic-interposition -fvisibility-inl
             ines-hidden -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings
              -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-
             fallthrough -Wno-maybe-uninitialized -Wno-class-memaccess -Wno-redundant-move -Wn
             o-pessimizing-move -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wsuggest-overrid
             e -Wno-comment -Wmisleading-indentation -fdiagnostics-color -ffunction-sections -
             fdata-sections -O3 -DNDEBUG -fPIC     -std=c++14 -MD -MT tools/gold/CMakeFiles/LL
             VMgold.dir/gold-plugin.cpp.o -MF tools/gold/CMakeFiles/LLVMgold.dir/gold-plugin.c
             pp.o.d -o tools/gold/CMakeFiles/LLVMgold.dir/gold-plugin.cpp.o -c /tmp/rutvikc2/s
             pack-stage/spack-stage-llvm-13.0.1-uz4giobhbnsvrlmkq6siezxk6hxedpuw/spack-src/llv
             m/tools/gold/gold-plugin.cpp
  >> 4594    /tmp/rutvikc2/spack-stage/spack-stage-llvm-13.0.1-uz4giobhbnsvrlmkq6siezxk6hxedpu
             w/spack-src/llvm/tools/gold/gold-plugin.cpp:38:10: fatal error: plugin-api.h: No 
             such file or directory
     4595       38 | #include <plugin-api.h>
     4596          |          ^~~~~~~~~~~~~~
     4597    compilation terminated.
```

### Information on your system

* **Spack:** 0.17.1-1457-4081597731
* **Python:** 3.8.10
* **Platform:** linux-ubuntu20.04-skylake_avx512
* **Concretizer:** clingo


### General information

- [X] I have run `spack debug report` and reported the version of Spack/Python/Platform
- [X] I have searched the issues of this repo and believe this is not a duplicate
- [X] I have run the failing commands in debug mode and reported the output


Discussion:
Note, I can build when using the command `spack install llvm @13.0.1 ~gold`, i.e. by disabling the gold plugin.
I can confirm the same issue also happens with `llvm@14.0.0`.
Summoning @scheibelp 
I got here after googling a different problem. I got the same error "plugin-api.h: No such file or directory" while building llvm tests (make check-all). The solution to my problem was installing the missing binutils-dev. Maybe this helps here too?
Thanks @ewokhias, that solved the problem! On my machine it detects `binutils` as external so it relies on the system-installed `binutils`, but for systems that don't have `binutils`. Spack should build it with `+headers` variant turned on. PR incoming.

Edit: PR draft here https://github.com/spack/spack/pull/30954
For what is worth, I got the same error when building llvm 8.0.0, so this isn't specific to 13+
I don't think we're gonna fix this in Spack. It's an issue in LLVM, they should check for the header s.t. it's an error caught during cmake/configure.

It happens when binutils is external and only runtime components are installed (e.g. binutils-dev is missing). If you're using externals you're telling Spack that it's a drop-in replacement for the Spack package.
> It happens when binutils is external and only runtime components are installed (e.g. binutils-dev is missing). If you're using externals you're telling Spack that it's a drop-in replacement for the Spack package.

That's indeed my case, but perhaps there should be a spack variant of binutils with only the runtime, to match what distro do?
> I don't think we're gonna fix this in Spack. It's an issue in LLVM, they should check for the header s.t. it's an error caught during cmake/configure.
> 
> It happens when binutils is external and only runtime components are installed (e.g. binutils-dev is missing). If you're using externals you're telling Spack that it's a drop-in replacement for the Spack package.

This issue happens for me when Spack detects binutils as an external package itself (with `spack external find`). This is a bug in Spack -- it adds binutils as an external package that is clearly _not_ a drop-in replacement for the Spack package.

Resolving patch:
diff --git a/var/spack/repos/builtin/packages/llvm/package.py b/var/spack/repos/builtin/packages/llvm/package.py
--- a/var/spack/repos/builtin/packages/llvm/package.py
+++ b/var/spack/repos/builtin/packages/llvm/package.py
@@ -254,7 +254,7 @@ class Llvm(CMakePackage, CudaPackage):
     depends_on("py-six", when="+lldb+python")
 
     # gold support, required for some features
-    depends_on("binutils+gold+ld+plugins", when="+gold")
+    depends_on("binutils+gold+ld+plugins+headers", when="+gold")
 
     # Older LLVM do not build with newer compilers, and vice versa
     with when("@16:"):


Test patch: