Repository: spack/spack

Issue:
`llvm+gold` with external `binutils` build failure because of missing `plugin-api.h` header 
### Steps to reproduce

```
git clone -c feature.manyFiles=true https://github.com/spack/spack.git
spack install llvm@13.0.1
```

### Error message

```
==> Installing llvm-13.0.1-uz4giobhbnsvrlmkq6siezxk6hxedpuw
==> No binary for llvm-13.0.1-uz4giobhbnsvrlmkq6siezxk6hxedpuw found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/09/09c50d558bd975c41157364421820228df66632802a4a6a7c9c17f86a7340802.tar.gz
==> No patches needed for llvm
==> llvm: Executing phase: 'cmake'
==> llvm: Executing phase: 'build'
==> Error: ProcessError: Command exited with status 1:
    'ninja' '-j16' '-v'

2 errors found in build log:
...
  >> 4592    FAILED: tools/gold/CMakeFiles/LLVMgold.dir/gold-plugin.cpp.o
     4593    /home/rutvikc2/spack/lib/spack/env/gcc/g++  -D_GNU_SOURCE -D__STDC_CONSTANT_MACRO
             S -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Itools/gold -I/tmp/rutvikc2/spack
             -stage/spack-stage-llvm-13.0.1-uz4giobhbnsvrlmkq6siezxk6hxedpuw/spack-src/llvm/to
             ols/gold -Iinclude -I/tmp/rutvikc2/spack-stage/spack-stage-llvm-13.0.1-uz4giobhbn
             svrlmkq6siezxk6hxedpuw/spack-src/llvm/include -isystem /home/rutvikc2/spack/opt/s
             pack/linux-ubuntu20.04-skylake_avx512/gcc-9.3.0/zlib-1.2.11-ulsjxh4r33cuzn6puqa37
             zmpj4n54yyb/include -std=c++11 -fPIC -fno-semantic-interposition -fvisibility-inl
             ines-hidden -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings
              -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-
             fallthrough -Wno-maybe-uninitialized -Wno-class-memaccess -Wno-redundant-move -Wn
             o-pessimizing-move -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wsuggest-overrid
             e -Wno-comment -Wmisleading-indentation -fdiagnostics-color -ffunction-sections -
             fdata-sections -O3 -DNDEBUG -fPIC     -std=c++14 -MD -MT tools/gold/CMakeFiles/LL
             VMgold.dir/gold-plugin.cpp.o -MF tools/gold/CMakeFiles/LLVMgold.dir/gold-plugin.c
             pp.o.d -o tools/gold/CMakeFiles/LLVMgold.dir/gold-plugin.cpp.o -c /tmp/rutvikc2/s
             pack-stage/spack-stage-llvm-13.0.1-uz4giobhbnsvrlmkq6siezxk6hxedpuw/spack-src/llv
             m/tools/gold/gold-plugin.cpp
  >> 4594    /tmp/rutvikc2/spack-stage/spack-stage-llvm-13.0.1-uz4giobhbnsvrlmkq6siezxk6hxedpu
             w/spack-src/llvm/tools/gold/gold-plugin.cpp:38:10: fatal error: plugin-api.h: No 
             such file or directory
     4595       38 | #include <plugin-api.h>
     4596          |          ^~~~~~~~~~~~~~
     4597    compilation terminated.
```

### Information on your system

* **Spack:** 0.17.1-1457-4081597731
* **Python:** 3.8.10
* **Platform:** linux-ubuntu20.04-skylake_avx512
* **Concretizer:** clingo


### General information

- [X] I have run `spack debug report` and reported the version of Spack/Python/Platform
- [X] I have searched the issues of this repo and believe this is not a duplicate
- [X] I have run the failing commands in debug mode and reported the output


Discussion:
Note, I can build when using the command `spack install llvm @13.0.1 ~gold`, i.e. by disabling the gold plugin.
I can confirm the same issue also happens with `llvm@14.0.0`.
Summoning @scheibelp 
I got here after googling a different problem. I got the same error "plugin-api.h: No such file or directory" while building llvm tests (make check-all). The solution to my problem was installing the missing binutils-dev. Maybe this helps here too?
Thanks @ewokhias, that solved the problem! On my machine it detects `binutils` as external so it relies on the system-installed `binutils`, but for systems that don't have `binutils`. Spack should build it with `+headers` variant turned on. PR incoming.

Edit: PR draft here https://github.com/spack/spack/pull/30954
For what is worth, I got the same error when building llvm 8.0.0, so this isn't specific to 13+
I don't think we're gonna fix this in Spack. It's an issue in LLVM, they should check for the header s.t. it's an error caught during cmake/configure.

It happens when binutils is external and only runtime components are installed (e.g. binutils-dev is missing). If you're using externals you're telling Spack that it's a drop-in replacement for the Spack package.
> It happens when binutils is external and only runtime components are installed (e.g. binutils-dev is missing). If you're using externals you're telling Spack that it's a drop-in replacement for the Spack package.

That's indeed my case, but perhaps there should be a spack variant of binutils with only the runtime, to match what distro do?
> I don't think we're gonna fix this in Spack. It's an issue in LLVM, they should check for the header s.t. it's an error caught during cmake/configure.
> 
> It happens when binutils is external and only runtime components are installed (e.g. binutils-dev is missing). If you're using externals you're telling Spack that it's a drop-in replacement for the Spack package.

This issue happens for me when Spack detects binutils as an external package itself (with `spack external find`). This is a bug in Spack -- it adds binutils as an external package that is clearly _not_ a drop-in replacement for the Spack package.

Resolving patch:
diff --git a/var/spack/repos/builtin/packages/binutils/package.py b/var/spack/repos/builtin/packages/binutils/package.py
--- a/var/spack/repos/builtin/packages/binutils/package.py
+++ b/var/spack/repos/builtin/packages/binutils/package.py
@@ -3,6 +3,7 @@
 #
 # SPDX-License-Identifier: (Apache-2.0 OR MIT)
 import os
+import pathlib
 import re
 
 import spack.build_systems.autotools
@@ -168,6 +169,20 @@ def determine_version(cls, exe):
         match = re.search(r"GNU (nm|readelf).* (\S+)", output)
         return Version(match.group(2)).dotted.up_to(3) if match else None
 
+    @classmethod
+    def determine_variants(cls, exes, version_str):
+        bin_dir = pathlib.Path(exes[0]).parent
+        include_dir = bin_dir.parent / "include"
+        plugin_h = include_dir / "plugin-api.h"
+
+        variants = "+gold" if find(str(bin_dir), "gold", recursive=False) else "~gold"
+        if find(str(include_dir), str(plugin_h), recursive=False):
+            variants += "+headers"
+        else:
+            variants += "~headers"
+
+        return variants
+
     def flag_handler(self, name, flags):
         spec = self.spec
 


Test patch: