Repository: spack/spack

Issue:
Installation issue: libflame fails to build on Cray EX with "undefined reference to `main'"
### Steps to reproduce the issue

<!-- Fill in the exact spec you are trying to build and the relevant part of the error message -->
```console
$ spack install amdlibflame threads=openmp
[...]
ranlib lib/x86_64-unknown-linux-gnu/libflame.a
/project/d110/timuel/spack/lib/spack/env/gcc/gcc -shared -Wl,-soname,libflame.so.3 -o lib/x86_64-unknown-linux-gnu/libflame.so -Wl,--whole-archive,libflame.a,--no-whole-archive -L/opt/cray/pe/mpich/8.1.4/ofi/gnu/9.1/lib -L/opt/cray/pe/dsmml/0.1.4/dsmml//lib -L/opt/cray/xpmem/2.2.40-7.0.1.0_1.9__g1d7a24d.shasta/lib64 -L/opt/gcc/10.2.0/snos/lib/gcc/x86_64-suse-linux/10.2.0 -L/opt/gcc/10.2.0/snos/lib/gcc/x86_64-suse-linux/10.2.0/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/opt/gcc/10.2.0/snos/lib/gcc/x86_64-suse-linux/10.2.0/../../.. -lblis-mt -lmpifort_gnu_91 -lmpi_gnu_91 -lxpmem -ldsmml -lgfortran -lquadmath -lpthread -lm -fopenmp
/usr/bin/ld: /usr/lib/../lib64/crt1.o: in function `_start':
/home/abuild/rpmbuild/BUILD/glibc-2.26/csu/../sysdeps/x86_64/start.S:110: undefined reference to `main'
collect2: error: ld returned 1 exit status
make: *** [Makefile:593: lib/x86_64-unknown-linux-gnu/libflame.so] Error 1
```

### Information on your system

```
* **Spack:** 0.16.2-3059-e321578bbe
* **Python:** 3.6.12
* **Platform:** cray-sles15-zen2
* **Concretizer:** original
```

```yaml
packages:
  ...
  cray-mpich:
    externals:
    - spec: cray-mpich@8.1.4%aocc
      prefix: /opt/cray/pe/mpich/8.1.4/ofi/aocc/2.2
    - spec: cray-mpich@8.1.4%gcc
      prefix: /opt/cray/pe/mpich/8.1.4/ofi/gnu/9.1
    buildable: False
...
```

```yaml
compilers:
- compiler:
    spec: gcc@10.2.0
    paths:
      cc: cc
      cxx: CC
      f77: ftn
      fc: ftn
    flags: {}
    operating_system: sles15
    target: any
    modules:
    - PrgEnv-gnu
    - gcc/10.2.0
    environment: {}
    extra_rpaths: []
```

### Additional information

<!-- Please upload the following files. They should be present in the stage directory of the failing build. Also upload any config.log or similar file if one exists. -->
* [spack-build-out.txt](https://github.com/spack/spack/files/6619683/spack-build-out.txt)
* [spack-build-env.txt](https://github.com/spack/spack/files/6619686/spack-build-env.txt)

@amd-toolchain-support ... my guess is that the Cray compiler wrappers interfere and switch to executable output mode with the `-o ...` occurring after the `-shared`, but that's really just a guess.

### General information

<!-- These boxes can be checked by replacing [ ] with [x] or by clicking them after submitting the issue. -->
- [x] I have run `spack debug report` and reported the version of Spack/Python/Platform
- [x] I have run `spack maintainers <name-of-the-package>` and @mentioned any maintainers
- [x] I have uploaded the build log and environment files
- [x] I have searched the issues of this repo and believe this is not a duplicate



Discussion:
Build and install of amdlibflame was successful with GCC compiler

Below command were tried:
spack install amdlibflame@3.0 threads=openmp

Issue seems to be with Fortran compilers used.

With below approach, one can ensure usage of right Fortran compilers (via spack) and install amdlibflame:
•	spack install gcc
•	gcc_install_path=$(spack cd -i gcc;pwd)
•	spack compiler add $gcc_install_path
•	spack install amdlibflame@3.0 threads=openmp

One need to ensure contents of $HOME/.spack/linux/compilers.yaml file should contain compiler values has depicted in the attached screen shot (Highlighted).
[
<img width="770" alt="spack_compilers_yaml_content" src="https://user-images.githubusercontent.com/73240730/121571467-af23aa80-ca40-11eb-9c97-8bb713f0882b.PNG">
](url)
While I believe that it works with a custom built vanilla gcc, this is not really an option here since I have to rely on the `cray-mpich` as well (which is afaik tied to the compiler to some extent).
As I suspected, the Cray compiler wrappers drop the `-shared` for some reason when forwarding the call:

```
$ /project/d110/timuel/spack/lib/spack/env/gcc/gcc -craype-verbose -o lib/x86_64-unknown-linux-gnu/libflame.so -shared -Wl,-soname,libflame.so.3 -Wl,--whole-archive,libflame.a,--no-whole-archive -L/opt/cray/pe/mpich/8.1.4/ofi/gnu/9.1/lib -L/opt/cray/pe/libsci/21.04.1.1/GNU/9.1/x86_64/lib -L/opt/cray/pe/dsmml/0.1.4/dsmml//lib -L/opt/cray/xpmem/2.2.40-7.0.1.0_1.9__g1d7a24d.shasta/lib64 -L/opt/gcc/10.2.0/snos/lib/gcc/x86_64-suse-linux/10.2.0 -L/opt/gcc/10.2.0/snos/lib/gcc/x86_64-suse-linux/10.2.0/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/opt/gcc/10.2.0/snos/lib/gcc/x86_64-suse-linux/10.2.0/../../.. -lsci_gnu_mpi_mp -lsci_gnu_mp -lsci_gnu_82_mpi -lsci_gnu_82 -ldl
-lmpifort_gnu_91 -lmpi_gnu_91 -lxpmem -ldsmml -lgfortran -lquadmath -lpthread -lm -fopenmp
gcc -march=znver2 -march=znver2 -mtune=znver2 -I/opt/cray/pe/libsci/21.04.1.1/GNU/9.1/x86_64/include -L/opt/cray/pe/mpich/8.1.4/ofi/gnu/9.1/lib -L/opt/cray/pe/libsci/21.04.1.1/GNU/9.1/x86_64/lib -L/opt/cray/pe/dsmml/0.1.4/dsmml//lib -L/opt/cray/xpmem/2.2.40-7.0.1.0_1.9__g1d7a24d.shasta/lib64 -L/opt/gcc/10.2.0/snos/lib/gcc/x86_64-suse-linux/10.2.0 -L/opt/gcc/10.2.0/snos/lib/gcc/x86_64-suse-linux/10.2.0/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/opt/gcc/10.2.0/snos/lib/gcc/x86_64-suse-linux/10.2.0/../../.. -L/opt/cray/pe/libsci/21.04.1.1/GNU/9.1/x86_64/lib -Wl,--disable-new-dtags -Wl,-rpath,/project/d110/timuel/spack/opt/spack/cray-sles15-zen2/gcc-10.2.0/amdlibflame-3.0-gvcr5zajxoq6blmvecoirkvtz7acnvno/lib -Wl,-rpath,/project/d110/timuel/spack/opt/spack/cray-sles15-zen2/gcc-10.2.0/amdlibflame-3.0-gvcr5zajxoq6blmvecoirkvtz7acnvno/lib64 -Wl,-rpath,/opt/cray/pe/libsci/21.04.1.1/GNU/9.1/x86_64/lib -Wl,-rpath,/opt/gcc/10.2.0/snos/lib/gcc/x86_64-suse-linux/10.2.0 -Wl,-rpath,/opt/gcc/10.2.0/snos/lib64 -o lib/x86_64-unknown-linux-gnu/libflame.so -Wl,-soname,libflame.so.3 -Wl,--whole-archive,libflame.a,--no-whole-archive -lsci_gnu_mpi_mp -lsci_gnu_mp -lsci_gnu_82_mpi -lsci_gnu_82 -ldl -lmpifort_gnu_91 -lmpi_gnu_91 -lxpmem -ldsmml -lgfortran -lquadmath -lpthread -lm -fopenmp
/usr/bin/ld: /usr/lib/../lib64/crt1.o: in function `_start':
/home/abuild/rpmbuild/BUILD/glibc-2.26/csu/../sysdeps/x86_64/start.S:110: undefined reference to `main'
collect2: error: ld returned 1 exit status
```

... and adding `--craype-prepend-opt=-shared` is the workaround. But this smells like a Cray compiler wrapper bug.
Replacing the shorthand notation

```
-Wl,--whole-archive,libflame.a,--no-whole-archive
```

with

```
-Wl,--whole-archive libflame.a -Wl,--no-whole-archive
```

fixes the issue.
Solved with the patch in the linked issue: https://patch-diff.githubusercontent.com/raw/flame/libflame/pull/52.patch

Resolving patch:
diff --git a/var/spack/repos/builtin/packages/amdlibflame/package.py b/var/spack/repos/builtin/packages/amdlibflame/package.py
--- a/var/spack/repos/builtin/packages/amdlibflame/package.py
+++ b/var/spack/repos/builtin/packages/amdlibflame/package.py
@@ -40,6 +40,7 @@ class Amdlibflame(LibflameBase):
     version('2.2', sha256='12b9c1f92d2c2fa637305aaa15cf706652406f210eaa5cbc17aaea9fcfa576dc')
 
     patch('aocc-2.2.0.patch', when="@:2.999", level=1)
+    patch('cray-compiler-wrapper.patch')
 
     provides('flame@5.2', when='@2:')
 


Test patch: