Repository: spack/spack

Issue:
Installation issue: amrex: Building for too many CUDA architectures
I am building `amrex` with `+cuda`. I specify `cuda_arch=75` since that is the only architecture I need. Yet it seems that `amrex` is built with many CUDA architectures, as seen in these lines of the log file – there are many `-gencode` options.

```
[ 98%] Building CUDA object Src/CMakeFiles/amrex.dir/Particle/AMReX_ParticleContainerBase.cpp.o
cd /tmp/eschnetter/spack-stage/spack-stage-amrex-21.02-c7s4awxn54dtoultvkzui5fotx5dtgzf/spack-build-c7s4awx/Src && /home/eschnetter/src/CarpetX/spack/opt/spack/linux-ubuntu18.04-skylake_avx512/gcc-10.2.0/cuda-11.2.0-63ngbzt2tj6swczbmzi4kspn3fx3pvyx/bin/nvcc -forward-unknown-to-host-compiler -ccbin=/home/eschnetter/src/CarpetX/Cactus/view-cuda-compilers/bin/g++ -Damrex_EXPORTS -I/tmp/eschnetter/spack-stage/spack-stage-amrex-21.02-c7s4awxn54dtoultvkzui5fotx5dtgzf/spack-src/Src/Base -I/tmp/eschnetter/spack-stage/spack-stage-amrex-21.02-c7s4awxn54dtoultvkzui5fotx5dtgzf/spack-src/Src/Boundary -I/tmp/eschnetter/spack-stage/spack-stage-amrex-21.02-c7s4awxn54dtoultvkzui5fotx5dtgzf/spack-src/Src/AmrCore -I/tmp/eschnetter/spack-stage/spack-stage-amrex-21.02-c7s4awxn54dtoultvkzui5fotx5dtgzf/spack-src/Src/Amr -I/tmp/eschnetter/spack-stage/spack-stage-amrex-21.02-c7s4awxn54dtoultvkzui5fotx5dtgzf/spack-src/Src/LinearSolvers/MLMG -I/tmp/eschnetter/spack-stage/spack-stage-amrex-21.02-c7s4awxn54dtoultvkzui5fotx5dtgzf/spack-src/Src/LinearSolvers/Projections -I/tmp/eschnetter/spack-stage/spack-stage-amrex-21.02-c7s4awxn54dtoultvkzui5fotx5dtgzf/spack-src/Src/Particle -I/tmp/eschnetter/spack-stage/spack-stage-amrex-21.02-c7s4awxn54dtoultvkzui5fotx5dtgzf/spack-build-c7s4awx -isystem=/home/eschnetter/src/CarpetX/spack/opt/spack/linux-ubuntu18.04-skylake_avx512/gcc-10.2.0/openmpi-4.0.5-tj2im63lhbnfvcstc4sowllpqjc4vsd3/include -m64 --expt-relaxed-constexpr --expt-extended-lambda -Wno-deprecated-gpu-targets -gencode=arch=compute_35,code=sm_35 -gencode=arch=compute_50,code=sm_50 -gencode=arch=compute_52,code=sm_52 -gencode=arch=compute_60,code=sm_60 -gencode=arch=compute_61,code=sm_61 -gencode=arch=compute_70,code=sm_70 -gencode=arch=compute_75,code=sm_75 -gencode=arch=compute_80,code=sm_80 -gencode=arch=compute_86,code=sm_86 -gencode=arch=compute_86,code=compute_86 -maxrregcount=255 -Xcudafe --diag_suppress=esa_on_defaulted_function_ignored --use_fast_math -Xcudafe --display_error_number --Wext-lambda-captures-this --Werror ext-lambda-captures-this --Werror cross-execution-space-call --generate-line-info --source-in-ptx -g -Xcompiler=-fPIC -Xcompiler=-fopenmp "-Xcompiler=-O0 -ggdb -Wall -Wno-sign-compare -Wno-unused-but-set-variable -Werror=return-type" -Xcompiler -pthread -std=c++14 -x cu -dc /tmp/eschnetter/spack-stage/spack-stage-amrex-21.02-c7s4awxn54dtoultvkzui5fotx5dtgzf/spack-src/Src/Particle/AMReX_ParticleContainerBase.cpp -o CMakeFiles/amrex.dir/Particle/AMReX_ParticleContainerBase.cpp.o
```

<!-- Thanks for taking the time to report this build failure. To proceed with the report please:

1. Title the issue "Installation issue: <name-of-the-package>".
2. Provide the information required below.

We encourage you to try, as much as possible, to reduce your problem to the minimal example that still reproduces the issue. That would help us a lot in fixing it quickly and effectively! -->

### Steps to reproduce the issue

<!-- Fill in the exact spec you are trying to build and the relevant part of the error message -->
```console
$ spack install amrex +cuda cuda_arch=75 ~fortran +openmp +particles +shared build_type=Debug
```

### Information on your system

```
$ spack debug report
* **Spack:** 0.16.1-1533-62f8087716
* **Python:** 3.6.9
* **Platform:** linux-ubuntu18.04-skylake_avx512
* **Concretizer:** original
```

### Additional information

<!-- Some packages have maintainers who have volunteered to debug build failures. Run `spack maintainers <name-of-the-package>` and @mention them here if they exist. -->

@mic84 @asalmgren

### General information

<!-- These boxes can be checked by replacing [ ] with [x] or by clicking them after submitting the issue. -->
- [x] I have run `spack debug report` and reported the version of Spack/Python/Platform
- [x] I have run `spack maintainers <name-of-the-package>` and @mentioned any maintainers
- [x] I have searched the issues of this repo and believe this is not a duplicate



Discussion:
@eschnett Thank you for reporting this. Can you try PR #22025 and see if it fixes this. Thank you.

Resolving patch:
diff --git a/var/spack/repos/builtin/packages/amrex/package.py b/var/spack/repos/builtin/packages/amrex/package.py
--- a/var/spack/repos/builtin/packages/amrex/package.py
+++ b/var/spack/repos/builtin/packages/amrex/package.py
@@ -169,7 +169,7 @@ def cmake_args(self):
             args.append('-DAMReX_CUDA_ERROR_CAPTURE_THIS=ON')
             args.append('-DAMReX_CUDA_ERROR_CROSS_EXECUTION_SPACE_CALL=ON')
             cuda_arch = self.spec.variants['cuda_arch'].value
-            args.append('-DCUDA_ARCH=' + self.get_cuda_arch_string(cuda_arch))
+            args.append('-DAMReX_CUDA_ARCH=' + self.get_cuda_arch_string(cuda_arch))
 
         if '+rocm' in self.spec:
             args.append('-DCMAKE_CXX_COMPILER={0}'.format(self.spec['hip'].hipcc))


Test patch: