Repository: pytorch
Issue: Error compiling PyTorch: `/usr/bin/ld: cannot find -lmagma /usr/bin/ld: cannot find -lnvToolsExt`
Labels: module: build,module: cuda,triaged

## 🐛 Bug

<!-- A clear and concise description of what the bug is. -->

The Pytorch installed with pip returned an error message: 

`RuntimeError: CUDA error: no kernel image is available for execution on the device`. 

The solution described in [this issue](https://github.com/pytorch/pytorch/issues/31285) is to build PyTorch from the source code. So, I uninstalled PyTorch and proceeded with source compilation.

Following the [instructions to build from source](https://github.com/pytorch/pytorch#from-source), I got another error message:

```
[4538/5008] Building CXX object caffe2/CMakeFiles/vec256_test_all_types_DEFAULT.dir/__/aten/src/ATen/test/vec256_test_all_types.cpp.o
ninja: build stopped: subcommand failed.
Building wheel torch-1.7.0a0
-- Building version 1.7.0a0
cmake -GNinja -DBUILD_PYTHON=True -DBUILD_TEST=True -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/scratch/11568881/face_identification/pytorch/torch -DCMAKE_PREFIX_PATH=/scratch/11568881/miniconda3 -DJAVA_HOME=/scratch/11568881/miniconda3 -DNUMPY_INCLUDE_DIR=/scratch/11568881/miniconda3/lib/python3.8/site-packages/numpy/core/include -DPYTHON_EXECUTABLE=/scratch/11568881/miniconda3/bin//python3 -DPYTHON_INCLUDE_DIR=/scratch/11568881/miniconda3/include/python3.8 -DPYTHON_LIBRARY=/scratch/11568881/miniconda3/lib/libpython3.8.so.1.0 -DTORCH_BUILD_VERSION=1.7.0a0 -DUSE_NUMPY=True /scratch/11568881/face_identification/pytorch
cmake --build . --target install --config Release -- -j 16
Traceback (most recent call last):
  File "/scratch/11568881/face_identification/pytorch//setup.py", line 727, in <module>
    build_deps()
  File "/scratch/11568881/face_identification/pytorch//setup.py", line 314, in build_deps
    build_caffe2(version=version,
  File "/scratch/11568881/face_identification/pytorch/tools/build_pytorch_libs.py", line 62, in build_caffe2
    cmake.build(my_env)
  File "/scratch/11568881/face_identification/pytorch/tools/setup_helpers/cmake.py", line 346, in build
    self.run(build_args, my_env)
  File "/scratch/11568881/face_identification/pytorch/tools/setup_helpers/cmake.py", line 141, in run
    check_call(command, cwd=self.build_dir, env=env)
  File "/scratch/11568881/miniconda3/lib/python3.8/subprocess.py", line 364, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--target', 'install', '--config', 'Release', '--', '-j', '16']' returned non-zero exit status 1.
[build.log](https://github.com/pytorch/pytorch/files/6599834/build.log)
[build.log](https://github.com/pytorch/pytorch/files/6599837/build.log)
```

The full log is attached.

Python 3.8.8
CentOS 7
NVIDIA Tesla K20m
CUDA 10.1

## To Reproduce

Steps to reproduce the behavior:

1. Installed PyTorch:
```
$ pip install torch==1.7.1+cu101 torchvision==0.8.2+cu101 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html
```
2. Executed the commands in [this issue](https://github.com/pytorch/pytorch/issues/31285).
3. Got this error message: 
```
RuntimeError: CUDA error: no kernel image is available for execution on the device
```
4. Uninstalled PyTorch
5. Executed the following commands:
```
# Install dependencies
$ conda install numpy ninja pyyaml mkl mkl-include setuptools cmake cffi typing_extensions future six requests dataclasses

# Add LAPACK support for the GPU if needed
$ conda install -c pytorch magma-cuda101

# Download version 1.7 from GitHub
$ git clone https://github.com/pytorch/pytorch --branch 1.7 --single-branch

$ cd pytorch
$ git submodule sync
$ git submodule update --init --recursive

$ export CMAKE_PREFIX_PATH=${CONDA_PREFIX:-"$(dirname $(which conda))/../"}

# Install PyTorch
$ python3 setup.py install > build.log 2>&1
```
6. Got the error message described in the attached `build.log`.


<!-- If you have a code sample, error messages, stack traces, please provide it here as well -->

## Expected behavior

I expect to run PyTorch without errors.
<!-- A clear and concise description of what you expected to happen. -->

## Environment

Please copy and paste the output from our
[environment collection script](https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py)
(or fill out the checklist below manually).

You can get the script and run it with:
```
wget https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py
# For security purposes, please check the contents of collect_env.py before running it.
python collect_env.py
```

 - PyTorch Version (e.g., 1.0):  1.7
 - OS (e.g., Linux): CentOS 7.6.1810
 - How you installed PyTorch (`conda`, `pip`, source): first with `pip`, then source.
 - Build command you used (if compiling from source):
 - Python version: 3.8.8
 - CUDA/cuDNN version: 10.1.168
 - GPU models and configuration: 2 x Tesla K20m
 - Any other relevant information:

```
Collecting environment information...
PyTorch version: N/A
Is debug build: N/A
CUDA used to build PyTorch: N/A
ROCM used to build PyTorch: N/A

OS: CentOS Linux 7 (Core) (x86_64)
GCC version: (GCC) 7.3.0
Clang version: Could not collect
CMake version: version 3.19.6
Libc version: glibc-2.17

Python version: 3.8 (64-bit runtime)
Python platform: Linux-3.10.0-957.21.2.el7.x86_64-x86_64-with-glibc2.10
Is CUDA available: N/A
CUDA runtime version: 10.1.168
GPU models and configuration:
GPU 0: Tesla K20m
GPU 1: Tesla K20m

Nvidia driver version: 418.67
cuDNN version: /usr/lib64/libcudnn.so.7.6.0
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.18.1
[pip3] torch==1.7.1+cu101
[pip3] torchaudio==0.7.2
[pip3] torchvision==0.8.2+cu101
[conda] blas                      1.0                         mkl
[conda] cudatoolkit               10.1.243             h036e899_8    nvidia
[conda] libblas                   3.9.0                     8_mkl    conda-forge
[conda] libcblas                  3.9.0                     8_mkl    conda-forge
[conda] liblapack                 3.9.0                     8_mkl    conda-forge
[conda] liblapacke                3.9.0                     8_mkl    conda-forge
[conda] magma-cuda101             2.5.2                         1    pytorch
[conda] mkl                       2020.4             h726a3e6_304    conda-forge
[conda] mkl-include               2021.2.0           h06a4308_296
[conda] mkl-service               2.3.0            py38he904b0f_0
[conda] mkl_fft                   1.3.0            py38h54f3939_0
[conda] mkl_random                1.1.1            py38h0573a6f_0
[conda] numpy                     1.18.5                   pypi_0    pypi
[conda] numpy-base                1.19.2           py38hfa32c7d_0
```
## Additional context

<!-- Add any other context about the problem here. -->


cc @malfet @seemethere @walterddr @ngimel