Repository: pytorch
Issue: Ubuntu 20.04 GeForce RTX 3070  / NVIDIA-SMI 460.73.01 Driver Version: 460.73.01 /  GPU Visibility Issue
Labels: module: binaries,module: cuda,triaged

## 🐛 Bug

<!-- A clear and concise description of what the bug is. -->

When running `CUDA_VISIBLE_DEVICES="0" python -c "import torch; torch.cuda.is_available()"`            

I get 
```
/home/jakub/miniconda3/lib/python3.8/site-packages/torch/cuda/__init__.py:52: UserWarning: CUDA initialization: CUDA unknown error - this may be due to an incorrectly set up environment, e.g. changing env variable CUDA_VISIBLE_DEVICES after program start. Setting the available devices to be zero. (Triggered internally at  /pytorch/c10/cuda/CUDAFunctions.cpp:109.)
  return torch._C._cuda_getDeviceCount() > 0

```

## To Reproduce
I've attempted many different ways to install Pytorch (incl. reinstalling CUDA/cuDNN/toolkit and compile from source). I would expect at least Docker or Conda to work. I can run other GPU applications in Docker, which makes me think my Docker installation is not the issue.

Steps to reproduce the behavior:

1. Docker 
`docker run --gpus all --rm -ti --ipc=host pytorch/pytorch:latest`
2. Pip `pip3 install torch torchvision torchaudio`
3. (Currently compiling from source), results now below
4. Conda
`conda install pytorch torchvision torchaudio cudatoolkit=11.1 -c pytorch -c nvidia`

<!-- If you have a code sample, error messages, stack traces, please provide it here as well -->
 
## Expected behavior

`CUDA_VISIBLE_DEVICES="0" python -c "import torch; torch.cuda.is_available()"`   returns `True`

<!-- A clear and concise description of what you expected to happen. -->

## Environment

Please copy and paste the output from our
[environment collection script](https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py)
(or fill out the checklist below manually).
PyTorch version: 1.8.1+cu111
Is debug build: False
CUDA used to build PyTorch: 11.1
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.2 LTS (x86_64)
GCC version: (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
Clang version: 10.0.0-4ubuntu1 
CMake version: version 3.16.3
Libc version: glibc-2.31

Python version: 3.8 (64-bit runtime)
Python platform: Linux-5.8.0-53-generic-x86_64-with-glibc2.10
Is CUDA available: False
CUDA runtime version: 11.2.152
GPU models and configuration: GPU 0: GeForce RTX 3070 Laptop GPU
Nvidia driver version: 460.73.01
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] efficientnet-pytorch==0.6.3
[pip3] numpy==1.18.1
[pip3] pytorch-lightning==1.3.1
[pip3] segmentation-models-pytorch==0.1.2
[pip3] torch==1.8.1+cu111
[pip3] torchaudio==0.8.1
[pip3] torchmetrics==0.3.2
[pip3] torchsummary==1.5.1
[pip3] torchvision==0.9.1+cu111
[conda] cudatoolkit               11.1.74              h6bb024c_0    nvidia
[conda] efficientnet-pytorch      0.6.3                    pypi_0    pypi
[conda] numpy                     1.20.0                   pypi_0    pypi
[conda] segmentation-models-pytorch 0.1.2                    pypi_0    pypi
[conda] torch                     1.8.1+cu111              pypi_0    pypi
[conda] torchaudio                0.8.1                    pypi_0    pypi
[conda] torchsummary              1.5.1                    pypi_0    pypi
[conda] torchvision               0.8.2                    pypi_0    pypi


You can get the script and run it with:
```
wget https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py
# For security purposes, please check the contents of collect_env.py before running it.
python collect_env.py
```

 - PyTorch Version (e.g., 1.0):
 - OS (e.g., Linux):
 - How you installed PyTorch (`conda`, `pip`, source):
 - Build command you used (if compiling from source):
 - Python version:
 - CUDA/cuDNN version: 
 - GPU models and configuration:
 - Any other relevant information:

## Additional context

<!-- Add any other context about the problem here. -->

Strangely, when I run 
```
nvcc -V                                                                                  
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Sun_Feb_14_21:12:58_PST_2021
Cuda compilation tools, release 11.2, V11.2.152
Build cuda_11.2.r11.2/compiler.29618528_0
```
The CUDA 11.2 does worry me, but e.g. here suggested that this not an issue:
https://medium.com/analytics-vidhya/install-cuda-11-2-cudnn-8-1-0-and-python-3-9-on-rtx3090-for-deep-learning-fcf96c95f7a1

Also, following this tutorial to the letter did not work for me.

cc @ezyang @seemethere @malfet @walterddr @ngimel