PyTorch cannot found CUDA version 11.0 for NVIDIA GeForce RTX 2070 SUPER

csesaswati: <p>Pytorch cannot find the CUDA in my GPU enabled machine. The command <strong>!python -m torch.utils.collect_env</strong> returns the following information,</p>
<pre><code class="lang-auto">Collecting environment information...
/home/user/anaconda3/envs/tf-gpu/lib/python3.6/site-packages/torch/cuda/__init__.py:52: UserWarning: CUDA initialization: CUDA unknown error - this may be due to an incorrectly set up environment, e.g. changing env variable CUDA_VISIBLE_DEVICES after program start. Setting the available devices to be zero. (Triggered internally at  /opt/conda/conda-bld/pytorch_1607370120218/work/c10/cuda/CUDAFunctions.cpp:100.)
  return torch._C._cuda_getDeviceCount() &gt; 0
PyTorch version: 1.7.1
Is debug build: False
CUDA used to build PyTorch: 11.0
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.1 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
Clang version: Could not collect
CMake version: version 3.16.3

Python version: 3.6 (64-bit runtime)
Is CUDA available: False
CUDA runtime version: 10.1.243
GPU models and configuration: GPU 0: GeForce RTX 2070 SUPER
Nvidia driver version: 450.119.03
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.19.2
[pip3] torch==1.7.1
[pip3] torchaudio==0.7.0a0+a853dff
[pip3] torchvision==0.8.2
[conda] blas                      1.0                         mkl  
[conda] cudatoolkit               11.0.221             h6bb024c_0  
[conda] mkl                       2020.2                      256  
[conda] mkl-service               2.3.0            py36he8ac12f_0  
[conda] mkl_fft                   1.3.0            py36h54f3939_0  
[conda] mkl_random                1.1.1            py36h0573a6f_0  
[conda] numpy                     1.19.2           py36h54aff64_0  
[conda] numpy-base                1.19.2           py36hfa32c7d_0  
[conda] pytorch                   1.7.1           py3.6_cuda11.0.221_cudnn8.0.5_0    pytorch
[conda] torchaudio                0.7.2                      py36    pytorch
[conda] torchvision               0.8.2                py36_cu110    pytorch
</code></pre>
<p><strong>torch.cuda.is_available()</strong> returned <strong>False</strong> and <strong>print(torch.backends.cudnn.enabled)</strong> returned <strong>True</strong></p>
<p>I can found the following specification using nvidia-smi commands</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/c/3c31ec11ee1765d0f6e891a14cc91492512c6a25.png" data-download-href="https://discuss.pytorch.org/uploads/default/3c31ec11ee1765d0f6e891a14cc91492512c6a25" title="Screenshot from 2024-03-27 15-44-06"><img src="https://discuss.pytorch.org/uploads/default/original/3X/3/c/3c31ec11ee1765d0f6e891a14cc91492512c6a25.png" alt="Screenshot from 2024-03-27 15-44-06" data-base62-sha1="8AvBzdKVLRDIQEinRDVtdWghGex" width="690" height="376" data-dominant-color="3C1831"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot from 2024-03-27 15-44-06</span><span class="informations">741×404 45.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>The GPU version of my machine is <strong>NVIDIA Corporation TU104 [GeForce RTX 2070 SUPER]</strong></p>
<p>How can I solve this problem?</p>

ptrblck: <aside class="quote no-group" data-username="csesaswati" data-post="1" data-topic="200261">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/c/35a633/48.png" class="avatar"> csesaswati:</div>
<blockquote>
<p><code>CUDA used to build PyTorch: 11.0</code></p>
</blockquote>
</aside>
<aside class="quote no-group" data-username="csesaswati" data-post="1" data-topic="200261">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/c/35a633/48.png" class="avatar"> csesaswati:</div>
<blockquote>
<p><code>CUDA runtime version: 10.1.243</code></p>
</blockquote>
</aside>
<p>It seems you are running into a version mismatch, so maybe try to create an empty and clean virtual environment and install the latest PyTorch release there. If PyTorch still has issues communicating with your GPU you might need to update the NVIDIA driver. CUDA 12.x requires &gt;=525.60.13 on Linux.</p>

csesaswati: <p>In the tf-gpu(user created environment name) environment I ran<br>
<strong>conda install pytorch==1.7.1 torchvision==0.8.2 torchaudio==0.7.2 cudatoolkit=11.0 -c pytorch</strong><br>
command to install PyTorch.</p>
<p>Also latest PyTorch requires Python 3.8 or later. CUDA 11.8. My CUDA version is not compatible with the latest requirement of PyTorch so I installed archived version.</p>
<p>If I do not update the driver what will be the PyTorch version I need to choose?</p>

ptrblck: <aside class="quote no-group" data-username="csesaswati" data-post="3" data-topic="200261">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/c/35a633/48.png" class="avatar"> csesaswati:</div>
<blockquote>
<p>Also latest PyTorch requires Python 3.8 or later. CUDA 11.8. My CUDA version is not compatible with the latest requirement of PyTorch so I installed archived version.</p>
</blockquote>
</aside>
<p>Your locally installed CUDA toolkit won’t be used as the PyTorch binaries ship with their own CUDA runtime. You would need to properly install a supported NVIDIA driver to execute PyTorch workloads on the GPU.</p>
<p>Was this setup working before at one point or is this a new workstation?</p>

csesaswati: <p>This is the 1st time I used PyTorch with CUDA setup. I found PyTorch 1.7 with CUDA 11.0 can be installed since a condo command version available on the PyTorch archive, so I tried it.</p>
<p>I am wondering if PyTorch 1.7 with CUDA 11.0 are compatible then why it is not work for me?</p>

ptrblck: <p>I don’t know why it’s not working for you.<br>
Using 470.82.01 as the driver and installing PyTorch via <code>conda install pytorch==1.7.0 cudatoolkit=11.0 -c pytorch</code> into a new and empty environment works for me:</p>
<pre data-code-wrap="python"><code class="lang-python">&gt;&gt;&gt; import torch
&gt;&gt;&gt; torch.__version__
'1.7.0'
&gt;&gt;&gt; torch.version.cuda
'11.0'
&gt;&gt;&gt; torch.randn(1).cuda()
tensor([0.8996], device='cuda:0')
</code></pre>

csesaswati: <p>I downgrade the CUDA version using the command</p>
<blockquote>
<p>conda install pytorch==1.7.1 torchvision==0.8.2 torchaudio==0.7.2 cudatoolkit=10.1 -c pytorch</p>
</blockquote>
<p>But still unable to find CUDA.</p>
<blockquote>
<p>!python -m torch.utils.collect_env<br>
PyTorch version: 1.7.1<br>
Is debug build: False<br>
CUDA used to build PyTorch: 10.1<br>
ROCM used to build PyTorch: N/A</p>
<p>OS: Ubuntu 20.04.1 LTS (x86_64)<br>
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0<br>
Clang version: Could not collect<br>
CMake version: version 3.16.3</p>
<p>Python version: 3.6 (64-bit runtime)<br>
Is CUDA available: False<br>
CUDA runtime version: 10.1.243<br>
GPU models and configuration: GPU 0: GeForce RTX 2070 SUPER<br>
Nvidia driver version: 450.119.03<br>
cuDNN version: Could not collect<br>
HIP runtime version: N/A<br>
MIOpen runtime version: N/A</p>
<p>Versions of relevant libraries:<br>
[pip3] numpy==1.19.2<br>
[pip3] torch==1.7.1<br>
[pip3] torchaudio==0.7.0a0+a853dff<br>
[pip3] torchvision==0.8.2<br>
[conda] blas                      1.0                         mkl<br>
[conda] cudatoolkit               10.1.243             h6bb024c_0<br>
[conda] mkl                       2020.2                      256<br>
[conda] mkl-service               2.3.0            py36he8ac12f_0<br>
[conda] mkl_fft                   1.3.0            py36h54f3939_0<br>
[conda] mkl_random                1.1.1            py36h0573a6f_0<br>
[conda] numpy                     1.19.2           py36h54aff64_0<br>
[conda] numpy-base                1.19.2           py36hfa32c7d_0<br>
[conda] pytorch                   1.7.1           py3.6_cuda10.1.243_cudnn7.6.3_0    pytorch<br>
[conda] torchaudio                0.7.2                      py36    pytorch<br>
[conda] torchvision               0.8.2                py36_cu101    pytorch</p>
</blockquote>
<p>Is there any special requirement while creating the environment on conda?</p>
<p>What would be the python version required with torch 1.7.1 and CUDA 10.1?</p>

csesaswati: <p>I created a clean environment using the command</p>
<blockquote>
<p>conda create --name experiment1</p>
</blockquote>
<p>Activate the environment</p>
<blockquote>
<p>conda activate experiment1</p>
</blockquote>
<p>install pytorch using the command</p>
<blockquote>
<p>conda install pytorch==1.7.0 cudatoolkit=11.0 -c pytorch</p>
</blockquote>
<p>Active python3 using</p>
<blockquote>
<p>python3</p>
</blockquote>
<p>Then used</p>
<pre><code class="lang-auto">&gt;&gt;&gt; import torch
&gt;&gt;&gt; torch.__version__
</code></pre>
<p>Which returned<br>
2.2.1+cu121’</p>
<p><code>&gt;&gt;&gt; torch.version.cuda</code></p>
<p>returned<br>
‘12.1’</p>

ptrblck: <p>Cross-post from <a href="https://discuss.pytorch.org/t/two-different-cudatoolkit-version-and-pytorch-version-for-same-conda-environment/200557/2">here</a>.</p>