Pytorch not detecting NVIDIA RTX 3080Ti and CUDA 11.6 due to sm_86 compatibility issue

sage-khan: <p>Hello.</p>
<p>I installed torch as per getting started locally guide. I have a dual xeon system with RTX 3080Ti and CUDA 11.6, CUDNN8.4.1 … They are all working fine. However with torch I get error</p>
<p>NVIDIA GeForce RTX 3080 Ti with CUDA capability sm_86 is not compatible with the current PyTorch installation.<br>
The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_70.<br>
If you want to use the NVIDIA GeForce RTX 3080 Ti GPU with PyTorch, please check the instructions at <a href="https://pytorch.org/get-started/locally/" class="inline-onebox" rel="noopener nofollow ugc">Start Locally | PyTorch</a></p>
<p>I have already followed the instructions but there is no success.</p>
<p>The whole steps are as under:</p>
<p><strong>metanet@metanet-Precision-Tower-7810:~/ProgramFiles/speechbrain/recipes$ pip3 install torch torchvision torchaudio --extra-index-url </strong><br>
Defaulting to user installation because normal site-packages is not writeable<br>
Looking in indexes: https//pypiorg/simple, https//pypingcnvidiacom, https//downloadpytorchorg/whl/cu116<br>
Requirement already satisfied: torch in /home/metanet/.local/lib/python3.8/site-packages (1.11.0)<br>
Requirement already satisfied: torchvision in /home/metanet/.local/lib/python3.8/site-packages (0.12.0)<br>
Requirement already satisfied: torchaudio in /home/metanet/.local/lib/python3.8/site-packages (0.11.0)<br>
Requirement already satisfied: typing-extensions in /home/metanet/.local/lib/python3.8/site-packages (from torch) (4.2.0)<br>
Requirement already satisfied: pillow!=8.3.*,&gt;=5.3.0 in /home/metanet/.local/lib/python3.8/site-packages (from torchvision) (9.1.1)<br>
Requirement already satisfied: numpy in /home/metanet/.local/lib/python3.8/site-packages (from torchvision) (1.21.6)<br>
Requirement already satisfied: requests in /usr/lib/python3/dist-packages (from torchvision) (2.22.0)</p>
<p><strong>metanet@metanet-Precision-Tower-7810:~/ProgramFiles/speechbrain/recipes$ nvidia-smi</strong><br>
Sat Jul  2 11:24:12 2022<br>
±----------------------------------------------------------------------------+<br>
| NVIDIA-SMI 510.47.03    Driver Version: 510.47.03    CUDA Version: 11.6     |<br>
|-------------------------------±---------------------±---------------------+<br>
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |<br>
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |<br>
|                               |                      |               MIG M. |<br>
|===============================+======================+======================|<br>
|   0  NVIDIA GeForce …  On   | 00000000:03:00.0  On |                  N/A |<br>
| 30%   45C    P8    32W / 350W |    494MiB / 12288MiB |     13%      Default |<br>
|                               |                      |                  N/A |<br>
±------------------------------±---------------------±---------------------+</p>
<p>±----------------------------------------------------------------------------+<br>
| Processes:                                                                  |<br>
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |<br>
|        ID   ID                                                   Usage      |<br>
|=============================================================================|<br>
|    0   N/A  N/A      2298      G   /usr/lib/xorg/Xorg                190MiB |<br>
|    0   N/A  N/A      2713      G   /usr/bin/gnome-shell               51MiB |<br>
|    0   N/A  N/A      3746      G   …257163255691216877,131072       81MiB |<br>
|    0   N/A  N/A     23276      G   …veSuggestionsOnlyOnDemand       92MiB |<br>
|    0   N/A  N/A     24491      G   …b/thunderbird/thunderbird       74MiB |<br>
±----------------------------------------------------------------------------+</p>
<p>**metanet@metanet-Precision-Tower-7810:~/ProgramFiles/speechbrain/recipes$ cat check-device.py **<br>
import torch</p>
<p>torch.cuda.is_available()<br>
torch.cuda.current_device()<br>
torch.cuda.device(0)<br>
torch.cuda.device_count()<br>
torch.cuda.get_device_name(0)</p>
<p>device = torch.device(‘cuda’ if torch.cuda.is_available() else ‘cpu’)<br>
print(‘Using device:’, device)<br>
print()</p>
<p>if device.type == ‘cuda’:<br>
print(torch.cuda.get_device_name(0))<br>
print(‘Memory Usage:’)<br>
print(‘Allocated:’, round(torch.cuda.memory_allocated(0)/1024<strong>3,1), ‘GB’)<br>
print('Cached:   ', round(torch.cuda.memory_reserved(0)/1024</strong>3,1), ‘GB’)</p>
<p><strong>$ python3 check-device.py</strong><br>
/home/metanet/.local/lib/python3.8/site-packages/torch/cuda/<strong>init</strong>.py:145: UserWarning:<br>
NVIDIA GeForce RTX 3080 Ti with CUDA capability sm_86 is not compatible with the current PyTorch installation.<br>
The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_70.<br>
If you want to use the NVIDIA GeForce RTX 3080 Ti GPU with PyTorch, please check the instructions at <a href="https://pytorch.org/get-started/locally/" class="inline-onebox" rel="noopener nofollow ugc">Start Locally | PyTorch</a></p>
<p>warnings.warn(incompatible_device_warn.format(device_name, capability, " ".join(arch_list), device_name))<br>
Using device: cuda</p>
<p>NVIDIA GeForce RTX 3080 Ti<br>
Memory Usage:<br>
Allocated: 0.0 GB<br>
Cached:    0.0 GB</p>

ptrblck: <p>This error:</p>
<pre><code class="lang-auto">The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_70.
</code></pre>
<p>would mean that the installed binary isn’t shipping with <code>sm_86</code> and is thus most likely using the CUDA10.2 runtime while you would need CUDA11.<br>
Your <code>pip install</code> command doesn’t install anything as all requirements are already satisfied, so uninstall PyToch via <code>pip uninstall torch -y</code> and <code>conda uninstall pytorch -y</code> until no packages can be found and install the right one.</p>

MartinP: <p>I had the same error message when executing <code>print(torch.cuda.get_device_name(0))</code>.</p>
<p>I tried different CUDA versions: 11.0, 11.3, 11.6, and 11.7 for an Nvidia 3080 GPU but all had the same issue. In then end going from Python3.6 to Python3.7 solved the issue.</p>

ptrblck: <p>Most likely you did not install the right PyTorch binary with this CUDA runtime, but might have tried to install the CUDA toolkit manually?<br>
E.g. PyTorch with the CUDA 11.6 runtime works on my 3090 and Python 3.6 is already EOL and was already deprecated in mid. January.</p>