[NEED HELP] Trouble with CUDA capability sm_86

Ilias_Giannakopoulos: <p>Hi all,</p>
<p>I am trying to train a network on my NVIDIA RTX 3070. I receive the following error:</p>
<p><em>NVIDIA GeForce RTX 3070 with CUDA capability sm_86 is not compatible with the current PyTorch installation. The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_70.</em><br>
<em>If you want to use the NVIDIA GeForce RTX 3070 GPU with PyTorch.</em></p>
<p>Any help will be appreciated!</p>

eqy: <p>It looks like your version of PyTorch is too old for compute capability 8.6 (Ampere). Are you able to update your installation or rebuild PyTorch?</p>

Ilias_Giannakopoulos: <p>Hi eqy,</p>
<p>I am trying with the latest stable version of PyTorch that works with CUDA 11.1 (also tried with 10.2, but didn’t have any luck).<br>
Could it be that NVIDIA 3070 works with CUDA 11.2 and higher, while PyTorch supports up to CUDA 11.1 for the moment?</p>
<p>Thanks in advance!</p>

eqy: <p>That seems strange as there should be Ampere support and has been for a while now IIRC. If you can build the latest version of PyTorch you can specify <code>TORCH_CUDA_ARCH_LIST="8.6"</code> in your environment to force it to build with SM 8.6 support.</p>

Ilias_Giannakopoulos: <p>Hi eqy,</p>
<p>Thanks a ton! I specified `TORCH_CUDA_ARCH_LIST on bash. After that I redownloaded PyTorch and the network seems to be training properly.</p>
<p>Best,</p>
<p>Ilias</p>

ptrblck: <p>NIT: <code>TORCH_CUDA_ARCH_LIST</code> will be used while building from source and won’t change the shipped compute capabilities in the binaries, which you can get via <code>print(torch.cuda.get_arch_list())</code>.</p>
<aside class="quote no-group" data-username="Ilias_Giannakopoulos" data-post="3" data-topic="120235">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ilias_giannakopoulos/48/7299_2.png" class="avatar"> Ilias_Giannakopoulos:</div>
<blockquote>
<p>also tried with 10.2, but didn’t have any luck</p>
</blockquote>
</aside>
<p>That’s expected, since Ampere GPUs need CUDA&gt;=11.0</p>
<aside class="quote no-group" data-username="Ilias_Giannakopoulos" data-post="3" data-topic="120235">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ilias_giannakopoulos/48/7299_2.png" class="avatar"> Ilias_Giannakopoulos:</div>
<blockquote>
<p>Could it be that NVIDIA 3070 works with CUDA 11.2 and higher, while PyTorch supports up to CUDA 11.1 for the moment?</p>
</blockquote>
</aside>
<p>No, the 3070 uses <code>sm_86</code>, which is natively supported in CUDA&gt;=11.1 and is binary compatible to <code>sm_80</code>, so would already work in CUDA=11.0.</p>
<p>In any case, good to hear it’s working now <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>

Ilias_Giannakopoulos: <p>Hi prtblck,</p>
<p>Thanks a lot for the clarifications. So the mistake was that I hadn’t specified <code>TORCH_CUDA_ARCH_LIST</code> on the PATH?</p>

ptrblck: <p>No, I don’t think so, as it doesn’t change the behavior of the binaries and is only used if you build PyTorch from source, which isn’t the case if I understand your workflow correctly.<br>
The original error message is raised, if you’ve installed a pip wheel or conda binary, which doesn’t support your architecture. Based on the message:</p>
<aside class="quote no-group" data-username="Ilias_Giannakopoulos" data-post="1" data-topic="120235">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ilias_giannakopoulos/48/7299_2.png" class="avatar"> Ilias_Giannakopoulos:</div>
<blockquote>
<p>The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_70.</p>
</blockquote>
</aside>
<p>it seems you were using a pip wheel with CUDA&lt;=10.2.</p>

Ilias_Giannakopoulos: <p>Indeed I did exactly that: pip with CUDA 10.2. I think I messed up the installation when I tried to download PyTorch with 11.1 later.</p>
<p>Thanks again for the help and the clarifications!</p>

rojas70: <aside class="quote no-group" data-username="Ilias_Giannakopoulos" data-post="7" data-topic="120235">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ilias_giannakopoulos/48/7299_2.png" class="avatar"> Ilias_Giannakopoulos:</div>
<blockquote>
<p><code>TORCH_CUDA_ARCH_LIST</code></p>
</blockquote>
</aside>
<p><a class="mention" href="/u/ptrblck">@ptrblck</a> I am having the same problem as <a class="mention" href="/u/ilias_giannakopoulos">@Ilias_Giannakopoulos</a> , I began by installing pytorch as indicated on their website:</p>
<pre><code class="lang-auto">pip3 install torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html
</code></pre>
<p>How might I verify if the pip wheel used CUDA&lt;=10.2? I wanted to learn how to check for this.</p>
<p>In fact, my problem is more nuanced. I can get the code to work when I run it on the python/ipython interpreters, but any time I try to debug with code or pycharm, I get the error message.</p>
<p>I have been on this for two weeks, trying and re-trying different combinations pytorch versions (and nvidia drivers/ cuda toolkit/libcudnn). I have checked many times the virtualenvironments I use and how I select them in code. I have tried everything I know except building from source, and have not been able to resolve this discrepancy on my system.</p>
<p>Environment Information: my setup tries to follow the Nvidia compatibility matrix: driver-470/toolkit 114/libcudnn8.2/pytorch1.9+cu111</p>
<p>PyTorch version: 1.9.0+cu111<br>
Is debug build: False<br>
CUDA used to build PyTorch: 11.1<br>
ROCM used to build PyTorch: N/A</p>
<p>OS: Ubuntu 20.04.2 LTS (x86_64)<br>
GCC version: (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0<br>
Clang version: Could not collect<br>
CMake version: Could not collect<br>
Libc version: glibc-2.31</p>
<p>Python version: 3.8.10 (default, Jun 2 2021, 10:49:15) [GCC 9.4.0] (64-bit runtime)<br>
Python platform: Linux-5.11.0-25-generic-x86_64-with-glibc2.29<br>
Is CUDA available: True<br>
CUDA runtime version: 11.4.100<br>
GPU models and configuration:<br>
GPU 0: NVIDIA GeForce RTX 3090<br>
GPU 1: NVIDIA GeForce RTX 3090</p>
<p>Nvidia driver version: 470.57.02<br>
cuDNN version: Could not collect<br>
HIP runtime version: N/A<br>
MIOpen runtime version: N/A</p>
<p>Versions of relevant libraries:<br>
[pip3] numpy==1.20.0<br>
[pip3] torch==1.9.0+cu111 &lt;---- Is this a problem? No cuda 114?<br>
[pip3] torchaudio==0.9.0<br>
[pip3] torchvision==0.10.0+cu111<br>
[conda] Could not collect</p>
<p>And sys.path looks good. Is there anything specific to IDEs that I may be missing on this?</p>

ptrblck: <p>Based on your <a href="https://github.com/pytorch/pytorch/issues/45021">cross-post</a> I would also assume that you pycharm is using another env with a different PyTorch installation.<br>
I would thus either create a new virtual env and reinstall PyTorch + pycharm there or make sure to uninstall all PyTorch installations in the current and base environment and reinstall it in the current env only.</p>

rojas70: <p>Thank you.</p>
<p>I discovered my virtual environment had problems. When I tried to install packages to it, they would be installed globally not locally. There was corruption throughout… I deleted it and started from scratch.</p>
<p>I came up with the following steps as a guide for anyone who would like to have a type of cheatsheet to verify their installation:</p>
<blockquote>
<p><strong>Nvidia Driver and CUDA Toolkit</strong></p>
<ol>
<li>If already installed, examine your Nvidia GPU driver version</li>
</ol>
<blockquote>
<blockquote>
<p>nvidia-smi</p>
</blockquote>
</blockquote>
<p>or</p>
<blockquote>
<blockquote>
<p>cat /proc/driver/nvidia/version</p>
</blockquote>
</blockquote>
<ol start="2">
<li>Learn its architecture</li>
</ol>
<p>sudo lshw -C display</p>
<ol start="3">
<li>Learn your current Linux kernel</li>
</ol>
<p>uname -a</p>
<ol start="4">
<li>Look up the Nvidia Compatibility Matrix to determine the correct driver, toolkit, and libcudnn</li>
</ol>
<p><a href="https://docs.nvidia.com/deeplearning/cudnn/support-matrix/index.html" class="inline-onebox" rel="noopener nofollow ugc">Support Matrix - NVIDIA Docs</a></p>
<p><a href="https://docs.nvidia.com/deeplearning/cudnn/support-matrix/index.html#cudnn-versions-linux" class="inline-onebox" rel="noopener nofollow ugc">Support Matrix - NVIDIA Docs</a> (gcc, glibc)</p>
<ol start="5">
<li>Install Driver</li>
</ol>
<p>sudo apt install nvidia-driver-XXX</p>
<ol start="6">
<li>Install CUDA Toolkit</li>
</ol>
<p><a href="https://developer.nvidia.com/cuda-downloads" rel="noopener nofollow ugc">https://developer.nvidia.com/cuda-downloads</a></p>
<ol start="7">
<li>Install libcudnnX (useful to do deep learning with cuda)</li>
</ol>
<p><a href="https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html" class="inline-onebox" rel="noopener nofollow ugc">Installation Guide - NVIDIA Docs</a></p>
<p>sudo apt install libcudnnX</p>
<ol start="8">
<li>Install pytorch</li>
</ol>
<ul>
<li>we will wait for this undtil you setup your virtualenv below.</li>
</ul>
<p><strong>Testing your system’s python setup</strong></p>
<ol>
<li>First note, the location of the system-wide python interpreter</li>
</ol>
<p>which python3</p>
<ol start="2">
<li>Note the location of teh system-wide pip</li>
</ol>
<p>which pip3</p>
<ol start="3">
<li>What packages are there globally (this command will also list packages that were installed via apt-get install)</li>
</ol>
<p>python3 -m pip list (or alternatively python3 -m pip freeze)</p>
<ol start="4">
<li>Create virtualenv if not yet created</li>
</ol>
<p>python3 -m venv <strong>name_for_your_env</strong></p>
<ol start="6">
<li>Usually, you will be asked to install the required files; normally the file “requirements.txt”. Examine it and become familiar with it. From <em>within your virtual environment</em> , install them via:</li>
</ol>
<blockquote>
<blockquote>
<p>python3 -m pip install -r requirements.txt</p>
</blockquote>
</blockquote>
<ol start="7">
<li>If not already installed, install pytorch.</li>
</ol>
<p>You can get the pip3/conda command from here. Most people recommend conda/docker installs. We are doing pip3 to have more flexiblity with the packages we need with different repos.</p>
<ul>
<li>go to <a href="https://pytorch.org/" rel="noopener nofollow ugc">https://pytorch.org/</a> (choose your config). You may get a command like:</li>
</ul>
<ol start="8">
<li>
<p>Note that if a package is properly installed, it should appear in your virtual_env/lib/pythonX.X/site-packages forlder.</p>
</li>
<li>
<p>Additionally, ensure your pythonpath is properly set (learn more about pythonpath/imports/sys.path here: <a href="https://chrisyeh96.github.io/2017/08/08/definitive-guide-python-imports.html" class="inline-onebox" rel="noopener nofollow ugc">The Definitive Guide to Python import Statements | Chris Yeh</a>)</p>
</li>
</ol>
<ul>
<li>
<p>pythonpath is a environment variable that contains paths to load python modules/scripts that are not binaries (i.e. located.</p>
</li>
<li>
<p>The pythonpath env variable is set in the .bashrc file found in your user folder (the user folder is located at ~/ and the “.” means it is a hidden file). Use your favorite editor to open it:</p>
</li>
</ul>
<blockquote>
<blockquote>
<p>emacs ~/.bashrc</p>
</blockquote>
</blockquote>
<ul>
<li>Look to see if you already set any pythonpath’s.</li>
</ul>
<p>export PYTHONPATH=$PYTHONPATH:/new/path1/goes/here:/new/path2/goes/here:</p>
<p><strong>Sanity Checks for torch/gpu</strong></p>
<ol>
<li>In your virtualenv, open a python interpreter:</li>
</ol>
<blockquote>
<blockquote>
<p>python3 (or even better ipython3 – you will need to install first pip3 install ipython).</p>
</blockquote>
</blockquote>
<ol start="2">
<li>Check the system path from which modules are loaded</li>
</ol>
<blockquote>
<blockquote>
<p>import sys</p>
</blockquote>
</blockquote>
<blockquote>
<blockquote>
<p>sys.path (should not see undesired paths here).</p>
</blockquote>
</blockquote>
<ol start="2">
<li>Import torch</li>
</ol>
<blockquote>
<blockquote>
<p>impor torch</p>
</blockquote>
</blockquote>
<ol start="3">
<li>Double check that this torch module is located inside your virtual environment</li>
</ol>
<blockquote>
<blockquote>
<p>import imp</p>
</blockquote>
</blockquote>
<blockquote>
<blockquote>
<p>imp.find_module(‘torch’) → should return a path in your virtualenv</p>
</blockquote>
</blockquote>
<ol start="4">
<li>Check the version of your torch module and cuda</li>
</ol>
<blockquote>
<blockquote>
<p>torch.<strong>version</strong></p>
</blockquote>
</blockquote>
<blockquote>
<blockquote>
<p>torch.version.cuda</p>
</blockquote>
</blockquote>
<ol start="5">
<li>Check the supported architectures</li>
</ol>
<p>torch.cuda.get_arch_list()</p>
<ol start="6">
<li>Check for the number of gpu detected</li>
</ol>
<blockquote>
<blockquote>
<p>torch.cuda.device_count()</p>
</blockquote>
</blockquote>
<ol start="7">
<li>Can you read the device?</li>
</ol>
<blockquote>
<blockquote>
<p>device=torch.device(‘cuda:0’) # 0 by default, if you have more gpu’s increase your index.</p>
</blockquote>
</blockquote>
</blockquote>

Edwardius: <p>Hi,</p>
<p>Having the exact same issue.</p>
<blockquote>
<p>NVIDIA GeForce RTX 3080 with CUDA capability sm_86 is not compatible with the current PyTorch installation.<br>
The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_70.</p>
</blockquote>
<p>How would you check the CUDA version within the pip wheel? Currently the I have torch 1.9.1 and torchvision 0.10.1 in the virtual environment. When doing ‘nvidia-smi’ I get a cuda version of 11.4. Any pointers?</p>

ptrblck: <p><code>torch.cuda.get_arch_list()</code> will return the compute capabilities used in your current PyTorch build and <code>torch.version.cuda</code> will return the used CUDA runtime.</p>

Edwardius: <p>10.2 is the CUDA runtime currently being used. How would I fix this? Is there a way to specify the CUDA version used in the pip wheel?</p>

ptrblck: <p>Yes, you can select the CUDA version in <a href="https://pytorch.org/get-started/locally/">this UI</a>.</p>

Edwardius: <p>And it doesn’t matter that my root contains cuda 11.4 because I’ll be downloading this version into the venv?</p>

ptrblck: <p>It depends a bit on your use case. The pip wheels and conda binaries ship with their own CUDA runtime and you will be able to run PyTorch code without using the local CUDA toolkit (as long as the right CUDA runtime is selected; e.g. for Ampere GPUs you have to use CUDA&gt;=11).<br>
However, if you want to build a custom CUDA extension, the local CUDA toolkit will be used and you should install a matching version to the runtime used in PyTorch.</p>

Nakkhatra: <p>Hi <a class="mention" href="/u/ilias_giannakopoulos">@Ilias_Giannakopoulos</a> , I am facing the same issue with my RTX 3070 and I haven’t installed torch globally, I installed torch only inside a conda environment and I’m using ubuntu as OS. How can I add TORCH_CUDA_ARCH_LIST? Is it possible to add this command while installing with conda?</p>

Nakkhatra: <p><a class="mention" href="/u/ilias_giannakopoulos">@Ilias_Giannakopoulos</a> I tried<br>
TORCH_CUDA_ARCH_LIST=“8.6” conda install pytorch==1.8.0 torchvision==0.9.0 torchaudio==0.8.0 cudatoolkit=10.2 -c pytorch</p>
<p>but it didn’t work.</p>