PyTorch with CUDA10.2 on 2080Ti - no CUDA-capable device is detected

username: ViennaFlying
cooked: <p>Hi,</p>
<p>I try to have a working Windows 10 Installation of the “CUDA Toolkit 10.2.89” with my “RTX 2080Ti” graphics card.</p>
<p>I understand that for only consuming CUDA runtime services within other 3rd party apps like PyTorch, I only would need the Win 10 Nvidia driver, as PyTorch brings its own CUDA runtime.</p>
<p>Unfortunately that did not work with PyTorch, even when I have a PyTorch version installed with CU102.</p>
<p>I use miniconda3 and Python 3.9.18 and installed PyTorch with following command<br>
pip install torch==1.10.0+cu102 torchvision==0.11.0+cu102 -f <a href="https://download.pytorch.org/whl/torch_stable.html" rel="noopener nofollow ugc">https://download.pytorch.org/whl/torch_stable.html</a></p>
<p>(TT) C:\Users\Roman&gt;pip list<br>
Package           Version</p>
<hr>
<p>cupy-cuda102      12.3.0<br>
fastrlock         0.8.2<br>
numpy             1.26.2<br>
Pillow            10.1.0<br>
pip               23.3.1<br>
setuptools        69.0.2<br>
torch             1.10.0+cu102<br>
torchvision       0.11.0+cu102<br>
typing_extensions 4.9.0<br>
wheel             0.42.0</p>
<p>python -c “from torch.utils.cpp_extension import CUDA_HOME; print(CUDA_HOME)” → No CUDA runtime is found, using CUDA_HOME=‘C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2’<br>
python -c “import torch; print(torch.<strong>version</strong>)” → 1.10.0+cu102<br>
python -c “import torch; print(torch.torch.<strong>path</strong>)” → [‘C:\Users\Roman\miniconda3\envs\TT\lib\site-packages\torch’]<br>
python -c “import torch; print(torch.version.cuda)” → 10.2<br>
python -c “import torch; print(torch.backends.cudnn.version())” → 7605<br>
python -c “import torch; print(torch.cuda.is_available())” → False (expect True)<br>
python -c “import torch; print(torch.cuda.device_count())” → 0 (expect 1)<br>
python -c “import torch; print(torch.cuda.get_device_name(0))” → RuntimeError: No CUDA GPUs are available (expect “GeForce RTX 2080 Ti”)<br>
python -c “import torch; torch.cuda.set_device(0)” → RuntimeError: No CUDA GPUs are available<br>
python -c “import torch; print(torch.cuda.current_device())” → RuntimeError: No CUDA GPUs are available (expect 0)<br>
python -c “import torch; print(torch.zeros(1).cuda())” → RuntimeError: No CUDA GPUs are available</p>
<p>So PyTorch can not find my CUDA GPU with the error “No CUDA GPUs are available”.</p>
<p>I have tried that with many different Nvidia drivers with all the same results. Currently I use the latest Nvidia driver 546.29.</p>
<p>So I then tried to focus on the CUDA Toolkit only, and according to the CUDA wiki, I installed the CUDA Toolkit 10.2.89. This was a 3-part installation of “cuda_10.2.89_win10_network.exe”, “cuda_10.2.1_win10.exe” and “cuda_10.2.2_win10.exe”.<br>
I also installed the recommended “cuDNN 8.7.0 for CUDA 10.2” according to NVidia docs as “cudnn-windows-x86_64-8.7.0.84_cuda10-archive.zip” and extracted the "bin, “include”, “lib\x64” files into the corresponding “CUDA\v10.2” subfolders “bin”, “include” and “lib\x64”.</p>
<p>The RTX 2080Ti is a “Turing” card with a Compute Capability of “7.5” and should have support for CUDA Toolkit SDK 10.0 - 10.2.</p>
<p>Acording to NVidia docs the minimal “Windows x86_64 Driver Version” for CUDA Toolkit “CUDA 10.2.89” must be &gt;= 441.22<br>
I have installed the latest Desktop Win10 Nvidia driver 546.29, which is also shown in the output of “nvidia-smi.exe”:<br>
C:&gt;nvidia-smi.exe<br>
Wed Dec 13 20:32:02 2023<br>
±--------------------------------------------------------------------------------------+<br>
| NVIDIA-SMI 546.29                 Driver Version: 546.29       CUDA Version: 12.3     |<br>
|-----------------------------------------±---------------------±---------------------+<br>
| GPU  Name                     TCC/WDDM  | Bus-Id        Disp.A | Volatile Uncorr. ECC |<br>
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |<br>
|                                         |                      |               MIG M. |<br>
|=========================================+======================+======================|<br>
|   0  NVIDIA GeForce RTX 2080 Ti   WDDM  | 00000000:0E:00.0  On |                  N/A |<br>
|  0%   47C    P8              26W / 300W |   2280MiB / 11264MiB |      0%      Default |<br>
|                                         |                      |                  N/A |</p>
<p>The installed CUDA Toolkit is reported as V10.2.89:<br>
C:&gt;nvcc -V<br>
nvcc: NVIDIA (R) Cuda compiler driver<br>
Copyright (c) 2005-2019 NVIDIA Corporation<br>
Built on Wed_Oct_23_19:32:27_Pacific_Daylight_Time_2019<br>
Cuda compilation tools, release 10.2, V10.2.89</p>
<p>When I try to run “deviceQuery.exe” I also get a “no CUDA-capable device is detected” error:<br>
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\extras\demo_suite&gt;deviceQuery.exe<br>
deviceQuery.exe Starting…</p>
<p>CUDA Device Query (Runtime API) version (CUDART static linking)</p>
<p>cudaGetDeviceCount returned 100<br>
 → no CUDA-capable device is detected<br>
Result = FAIL</p>
<p>I also checked within “NVIDIA Control Panel” → “Manage 3D Settings” → “Global Settings” that the Setting “CUDA - GPUs” is set to “All” and the checkbox for “NVIDIA GeForce RTX 2080 Ti” is checked.<br>
I only have this ONE graphics card in this PC, and there are no IGPUs from the processor, which is a “AMD Ryzen 7 3700X 8-Core Processor”.<br>
Windows 10 Pro 22H2 19045.3803, Windows Feature Experience Pack 1000.19053.1000.0</p>
<p>Environment Variables (only list the variables relevant for this problem):<br>
C:\SET<br>
CommonProgramFiles=C:\Program Files\Common Files<br>
CommonProgramFiles(x86)=C:\Program Files (x86)\Common Files<br>
CommonProgramW6432=C:\Program Files\Common Files<br>
CUDA_PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2<br>
CUDA_PATH_V10_2=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2<br>
CUDNN_PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2<br>
DriverData=C:\Windows\System32\Drivers\DriverData<br>
HOMEDRIVE=C:<br>
HOMEPATH=\Users\Roman<br>
LD_LIBRARY_PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\lib\x64<br>
LOCALAPPDATA=C:\Users\Roman\AppData\Local<br>
NUMBER_OF_PROCESSORS=16<br>
NVCUDASAMPLES10_2_ROOT=C:\ProgramData\NVIDIA Corporation\CUDA Samples\v10.2<br>
NVCUDASAMPLES_ROOT=C:\ProgramData\NVIDIA Corporation\CUDA Samples\v10.2<br>
NVTOOLSEXT_PATH=C:\Program Files\NVIDIA Corporation\NvToolsExt<br>
OS=Windows_NT<br>
Path=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\bin;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\lib\x64;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\include;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\libnvvp;c:\windows\system32;c:\windows;c:\windows\system32\wbem;c:\windows\system32\windowspowershell\v1.0;c:\windows\system32\openssh;c:\windows\system32;c:\program files\dotnet;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files\Git\cmd;C:\Program Files\NVIDIA Corporation\Nsight Compute 2019.5.0;C:\Users\Roman\AppData\Local\Programs\Python\Python39\Scripts;C:\Users\Roman\AppData\Local\Programs\Python\Python39;C:\Users\Roman\AppData\Local\Microsoft\WindowsApps;C:\Users\Roman\AppData\Local\Programs\Microsoft VS Code\bin<br>
<a href="http://PATHEXT=.COM" rel="noopener nofollow ugc">PATHEXT=.COM</a>;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC<br>
PROCESSOR_ARCHITECTURE=AMD64<br>
PROCESSOR_IDENTIFIER=AMD64 Family 23 Model 113 Stepping 0, AuthenticAMD<br>
PROCESSOR_LEVEL=23<br>
PROCESSOR_REVISION=7100<br>
ProgramData=C:\ProgramData<br>
ProgramFiles=C:\Program Files<br>
ProgramFiles(x86)=C:\Program Files (x86)<br>
ProgramW6432=C:\Program Files<br>
SystemDrive=C:<br>
SystemRoot=C:\WINDOWS<br>
TEMP=C:\Users\Roman\AppData\Local\Temp<br>
TMP=C:\Users\Roman\AppData\Local\Temp<br>
windir=C:\WINDOWS</p>
<p>I really run out of ideas what can be wrong with my setup?<br>
My problem seems NOT to be related to Python or PyTorch, as also from a pure “NVIDIA Developer view” it does not work.<br>
What else I can check?</p>
<p>Greetings,<br>
Roman (from Vienna)</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="ViennaFlying" data-post="1" data-topic="193684">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/v/49beb7/48.png" class="avatar"> ViennaFlying:</div>
<blockquote>
<p>My problem seems NOT to be related to Python or PyTorch, as also from a pure “NVIDIA Developer view” it does not work.</p>
</blockquote>
</aside>
<p>I agree as no application is able to find the device. Was this setup ever working? If so, what changed? Since the issue is not related to PyTorch I would recommend asking in e.g. the NVIDIA board additionally for more troubleshooting.</p>

username: ViennaFlying
cooked: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a> , thanks for coming back to me. No, this setup was never working on this windows installation/card. I already asked about my problem on the Nvidia developer forum <a href="https://forums.developer.nvidia.com/t/cuda-10-2-on-2080ti-no-cuda-capable-device-is-detected/276033" class="inline-onebox" rel="noopener nofollow ugc">Cuda 10.2 on 2080Ti - no CUDA-capable device is detected - CUDA Setup and Installation - NVIDIA Developer Forums</a></p>
<p>But as you seem to be an expert, I want to ask you to clarify some PyTorch questions for me, with regards to CUDA.</p>
<p>A)<br>
As I understand it, PyTorch brings its own cuda runtime, if you use a PyTorch version which is compiled with CUDA support (like the PyTorch+cuXXX versions).<br>
Does that mean for such a PyTorch version I do NOT need to install the Nvidia Cuda Toolkit, and I only need a recent Nvidia Video Driver installed, which brings the Nvidia side of the Cuda runtime?<br>
I only would need the Cuda Toolkit if I want to compile my own version of PyTorch with cuda support?</p>
<p>B)<br>
Can I only use such a PyTorch version, which is compiled with a supported CUDA SDK version matching my Nvidia’s Architecture and Compute Capabilities, or is there a backward capability?<br>
My Geforce RTX 2080Ti has a “Turing” architecture and is listed with a Compute Capability of “7.5”.<br>
According to <a href="https://en.wikipedia.org/wiki/CUDA#GPUs_supported" rel="noopener nofollow ugc">CUDA - Wikipedia</a> the Cuda SDK Support for my card then would be 10.0 - 10.2.<br>
Does that mean I only can use PyTorch which is compiled against Cuda 10.0 / 10.1 / 10.2, or can I also use e.g. “PyTorch+Cuda11.8” or “PyTorch+Cuda12.1” and that would be backwards compatible for 10.x?<br>
Or does the PyTorch compiled cuda version need to <em>exactly</em> match (and if yes against what version of which component on my windows system)? If I not need to have the Cuda runtime installed then against which component would it need to match?</p>
<p>Thanks for your clarifying answers,<br>
Roman</p>

username: ptrblck
cooked: <p>To A) Yes, your explanation is correct and the PyTorch binaries with CUDA support will install all needed CUDA runtime dependencies. You would only need to install an NVIDIA driver which supports the corresponding CUDA version.</p>
<p>To B)</p>
<aside class="quote no-group" data-username="ViennaFlying" data-post="3" data-topic="193684">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/v/49beb7/48.png" class="avatar"> ViennaFlying:</div>
<blockquote>
<p>My Geforce RTX 2080Ti has a “Turing” architecture and is listed with a Compute Capability of “7.5”.<br>
According to <a href="https://en.wikipedia.org/wiki/CUDA#GPUs_supported">CUDA - Wikipedia</a> the Cuda SDK Support for my card then would be 10.0 - 10.2.</p>
</blockquote>
</aside>
<p>The support matrix shows also that all CUDA 11 and 12 versions still support compute capability 7.5 and thus your Turing architecture.</p>
<aside class="quote no-group" data-username="ViennaFlying" data-post="3" data-topic="193684">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/v/49beb7/48.png" class="avatar"> ViennaFlying:</div>
<blockquote>
<p>Does that mean I only can use PyTorch which is compiled against Cuda 10.0 / 10.1 / 10.2, or can I also use e.g. “PyTorch+Cuda11.8” or “PyTorch+Cuda12.1” and that would be backwards compatible for 10.x?</p>
</blockquote>
</aside>
<p>You can use any current stable or nightly PyTorch binary with CUDA 11.8 and 12.1 as we are explicitly building PyTorch for Turing GPUs in these binaries. The binaries support compute capability 3.7 - 9.0 if PyTorch+CUDA 11.8 is installed and 5.0 - 9.0 if PyTorch+CUDA 12.1 is installed.<br>
We mostly don’t use backwards compatibility but build for these architectures explicitly. Compute capability 8.9 is an exception as it’s binary compatible to 8.6 as well as 8.0 and explicitly building for it won’t give you any advantage besides increasing the binary size.</p>

username: ViennaFlying
cooked: <p>Thanks for the comprehensive answers.</p>
<p>So then basically with a recent Nvidia driver I should be able to run any PyTorch+cuXXX binary with my 2080Ti.<br>
Unfortunately I still face the error “RuntimeError: No CUDA GPUs are available”, regardless what I try.</p>
<p>I already installed several Nvidia drivers (clean install), always with removing any previous driver with DDU. I also checked there were no stale contents in the registry or within the C:\Windows\System32\DriverStore\FileRepository directory.</p>
<p>My Computer has no internal iGPU processor.</p>
<p>Do you have any ideas what else could be the problem for this error?</p>
<p>Thanks,<br>
Roman</p>

username: ptrblck
cooked: <p>Not really as I’m not using Windows. In the past users were seeing similar issues when their laptop turned off the dedicated GPU to save power, but unsure if this could be related.</p>