Torch.cuda.is_available() returns False with CUDA 12.6

username: pryce
cooked: <p>I’ve combed through a few forum posts on this topic, but none of the solutions I’ve seen have worked. I installed PyTorch to my environment with the following command: <code>pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124</code>.</p>
<p>Output from ‘torch.__version__’:</p>
<pre><code class="lang-auto">2.4.0+cpu
</code></pre>
<p>Output from ‘nvcc --version’:</p>
<pre><code class="lang-auto">nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Tue_Feb_27_16:28:36_Pacific_Standard_Time_2024
Cuda compilation tools, release 12.4, V12.4.99
Build cuda_12.4.r12.4/compiler.33961263_0
</code></pre>
<p>Output from ‘nvidia-smi’:</p>
<pre><code class="lang-auto">+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 560.94                 Driver Version: 560.94         CUDA Version: 12.6     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                  Driver-Model | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 3050 ...  WDDM  |   00000000:01:00.0 Off |                  N/A |
| N/A   45C    P0              9W /   40W |       0MiB /   4096MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+
</code></pre>
<p>I do not know why one of these shows 12.4 and the other shows 12.6.</p>
<p>I tried a fresh install of both torch and of the CUDA toolkit, neither of which had any effect. I saw in one forum post that rolling torch’s supported CUDA version back to 12.1 if you have 12.6 installed, but this did not work either. Am I missing something obvious?</p>

username: ptrblck
cooked: <aside class="quote no-group quote-modified" data-username="pryce" data-post="1" data-topic="209168">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/5e9695/48.png" class="avatar"> pryce:</div>
<blockquote>
<p>Output from ‘torch.<strong>version</strong>’:</p>
<pre><code class="lang-auto">2.4.0+cpu
</code></pre>
</blockquote>
</aside>
<p>You have installed the CPU-only binary and should install the PyTorch binary with CUDA support instead.</p>
<aside class="quote no-group" data-username="pryce" data-post="1" data-topic="209168">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/5e9695/48.png" class="avatar"> pryce:</div>
<blockquote>
<p>and of the CUDA toolkit</p>
</blockquote>
</aside>
<p>Your locally CUDA toolkit won’t be used unless you build PyTorch from source or a custom CUDA extension.</p>

username: pryce
cooked: <aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="209168" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>You have installed the CPU-only binary and should install the PyTorch binary with CUDA support instead.</p>
</blockquote>
</aside>
<p>This is what I’m confused about, I use the command provided in the ‘Start Locally’ portion of the PyTorch page, which yields the following on completion:</p>
<p>Successfully installed torch-2.4.1+cu124 torchaudio-2.4.1+cu124 torchvision-0.19.1+cu124<br>
However, when I run the program again, I get the same ‘2.4.0+cpu’ for version.</p>
<aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="209168" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>Your locally CUDA toolkit won’t be used unless you build PyTorch from source or a custom CUDA extension.</p>
</blockquote>
</aside>
<p>Is the toolkit interfering with torch in some way?</p>

username: ptrblck
cooked: <p>You’ve most likely installed multiple PyTorch binaries and the CPU-only one is being picked. Uninstall all previous PyTorch installations and reinstall the latest release with CUDA only.</p>
<aside class="quote no-group" data-username="pryce" data-post="3" data-topic="209168">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/5e9695/48.png" class="avatar"> pryce:</div>
<blockquote>
<p>Is the toolkit interfering with torch in some way?</p>
</blockquote>
</aside>
<p>No, since the PyTorch binary itself shows it does not have CUDA support: <code>2.4.0+cpu</code>.</p>

username: pryce
cooked: <p>That was the problem, thank you!</p>

username: Sai_Singireddy
cooked: <p>did it worked i have the same issue</p>

username: eaysu
cooked: <p>i have similar problem.</p>
<p>output from ‘torch.<strong>version</strong>’:</p>
<pre><code class="lang-auto">2.6.0.dev20241107+cu124
</code></pre>
<p>output from ‘nvcc --version’</p>
<pre><code class="lang-auto">nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Thu_Sep_12_02:18:05_PDT_2024
Cuda compilation tools, release 12.6, V12.6.77
Build cuda_12.6.r12.6/compiler.34841621_0
</code></pre>
<p>output from ‘nvidia-smi’:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/e/7eff8cb3f75c763b8f57c66009c92e47e32babd2.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/7eff8cb3f75c763b8f57c66009c92e47e32babd2" title="Ekran Resmi 2024-11-11 15.25.49"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/7/e/7eff8cb3f75c763b8f57c66009c92e47e32babd2_2_690x484.jpeg" alt="Ekran Resmi 2024-11-11 15.25.49" data-base62-sha1="i7tFM8TaBXOS3yV6CeBSyIzxs9c" width="690" height="484" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/7/e/7eff8cb3f75c763b8f57c66009c92e47e32babd2_2_690x484.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/7/e/7eff8cb3f75c763b8f57c66009c92e47e32babd2_2_1035x726.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/7/e/7eff8cb3f75c763b8f57c66009c92e47e32babd2_2_1380x968.jpeg 2x" data-dominant-color="7E8081"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Ekran Resmi 2024-11-11 15.25.49</span><span class="informations">1920×1349 151 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I think there is a mismatch between PyTorch and cuda versions. But I couldn’t find PyTorch version for 12.6.</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="eaysu" data-post="7" data-topic="209168">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/e/b5e925/48.png" class="avatar"> eaysu:</div>
<blockquote>
<p>I think there is a mismatch between PyTorch and cuda versions</p>
</blockquote>
</aside>
<p>This is irrelevant as already explained:</p>
<aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="209168">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>Your locally CUDA toolkit won’t be used unless you build PyTorch from source or a custom CUDA extension.</p>
</blockquote>
</aside>

username: eaysu
cooked: <p>Thanks for the information. However, if it does not work even though I downloaded PyTorch from official source, could the problem be related to the Ubuntu version of my system? (my ubuntu version is 24.04)</p>

username: ptrblck
cooked: <p>No, since Ubuntu 24.04 is working fine for me.<br>
I don’t know what kind of issues you are seeing, but in case the right PyTorch binary was installed but cannot communicate with the GPU, check if the NVIDIA drivers were properly installed.</p>

username: eaysu
cooked: <p>I’m sure I installed the nvidia drivers properly. The outputs of the “nvdia-smi” and “nvcc --version” commands match the versions I installed. But PyTorch cannot see these drivers.</p>

username: eaysu
cooked: <p>Update: Torch 2.5.0+cu124 runs successfully on my current system (ubuntu 24.04).</p>

username: coffeeZealot
cooked: <p>After so many hours of trying I finally got it working! I have PopOS and CUDA 12.6. All I had to do is update the driver (and restart the laptop):</p>
<p><code>sudo apt install system76-driver-nvidia</code></p>
<p>Torch version:</p>
<pre><code class="lang-auto">&gt;&gt;&gt; print(torch.__version__)
2.5.1
&gt;&gt;&gt; print(torch.version.cuda)
12.4
</code></pre>