Does Pytorch work with CUDA 12.5?

Juanouo: <p>Hello! I’m new to PyTorch with CUDA and I’m trying to set it up on WSL. My CUDA version is 12.5, but the version matrix goes up to 12.4. Is this outdated or should I downgrade my CUDA for Pytorch to work?</p>
<p>Thanks a lot</p>

ptrblck: <p>Your locally installed CUDA toolkit won’t be used unless you build PyTorch from source or a custom CUDA extension since the PyTorch binaries ship with its own CUDA dependencies.</p>

S_Houston: <p>I’m also having issues getting CUDA and PyTorch to work.</p>
<p>I have CUDA 12.5 installed and PyTorch 2.2 installed in my Anaconda environment, however when checking if my GPU is available it always returns FALSE.</p>
<p>I’m running this relatively simple script to check if available:</p>
<p>import torch</p>
<h1><a name="check-if-cuda-is-available-1" class="anchor" href="#check-if-cuda-is-available-1"></a>Check if CUDA is available</h1>
<p>if torch.cuda.is_available():<br>
print(“CUDA is available. GPU will be used.”)<br>
device = torch.device(‘cuda’)<br>
else:<br>
print(“CUDA is not available. CPU will be used.”)<br>
device = torch.device(‘cpu’)</p>
<h1><a name="print-out-the-device-being-used-2" class="anchor" href="#print-out-the-device-being-used-2"></a>Print out the device being used</h1>
<p>print(f"Device being used: {device}")</p>
<h1><a name="create-a-tensor-and-move-it-to-the-gpu-3" class="anchor" href="#create-a-tensor-and-move-it-to-the-gpu-3"></a>Create a tensor and move it to the GPU</h1>
<p>tensor = torch.rand(3, 3).to(device)<br>
print(tensor)</p>
<p>Very frustrating as I am trying to train a custom YOLO model on my GPU at present following updating my laptop to a i7-13800H with a RTX-4060 8GB GPU.</p>

semaegrii: <p>hello, I am trying to install the pytorch version compatible with cuda 12.5 but I have not been successful. cuda 12.1 is currently active on the website. But I cannot find a version compatible with 12.5. When I install 12.1, cuda is active but I get the following error.</p>
<blockquote>
<blockquote>
<blockquote>
<p>import torch<br>
torch.cuda.is_available()<br>
/home/user/conda/lib/python3.10/site-packages/torch/cuda/<strong>init</strong>.py:118: UserWarning: CUDA initialization: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 500: named symbol not found (Triggered internally at …/c10/cuda/CUDAFunctions.cpp:108.)<br>
return torch._C._cuda_getDeviceCount() &gt; 0<br>
False</p>
</blockquote>
</blockquote>
</blockquote>
<p>Translated with <a href="http://DeepL.com" rel="noopener nofollow ugc">DeepL.com</a> (free version)</p>

semaegrii: <p>hello, I am trying to install the pytorch version compatible with cuda 12.5 but I have not been successful. cuda 12.1 is currently active on the website. But I cannot find a version compatible with 12.5. When I install 12.1, cuda is active but I get the following error.</p>
<blockquote>
<blockquote>
<blockquote>
<p>import torch<br>
torch.cuda.is_available()<br>
/home/user/conda/lib/python3.10/site-packages/torch/cuda/<strong>init</strong>.py:118: UserWarning: CUDA initialization: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 500: named symbol not found (Triggered internally at …/c10/cuda/CUDAFunctions.cpp:108.)<br>
return torch._C._cuda_getDeviceCount() &gt; 0<br>
False</p>
</blockquote>
</blockquote>
</blockquote>

ptrblck: <aside class="quote no-group" data-username="semaegrii" data-post="5" data-topic="203718">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/semaegrii/48/67300_2.png" class="avatar"> semaegrii:</div>
<blockquote>
<p>But I cannot find a version compatible with 12.5.</p>
</blockquote>
</aside>
<aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="203718" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>Your locally installed CUDA toolkit won’t be used unless you build PyTorch from source or a custom CUDA extension since the PyTorch binaries ship with its own CUDA dependencies.</p>
</blockquote>
</aside>
<p>You can install any PyTorch binary with CUDA support (11.8, 12.1, or 12.4) as long as your NVIDIA driver is properly installed and can communicate with your GPU, which does not seem to be the case.</p>
<p>We are sometimes seeing similar issues and just yesterday it seems a <a href="https://discuss.pytorch.org/t/torch-cuda-is-available-is-false-cuda-12-2-rtx-4070/185250/23">restart of the laptop</a> was needed.</p>

semaegrii: <p>I changed my nvidia driver and my problem was solved, thank you</p>

markuf: <p>Just wanted to double down and confirm the restart worked for me… Kept checking if my GPU was being used through the torch commands in my notebook and it said False. Did everything, installed the torch, cuda 12.5, etc and it didn’t work until I read your message about the restart… then it said true <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"> .</p>

gpieee: <p>i have pytorch version 2.2 and cuda 12.5. No gpu shown. tried restarting the laptop. still no difference. pls help.</p>

gpieee: <p>UPDATE :  I changed conda environment and used python venv and it worked.</p>

Whoamac: <p>Hello guys <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
<p>I’m facing a similar situation here. I purchased a RTX 4080 SUPER which was released earlier this year. Its cuda 12.5 out of the box.</p>
<p>Tried having PyTorch recognize gpu : fail<br>
Tried downgrading the driver in order to be able to use 12.4 or earlier : fail - no such drivers exists for this gpu.</p>
<p>Had to replace the gpu by a RTX 3080 instead… Wanted to use the more recent one but hitting a wall.</p>
<p>Any ideas ?</p>
<p>Update : We did install the cuda toolkits and even the toolkit is unable to provide a suitable driver for the RTX 4080 SUPER, its not recognizing the gpu</p>

ptrblck: <aside class="quote no-group" data-username="Whoamac" data-post="12" data-topic="203718">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/whoamac/48/70680_2.png" class="avatar"> Whoamac:</div>
<blockquote>
<p>Tried downgrading the driver in order to be able to use 12.4 or earlier : fail - no such drivers exists for this gpu.</p>
</blockquote>
</aside>
<p>That’s not true and you can install any recent driver, such as <code>550.90.07 </code>, by selecting the right GPU <a href="https://www.nvidia.com/download/index.aspx">here</a>:</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/d/6/d66319d5ee097a538cdaf1e7315c3bc9f96804d7.png" alt="image" data-base62-sha1="uAyv5sedhUJqsExaKpUVfxfwCSr" width="610" height="327"></p>
<aside class="quote no-group" data-username="Whoamac" data-post="12" data-topic="203718">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/whoamac/48/70680_2.png" class="avatar"> Whoamac:</div>
<blockquote>
<p>Update : We did install the cuda toolkits and even the toolkit is unable to provide a suitable driver for the RTX 4080 SUPER, its not recognizing the gpu</p>
</blockquote>
</aside>
<p>It seems you are facing more setup issues unrelated to PyTorch so make sure your system is able to recognize and use the GPU before installing PyTorch.</p>

Whoamac: <p>Perfect. Will have to go as you say. If its supposed to work, we should be able to figure it out <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=12" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20"></p>

ellby: <p>Hello, I have a NVIDIA RTX A500 Laptop GPU and installed CUDA 12.5 (nvcc --version). In my conda environment I have pytorch 2.3.1+cpu but when running torch.cuda.is_available() it still returns false. How would I go about fixing this?</p>

ptrblck: <aside class="quote no-group" data-username="ellby" data-post="15" data-topic="203718">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ellby/48/64482_2.png" class="avatar"> ellby:</div>
<blockquote>
<p>In my conda environment I have pytorch 2.3.1+cpu but when running torch.cuda.is_available() it still returns false.</p>
</blockquote>
</aside>
<p>Which is expected since you’ve installed the CPU-only binary. Install a PyTorch binary with CUDA runtime dependencies instead.</p>

RAHMA_BEN_JEMAA: <p>hello i have cuda 12.5 which pytorch version i can install ?</p>

ptrblck: <p>Take a look at my previous post: <a href="https://discuss.pytorch.org/t/does-pytorch-work-with-cuda-12-5/203718/6" class="inline-onebox">Does Pytorch work with CUDA 12.5? - #6 by ptrblck</a></p>

menma: <p>I don’t think it’s possible to successfully use the CUDA12.5 version of PyTorch at this time.<br>
Recently, I got a RTX4070 ti S GPU, and I installed 555.85 driver. I have encountered a lot of trouble since I tried to use latest version CUDA(12.5) to enable my pytorch. And, CUDA12.4 got the same problem.</p>
<p>The problem is:<br>
Pytorch docs offer command to install pytorch preview(nightly) using cuda12.4. It does not work for me. My host cuda version is 12.5.</p>
<p>The solution is to downgrade the latest cuda12.5 to cuda12.1 which support stable(2.3.1) pytorch:</p>
<ol>
<li>using conda to create a new environment;</li>
<li>following the anaconda docs(<a href="https://anaconda.org/nvidia/cuda-toolkit" rel="noopener nofollow ugc">Cuda Toolkit :: Anaconda.org</a>]) to install cuda12.1</li>
</ol>
<pre><code class="lang-auto">conda install nvidia/label/cuda-12.1.1::cuda-toolkit
</code></pre>
<ol start="3">
<li>following the pytorch docs (<a href="https://pytorch.org/#:~:text=conda%20install%20pytorch%20torchvision%20torchaudio%20pytorch%2Dcuda%3D12.1%20%2Dc%20pytorch%20%2Dc%20nvidia" class="inline-onebox" rel="noopener nofollow ugc">PyTorch</a>) to install stable(2.3.1) pytorch</li>
</ol>
<pre><code class="lang-auto">conda install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia
</code></pre>
<p>finally, I am able to use the cuda version pytorch on the relatively new GPU.<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/3/6/36f214712d548976d0f737535c47f638b9a722c9.png" alt="image" data-base62-sha1="7Q4r6bZ7ayvzFYtVwUuCdlgksGJ" width="457" height="184"></p>
<p>If someone manage to get the pytorch work with CUDA12.5, please hit me. Thanks a lot!!!</p>

ruina: <p>I was running into the same problems. I was able to solve this by:</p>
<ol>
<li>Uninstalling pytorch, pytorchvision, and pytorchaudio:</li>
</ol>
<pre><code class="lang-auto">pip uninstall torch torchvision torchaudio
</code></pre>
<ol start="2">
<li>Re-installing the following:</li>
</ol>
<pre data-code-wrap="bash"><code class="lang-bash">pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124
</code></pre>
<p>Afterwards, it was able to recognize my NVidia 4090 and I could use it.</p>

muthu_kamalan_m: <p>Hi <span class="mention">@samaegrii</span>,</p>
<p>could you elaborate how you changed drivers. is it properly communicate?</p>
<p>I run pop os 22.04 LTS. cuda==12.5 and not finding any compatable resources.followed few blogs but didn’t helped.</p>
<ul>
<li><a href="https://blog.csdn.net/qq_43652666/article/details/140501061" class="inline-onebox" rel="noopener nofollow ugc">ubuntu2204配置anaconda&amp;cuda&amp;4090nvidia驱动_ubuntu2204 nvidia driver-CSDN博客</a></li>
<li><a href="https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#post-installation-actions" class="inline-onebox" rel="noopener nofollow ugc">CUDA Installation Guide for Linux</a></li>
</ul>