Torch.cuda.is_available() is False - CUDA:12.2 - rtx 4070

TTTest: <p>Machine learning newbie here, stuck on the first step of learning PyTorch of installing CUDA. I’ve been trying to get CUDA working on my system for the past day to no avail. I’ve created multiple environments then tried installing pytorch using the below config of pip and also conda but no of them worked<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/b/6b3ebb89944b7a6207dd223f3426c96763095af5.png" data-download-href="https://discuss.pytorch.org/uploads/default/6b3ebb89944b7a6207dd223f3426c96763095af5" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/6/b/6b3ebb89944b7a6207dd223f3426c96763095af5_2_690x263.png" alt="image" data-base62-sha1="fiJuA293ufBdfuT5oyZLMhd0uyh" width="690" height="263" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/6/b/6b3ebb89944b7a6207dd223f3426c96763095af5_2_690x263.png, https://discuss.pytorch.org/uploads/default/original/3X/6/b/6b3ebb89944b7a6207dd223f3426c96763095af5.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/6/b/6b3ebb89944b7a6207dd223f3426c96763095af5.png 2x" data-dominant-color="F4DBD7"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">845×323 19.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I’ve tried downloading multiple versions of the cuda toolkit (cuda_12.2.0_536.25_windows.exe and cuda_11.8.0_windows_network.exe) and they both didn’t help. I also tried using <code>conda install cudatoolkit</code> in another environment but that didn’t work as well even though I also instead python-cuda==11.8.</p>
<p>Tbh I don’t really know what versions of one software need what versions of the other software. I am very confused. Like what does the CUDA version mean in nvidia-smi? Is this a driver version? hardware? does it support older version of pytorch-cuda ? or older versions of cuda toolkit? Is CUDA toolkit version supposed to be the same version was my CUDA version in nvidia-smi and pytorch-cuda?</p>
<p>nvidia-msi:<br>
±--------------------------------------------------------------------------------------+<br>
| NVIDIA-SMI 536.25                 Driver Version: 536.25       CUDA Version: 12.2     |<br>
|-----------------------------------------±---------------------±---------------------+<br>
| GPU  Name                     TCC/WDDM  | Bus-Id        Disp.A | Volatile Uncorr. ECC |<br>
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |<br>
|                                         |                      |               MIG M. |<br>
|=========================================+======================+======================|<br>
|   0  NVIDIA GeForce RTX 4070      WDDM  | 00000000:01:00.0  On |                  N/A |<br>
|  0%   31C    P8               7W / 200W |   1015MiB / 12282MiB |      1%      Default |<br>
|                                         |                      |                  N/A |<br>
±----------------------------------------±---------------------±---------------------+</p>

ptrblck: <p>You would only need to properly install the NVIDIA driver, not the CUDA toolkit, since PyTorch ships with all CUDA dependencies.</p>

TTTest: <p>Really? That’s all I need to install? All the youtube videos I watch said to install the cuda toolkit from <a href="https://developer.nvidia.com/cuda-downloads" rel="noopener nofollow ugc">https://developer.nvidia.com/cuda-downloads</a></p>
<p>How do I check if I’ve properly installed the NVIDIA driver? It looks fine to me yet<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/a/1/a1cb5d765be1a25054a39a8a5ab8de81c085d27b.png" alt="image" data-base62-sha1="n5izZbgEB5x8QWVxHWYyj3kXAFR" width="679" height="484"></p>
<p>And so I can install pytorch with any compute platform version like CUDA 11.7 or CUDA 11.8 or 12.1 and get Torch.cuda.is_available() == True?</p>

TTTest: <p>I managed to get it working.This is what I did. idk how and why it works but it does.</p>
<p>I uninstalled python but kept anaconda (along with anaconda’s python, pip, etc files)<br>
I uninstalled all the CUDA toolkits and their associated packages like NSight then installed the 11.8 toolkit.<br>
I installed the pytorch with the following command:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/3/53bdc3478380fac6a6939e9a88d801939d3988c9.png" data-download-href="https://discuss.pytorch.org/uploads/default/53bdc3478380fac6a6939e9a88d801939d3988c9" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/5/3/53bdc3478380fac6a6939e9a88d801939d3988c9.png" alt="image" data-base62-sha1="bWOc5ybyuSnms6jdHOLlNfHcHEZ" width="690" height="273" data-dominant-color="F4DCD8"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">842×334 19.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
I did NOT setup a conda environment using <code>conda create -n env</code> this time, instead I’m doing everything in the base conda environment. In this base conda environment I ran the pip install from above.</p>

ptrblck: <aside class="quote no-group" data-username="TTTest" data-post="3" data-topic="185250">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/t/ee7513/48.png" class="avatar"> TTTest:</div>
<blockquote>
<p>Really? That’s all I need to install? All the youtube videos I watch said to install the cuda toolkit from</p>
</blockquote>
</aside>
<p>Yes, you would only need an NVIDIA driver and your locally installed CUDA toolkit will be used if you are building from source or custom CUDA extensions. I cannot comment on the Youtube videos you were watching.</p>
<p>Good to hear it’s working now!</p>

TTTest: <p>To double check the installation process above is correct I uninstalled Python and Anaconda while making sure to remove the folders left behind after their installation. I then re-installed anaconda only, not python. I then pip installed pytorch and its related packages in my base environment using this:<br>
<code>pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu121</code><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/3/53bdc3478380fac6a6939e9a88d801939d3988c9.png" data-download-href="https://discuss.pytorch.org/uploads/default/53bdc3478380fac6a6939e9a88d801939d3988c9" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/5/3/53bdc3478380fac6a6939e9a88d801939d3988c9.png" alt="image" data-base62-sha1="bWOc5ybyuSnms6jdHOLlNfHcHEZ" width="690" height="273" data-dominant-color="F4DCD8"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">842×334 19.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Note I still have the 11.8 CUDA toolkit from before</p>

ptrblck: <aside class="quote no-group" data-username="TTTest" data-post="6" data-topic="185250">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/t/ee7513/48.png" class="avatar"> TTTest:</div>
<blockquote>
<p>Note I still have the 11.8 CUDA toolkit from before</p>
</blockquote>
</aside>
<p>Again, your locally installed CUDA toolkit won’t be used.<br>
And yes, <code>conda</code> can be used to manage your virtual environments and installing PyTorch using the provided install commands for the pip wheels or conda binaries in an empty environment will work.</p>

TTTest: <p>Yup. I ended up uninstalling the CUDA toolkit and <code>torch.cuda.is_available()</code> was still <code>True</code></p>

Jalal_Nouri: <p>Hello. I have the same problem.<br>
Nvidia GTX 1650<br>
driver version: 536.99<br>
CUDA Toolkit 12.2</p>
<p>I install Python with this command: “pip3 install --pre torch torchvision torchaudio --index-url <a href="https://download.pytorch.org/whl/nightly/cu121" rel="noopener nofollow ugc">https://download.pytorch.org/whl/nightly/cu121</a>”, that I receive this message “Requirement already satisfied” in anaconda prompt</p>
<p>and when I run “torch.cuda.is_available()” in jupyter notebook, the output is “False” <img src="https://discuss.pytorch.org/images/emoji/apple/frowning.png?v=12" title=":frowning:" class="emoji" alt=":frowning:" loading="lazy" width="20" height="20"><br>
pleeeeease, can you help me? I’ve searched a lot on the internet and I’ve tried many ways other than uninstalling the 12.2 version and installing the 11.8 version.</p>

ptrblck: <p>Most likely a CPU-only binary is found so uninstall all previous PyTorch installation and install the desired one afterwards.</p>

jsbach4770: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> My device also is RTX 4070. Is there any way to install pytorch with cuda 12.2? Or other command can do it? THX</p>

Jalal_Nouri: <p>thanks a lot.<br>
Your suggestion worked exactly for me and my problem was solved.</p>

ptrblck: <p>If you really need CUDA 12.2 for whatever reason, you would need to build from source. If 12.1 would work you can install the nightly binaries as already explained.</p>

Quantam: <p>Hi I have 12.2 with a RTX 3050 mobile, if I select Stable(2.10) with 12.1 will it work.</p>
<p>Followup question I also need to install the cudNN (8.9.4 for cuda 12.x) before doing all this.</p>

ptrblck: <aside class="quote no-group" data-username="Quantam" data-post="14" data-topic="185250">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/quantam/48/64932_2.png" class="avatar"> Quantam:</div>
<blockquote>
<p>if I select Stable(2.10) with 12.1 will it work.</p>
</blockquote>
</aside>
<p>Yes, it’ll work.</p>
<aside class="quote no-group" data-username="Quantam" data-post="14" data-topic="185250">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/quantam/48/64932_2.png" class="avatar"> Quantam:</div>
<blockquote>
<p>Followup question I also need to install the cudNN (8.9.4 for cuda 12.x) before doing all this.</p>
</blockquote>
</aside>
<p>No, since the PyTorch binaries ship with all CUDA dependencies, including cuDNN, cuBLAS, NCCL, etc.</p>

Quantam: <p>Thank you so much. Will try and update with the results</p>

Kenil_Shah: <p>What do you mean? Can you please explain in brief. I have RTX 4060 with cuda 12.4 . When, I execute</p>
<p>torch.cuda.is_available()</p>
<p>I am getting false. What should I do?</p>

ptrblck: <p>You might have installed the CPU-only binary or your NVIDIA driver is not properly installed. Check if your PyTorch binary ships with CUDA dependencies via <code>print(torch.version.cuda)</code> and make sure a valid version is returned.</p>

Harry3105: <p>hello, i have RTX 4070 as well. i have tried several things to activate CUDA but nothing works. The NVIDIA driver is up to date. i have tried to install pytorch with command pip install torch torchvision torchaudio --index-url <a href="https://download.pytorch.org/whl/cu121" rel="noopener nofollow ugc">https://download.pytorch.org/whl/cu121</a></p>
<p>how do i know which CUDA version i need/can use? according to your explanations I dont need to install CUDA toolkit.</p>
<p>which command i need to use to install pytorch with the CUDA dependencies?</p>
<p>Thanks a lot for your support!</p>

ptrblck: <aside class="quote no-group" data-username="Harry3105" data-post="19" data-topic="185250">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/harry3105/48/64119_2.png" class="avatar"> Harry3105:</div>
<blockquote>
<p>how do i know which CUDA version i need/can use? according to your explanations I dont need to install CUDA toolkit.</p>
</blockquote>
</aside>
<p>That’s correct. Your locally installed CUDA toolkit won’t be used unless you build PyTorch from source or a custom CUDA extension.</p>
<aside class="quote no-group" data-username="Harry3105" data-post="19" data-topic="185250">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/harry3105/48/64119_2.png" class="avatar"> Harry3105:</div>
<blockquote>
<p>which command i need to use to install pytorch with the CUDA dependencies?</p>
</blockquote>
</aside>
<p><code>pip install torch</code> will install the latest stable PyTorch release with CUDA 12.1 dependencies. For other versions you should check the commands provided in the install matrix.</p>