Install pytorch with Cuda 12.1

mohamed_alqablawi: <p>hello,<br>
I have a GPU Nvidia GTX  1650 with Cuda 12.1.<br>
I want to install the pytorch with Cuda, but the latest version is Cuda 11.8 on the website.<br>
Is it possible to install version 11.8 and I have 12.1?</p>

ptrblck: <p>Yes, the PyTorch binaries ship with their own CUDA runtime, cuDNN, NCCL etc. and will work with newer drivers. Your local CUDA toolkit will be used if you build PyTorch from source or a custom CUDA extension.</p>

Shisui: <p>Torch built with cuda1.18 works under cuda12.1</p>
<p>I compiled a new version with cuda12.1.<br>
There seems to be no performance advantage</p>

r_AI: <p>I banged my head for a couple of days trying to get PyTorch (2.0.1+cu118) working with cuda12.1 in Unbuntu 20.04.</p>
<ul>
<li>Installing PyTorch via conda did not work. torch.cuda.is_available() returned False</li>
<li>Compiling PyTorch did not work (for me). torch.cuda.is_available() returned False</li>
<li>Installing PyTorch via PIP worked. This returns:
<ul>
<li>torch.cuda.is_available() is True</li>
<li>torch.backends.cudnn.version() is 8700</li>
<li>torch.backends.cudnn.enabled is True</li>
</ul>
</li>
</ul>

masterov: <aside class="quote no-group" data-username="r_AI" data-post="4" data-topic="174294">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/r_ai/48/60241_2.png" class="avatar"> r_AI:</div>
<blockquote>
<p>Installing PyTorch via PIP worked</p>
</blockquote>
</aside>
<p>Could you please be more specific? What is the exact command that you have used?</p>

1040242795: <p>No performance advantage compared to what? Also, where can you learn about your specific compilation process?</p>

r_AI: <p>This is my shell setup script:</p>
<pre><code class="lang-auto">conda_forge_packages="pypdf2 screeninfo opencv matplotlib scipy pandas openpyxl statsmodels"
conda update -n base -c defaults conda
conda create -n my_env -c conda-forge python=3.10 ${conda_forge_packages}
source ~/miniconda3/etc/profile.d/conda.sh
conda activate my_env
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
</code></pre>

Shisui: <ol>
<li>Latest pytorch built with cuda11.8</li>
<li>Official pytorch document</li>
</ol>

cristianduguet: <p>Then there must be something I’m doing wrong because for me, it is not compatible.  I’m getting the error:</p>
<pre><code class="lang-auto">RuntimeError:
      The detected CUDA version (12.1) mismatches the version that was used to compile
      PyTorch (11.8). Please make sure to use the same CUDA versions.
</code></pre>
<p>I have the nvidia drivers <code>530.30.02</code> with  CUDA <code>12.1</code>.  I’m currently following the installation options of <a href="https://docs.nerf.studio/en/latest/quickstart/installation.html#pytorch" rel="noopener nofollow ugc">NerfStudio</a>.</p>
<p>Their install instructions are:<br>
<code>pip install torch==2.0.1+cu118 torchvision==0.15.2+cu118 --extra-index-url https://download.pytorch.org/whl/cu118</code></p>
<p>And the error happens when I try to install<br>
<code>pip install ninja git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch</code></p>

ptrblck: <p>The error is raised if you are trying to build a third-party module from source, which will use your locally installed CUDA toolkit. Either downgrade your local CUDA toolkit to 11.8 or update your PyTorch binaries to 12.1 to use matching versions.</p>

Santhiya: <p>This worked for me on Windows 10 with RTX A2000, with<br>
CUDA 12.1,<br>
cuDNN 8.8.1,<br>
python 3.10 / anaconda<br>
torch 2.0.1+cu118</p>

Alfasi: <p>I get FALSE for: torch.cuda.is_available()</p>
<p>CUDA 12.1<br>
torch: 2.0.1<br>
torchvision: 0.15.2<br>
python 3.10</p>
<p>Any suggestions?</p>

ptrblck: <p>Make sure the right PyTorch binary was installed by checking <code>torch.version.cuda</code> as well as the output of <code>pip list | grep torch</code> which should indicate the CUDA dependencies.</p>

Alfasi: <p>Thank you, i tried it.</p>
<p>torch.version.cuda:<br>
I get “None”</p>
<p>pip list | findstr torch (I’m on Windows):<br>
torch: 2.0.1<br>
torchvision: 0.15.2</p>
<p>Any thoughts?</p>

ptrblck: <p>This would mean that you’ve installed the CPU-only binary and would need to select a proper CUDA dependency from the <a href="https://pytorch.org/get-started/locally/">install matrix</a>.</p>

Alfasi: <p>Thank you! I always used the CUDA 11.8 from the install matrix, but looks like it’s working now. If it helps others, here what I did:</p>
<p>I uninstalled torch through Termianl using “pip uninstall torch”</p>
<p>Then reinstalled using:<br>
pip install torch torchvision --pre -f <a href="https://download.pytorch.org/whl/nightly/cu121/torch_nightly.html" rel="noopener nofollow ugc">https://download.pytorch.org/whl/nightly/cu121/torch_nightly.html</a></p>
<p>This time it somehow installed correctly and now I get:</p>
<p>torch.cuda.is_available():<br>
True</p>
<p>torch.version.cuda:<br>
‘12.1’</p>
<p>pip list | findstr torch:<br>
torch                     2.1.0.dev20230604+cu121<br>
torchvision               0.16.0.dev20230605+cu121</p>

VictorJuliani: <p>What worked for me was:</p>
<pre><code class="lang-auto">pip uninstall torch
pip cache purge
pip install torch torchvision --pre -f https://download.pytorch.org/whl/nightly/cu121/torch_nightly.html
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117
</code></pre>
<p>Cuda version with returned 12.1 with this command</p>
<pre><code class="lang-auto">nvidia-smi.exe
</code></pre>
<pre><code class="lang-auto">pip list | grep torch
</code></pre>
<p>returns this list</p>
<p><em>pytorch-lightning             1.6.4</em><br>
<em>torch                         2.0.1+cu117</em><br>
<em>torchaudio                    2.0.2</em><br>
<em>torchmetrics                  0.9.2</em><br>
<em>torchvision                   0.15.2+cu117</em></p>
<p>and</p>
<pre><code class="lang-auto">    import torch
    torch.cuda.is_available()
</code></pre>
<p>returns ‘True’</p>
<p>not sure if this is relevant, but i was running whisper and whisper was forcing cpu, now it works fine</p>

MohamedAliRashad: <p>Any updates on this issue ?</p>

ptrblck: <p>On what issue exactly? The CUDA 12.1 nightly binaries are available and install issues are caused by a broken environment in 99+% of cases. If you have trouble installing binaries, please give more information what fails and what you’ve tried.</p>

Dilip_Pal: <p>HI, it did not worked for me. I am on Ubunt 20.04 and NVIDIA RTX3080. I am novice on linux. Can anyone help with step wise document on how to install pytorch with CUda activated?</p>
<p>Thanks,</p>