Is CUDA 12.0 supported with any pytorch version?

jsmidt: <p>If you look at <a href="https://pytorch.org/get-started/previous-versions/" rel="noopener nofollow ugc">this page</a>, there are commands how to install a variety of pytorch versions given the CUDA version. However, the only CUDA 12 version seems to be 12.1.  My cluster machine, for which I do not have admin right to install something different, has CUDA 12.0.</p>
<p>I tried to modify one of the lines like:</p>
<pre><code class="lang-auto">conda install pytorch==2.2.0 torchvision==0.17.0 torchaudio==2.2.0 pytorch-cuda=12.0 -c pytorch -c nvidia
</code></pre>
<p>But this did not work.  Is there a way to install pytorch with CUDA 12.0?  If so, is there documentation on how to go about that?  Thanks!</p>

ptrblck: <p>Yes, the current PyTorch code base supports all CUDA 12 toolkit versions if you build from source. The install matrix on the website shows the prebuilt binaries which ship with their own CUDA runtime dependencies. If you install these your locally installed CUDA toolkit won’t be used. You would only need to properly install an NVIDIA driver.</p>

jsmidt: <p>Thank you <a class="mention" href="/u/ptrblck">@ptrblck</a>.  So are you saying I need to build from source against CUDA 12.0?  Or are you saying I can type something like:</p>
<pre><code class="lang-auto">pip3 install torch torchvision torchaudio
</code></pre>
<p>And the torch binary installed with ignore CUDA 12.0 on the system and use it’s own internal CUDA 12.1 libraries?  Or are you saying both. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>

jsmidt: <p>For posterity, using</p>
<pre><code class="lang-auto">pip3 install torch torchvision torchaudio
</code></pre>
<p>did work assuming I had the CUDA 12.0 driver installed as verified by the nvidia-smi command.  The only issue is I had to update my environment variables for this one library for whatever reason:</p>
<pre><code class="lang-auto">export LD_LIBRARY_PATH=$HOME/.local/lib/python3.10/site-packages/nvidia/nvjitlink/lib:$LD_LIBRARY_PATH
</code></pre>

ptrblck: <aside class="quote no-group" data-username="jsmidt" data-post="3" data-topic="197636">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jsmidt/48/67821_2.png" class="avatar"> jsmidt:</div>
<blockquote>
<p>And the torch binary installed with ignore CUDA 12.0 on the system and use it’s own internal CUDA 12.1 libraries?</p>
</blockquote>
</aside>
<p>Yes, this is the case. If you have a local CUDA toolkit installed and are seeing library conflicts, removing the locally installed CUDA toolkit from the <code>LD_LIBRARY_PATH</code> can be used as a workaround.</p>
<p>Good to hear it’s working now! <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>

Asritha_Penmetsa: <p>Hello Sir,<br>
My Cuda Version is 12.5 and Driver version is 555.85 . There is no available compatible pytorch version for this. There is a trouble when running this nvidia/tts_en_fastpitch. What needs to be done to sort out this issue?<br>
Thanks.</p>

ptrblck: <p>My previous post explains the support: <a href="https://discuss.pytorch.org/t/is-cuda-12-0-supported-with-any-pytorch-version/197636/2" class="inline-onebox">Is CUDA 12.0 supported with any pytorch version? - #2 by ptrblck</a></p>

anilkeshwani: <p>Is it right to interpret the fact that “the torch binary installed [will] ignore CUDA 12.0 on the system and use it’s own internal CUDA 12.1 libraries” as meaning that CUDA is statically linked to the PyTorch install, or am I thinking about this all wrong?</p>

ptrblck: <p>No, only the CUDA runtime is statically linked to PyTorch as is the common approach. Other CUDA dependencies will be installed as dependencies as pip packages and will be dynamically opened during the runtime.</p>

wren_101: <p>This finally worked for me! After grappling with grok for hours… Thanks!</p>