Cuda versioning and pytorch compatibility

juanpajedrez3112: <p>hi everyone,</p>
<p>I am pretty new at using pytorch. Currently, I have been trying to understand the concepts of using CUDA for performing better loading data and increasing speed for training models.</p>
<p>I took a look into my system, I currently have an NVIDIA GTX1650 that contains CUDA v-11, yet I see that hasn’t been installed. Normally, when I work in python, I use virtual environments to set all the libraries I use for a project. With pytorch, I saw you can run on the CPU or use CUDA.</p>
<p>Currently, the latest version is pytorch 2.1.0 which goes until CUDA 11.8 or 12.1. I may have a couple of questions regarding how to properly set my graphics card for usage.</p>
<p>1.) Since the drivers say the latest version is CUDA 11. Does that mean I have to download the 11.0 CUDA from NVIDIA? Since other versions would not work?</p>
<p>2.) Pytorch versioning must also meet the highest available CUDA version? In other words, downgrade pytorch 2.1.0?</p>
<p>3.) Is there a better way than installing for local venvs? (Conda for example).</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/d/b/db75e4795959868d803e25f59fc50c64bf11af71.png" data-download-href="https://discuss.pytorch.org/uploads/default/db75e4795959868d803e25f59fc50c64bf11af71" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/d/b/db75e4795959868d803e25f59fc50c64bf11af71.png" alt="image" data-base62-sha1="vjr9gEL1qNS55AshvgaTy93O4AF" width="690" height="301" data-dominant-color="1B1B1B"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">733×320 7.99 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Thank you so much for your time.</p>

ptrblck: <ol>
<li>
<p>No, you don’t need to download a full CUDA toolkit and would only need to install a compatible NVIDIA driver, since PyTorch binaries ship with their own CUDA dependencies. Your current driver should allow you to run the PyTorch binary with CUDA 11.8, but would fail to run the binary with CUDA 12.1 Update 1 as it’s too old.</p>
</li>
<li>
<p>I don’t understand this question, since PyTorch 2.1.0 is the latest release so what do you want to downgrade from?</p>
</li>
<li>
<p>You don’t need to install virtual environments if you want to use a single PyTorch version only.</p>
</li>
</ol>

juanpajedrez3112: <p>Hi!</p>
<p>Thank you for the quick response!</p>
<p>For the second question. I meant to go from PyTorch 2.1.0 to (for instance). To 1.8.0.</p>
<p>Thank you !</p>

ptrblck: <p>Thanks for clarifying. No you don’t need to downgrade PyTorch and can use the latest <code>2.1.0+cu118</code> release with your driver.</p>

Nawed_Ali: <p>hey! can you share whether torch 2.2.0 (latest) would support the latest 12.3 update for cuda</p>
<p>Facing this issue -<br>
Installed CUDA version 12.3 does not match the version torch was compiled with 11.8, unable to compile cuda/cpp extensions without a matching cuda version</p>

ptrblck: <p>If you want to build a custom CUDA extension, your locally installed CUDA toolkit has to match the one used in the binaries in the major version and ideally should also match the minor version. Different major releases will error out.</p>

Mithilesh_Kumar: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a>,</p>
<p>I have ‘NVIDIA T1200 Laptop GPU’ in my laptop. Which version of Cuda and Torch will go hand in hand. I tried installing ‘cuda_12.3.2_546.12_windows’ &amp; ‘cuda_11.0.2_451.48_win10’. But failing at both. I have window 11 installed in my system. Here are my system specifications:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/7/17eae07e083cd0a6a338e35b56fa4ecc2aa6c8ab.png" data-download-href="https://discuss.pytorch.org/uploads/default/17eae07e083cd0a6a338e35b56fa4ecc2aa6c8ab" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/1/7/17eae07e083cd0a6a338e35b56fa4ecc2aa6c8ab.png" alt="image" data-base62-sha1="3pAcBvO02nvZnA3hP5bRL0SxszF" width="690" height="414" data-dominant-color="F4F4F4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1357×815 20.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Your help would be greatly appriciated.</p>
<p>Regards,<br>
Mithilesh</p>

ptrblck: <p>I don’t know what exactly you want to build, but note that your locally installed CUDA toolkit won’t be used unless you want to build PyTorch from source or a custom CUDA extension. Could you thus describe what exactly you want to build?</p>

hamedB: <p>Dear <a class="mention" href="/u/ptrblck">@ptrblck</a></p>
<p>I try to install pytorch on my local machine via conda command.<br>
I use the conda command from PyTorch website:</p>
<blockquote>
<p>conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia</p>
</blockquote>
<p>using above command the conda command remain in a loop. When I remove pytroch-cuda=11.8, the command successfully run and all other lib. are installed.</p>
<p>Next I enter the below command to install pytorch-cuda:</p>
<blockquote>
<p>conda install pytorch-cuda=11.8 -c pytorch -c nvidia</p>
</blockquote>
<p>Unfortunately, it is not installed and I receive the following message.</p>
<blockquote>
<p>Collecting package metadata (current_repodata.json): - WARNING conda.models.version:get_matcher(542): Using .* with relational operator is superfluous and deprecated and will be removed in a future version of conda. Your spec was 1.7.1.<em>, but conda is ignoring the .</em> and treating it as 1.7.1<br>
done<br>
Solving environment: failed with initial frozen solve. Retrying with flexible solve.<br>
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.<br>
Solving environment: /</p>
</blockquote>
<p>I was wondering whether you could know what would be issue.</p>
<p>thanks</p>

ptrblck: <p>I would create a new and clean conda environment as it seem the conflict resolution might either fail or takes too long.</p>

hamedB: <p><a class="mention" href="/u/ptrblck">@ptrblck</a><br>
Thanks for your quick reply. The thing is that I have created several new conda env., even I tried removing the whole Anaconda folder and reinstall again Anaconda. However, it did not work. When I enter the command to install pytorch, torchvision, and torchaudio, everything is fine. Once I add pytorch-cuda to this command, or I want to install pytorch-cuda after others, such as pytorch and torchvision, the install command remains in loop and finally stopped with the error</p>
<blockquote>
<p>Solving environment: Killed</p>
</blockquote>
<p>One more thing, I have a yml file including pytorch 1.13 and cuda 11.6. It is installed with conda successfully. However, the problem is with new version.</p>
<p>The reason I want to install new pytorch version is because in pytorch 1.13 it seems the output of hook are not stable. And the hook outputs are slightly changed in different running.</p>
<p>Thanks again for your inputs.</p>

ptrblck: <p>I’ve just installed the latest stable conda package in a new environment and it works fine in <a href="https://discuss.pytorch.org/t/glibcxx-3-4-30-not-found/196875/5">this thread</a>. If <code>conda</code> has issues with the conflict resolution or needs too much RAM you could consider using <code>mamba</code> which should be faster.</p>

varsha1: <p>Hi,</p>
<p>where I can find the details like for a specific cuda version which version of pytorch is compatible<br>
I have cuda11.4 what should be the pytorch version</p>

ptrblck: <p>Check my <a href="https://discuss.pytorch.org/t/cuda-versioning-and-pytorch-compatibility/189777/8">previous post</a> in this thread.</p>

Ameneh_Shamekhi: <p>Hi,<br>
I did install cuda and pytorch using this command:<br>
<code>"conda install pytorch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 pytorch-cuda=11.7 -c pytorch -c nvidia"</code><br>
and now I have the following packages on my conda env:<br>
cuda-cudart               11.7.99                       0    nvidia<br>
cuda-cupti                11.7.101                      0    nvidia<br>
cuda-libraries            11.7.1                        0    nvidia<br>
cuda-nvrtc                11.7.99                       0    nvidia<br>
cuda-nvtx                 11.7.91                       0    nvidia<br>
cuda-runtime              11.7.1                        0    nvidia<br>
nvidia-cuda-cupti-cu11    11.7.101                 pypi_0    pypi<br>
nvidia-cuda-nvrtc-cu11    11.7.99                  pypi_0    pypi<br>
nvidia-cuda-runtime-cu11  11.7.99                  pypi_0    pypi<br>
pytorch                   2.0.1           py3.8_cuda11.7_cudnn8.5.0_0    pytorch<br>
pytorch-cuda              11.7                 h778d358_5    pytorch<br>
pytorch-mutex             1.0                        cuda    pytorch<br>
$ conda list | grep ‘torch’<br>
ffmpeg                    4.3                  hf484d3e_0    pytorch<br>
pytorch                   2.0.1           py3.8_cuda11.7_cudnn8.5.0_0    pytorch<br>
pytorch-cuda              11.7                 h778d358_5    pytorch<br>
pytorch-mutex             1.0                        cuda    pytorch<br>
pytorch-triton            2.1.0+440fd1bf20          pypi_0    pypi<br>
torch-tb-profiler         0.4.1                    pypi_0    pypi<br>
torchaudio                2.1.0.dev20230628+cu118          pypi_0    pypi<br>
torchtriton               2.0.0                      py38    pytorch<br>
torchvision               0.16.0.dev20230628+cu118          pypi_0    pypi<br>
but I still get the following error:</p>
<pre><code class="lang-auto">RuntimeError: CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
</code></pre>
<p>Please let me know if you have any suggestions.</p>

ptrblck: <p>Which GPU are you using?</p>

Ameneh_Shamekhi: <p>It’s a t4 machine. Standard NC4as T4 v3</p>

ptrblck: <p>The release works for me on a T4:</p>
<pre data-code-wrap="python"><code class="lang-python">python -c "import torch; print(torch.__version__); print(torch.version.cuda); print(torch.cuda.get_device_properties(0)); print(torch.randn(1).cuda())"
2.0.1
11.7
_CudaDeviceProperties(name='Tesla T4', major=7, minor=5, total_memory=15978MB, multi_processor_count=40)
tensor([-0.0716], device='cuda:0')
</code></pre>

Ameneh_Shamekhi: <p>I get similar output when I run "python -c “import torch; print(torch.<strong>version</strong>); print(torch.version.cuda); print(torch.cuda.get_device_properties(0)); print(torch.randn(1).cuda())”<br>
"  so what does this error mean:</p>
<aside class="quote no-group" data-username="Ameneh_Shamekhi" data-post="16" data-topic="189777">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ameneh_shamekhi/48/70028_2.png" class="avatar"> Ameneh_Shamekhi:</div>
<blockquote>
<pre><code class="lang-auto">RuntimeError: CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
</code></pre>
</blockquote>
</aside>
<p>Should I uninstall the normal pytorch (the non cuda version)?</p>

ptrblck: <p>The error might be raised by another library not building for your GPU architecture as PyTorch itself is able to use your device. You should try to narrow down which line of code fails and which library is used.</p>