Would pytorch for cuda 11.6 work when cuda is actually 12.0

new_to_dl: <p>I will be using pytorch for a deep learning application. My computer has an NVIDIA GPU already.</p>
<p>I installed CUDA following <a href="https://docs.nvidia.com/cuda/cuda-installation-guide-microsoft-windows/index.html" class="inline-onebox" rel="noopener nofollow ugc">CUDA Installation Guide for Microsoft Windows</a></p>
<p>Following that I installed pytorch in a conda environment with the pip command suggested on <a href="http://pytorch.org" rel="noopener nofollow ugc">pytorch.org</a></p>
<p><code>pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu116</code></p>
<p>However, when I check the CUDA version I installed with <code>nvcc - version</code>, I get</p>
<p><code>... Cuda compilation tools, release 12.0, V12.0.76 Build cuda_12.0.r12.0/compiler.31968024_0</code></p>
<p>My installation seems to be okay when I run the code <code>torch.cuda.is_available()</code></p>
<p>as it returns True and identifies the name of the GPU, correctly.</p>
<p>Just wanted to check whether it is fool-proof when I actually feed in a deep learning model with a big dataset, since CUDA version is 12.0 but I installed pytorch suitable for 11.6.</p>
<p>Many thanks already.</p>

ptrblck: <p>Yes, your setup will work since the PyTorch binaries ship with their own CUDA runtime (as well as other CUDA libs such as cuBLAS, cuDNN, NCCL, etc.). The locally installed CUDA toolkit (12.0 in your case) will only be used if you are building PyTorch from source or a custom CUDA extension.<br>
The NVIDIA drivers are backwards-compatible so the newer driver (I would guess 525.60) will also work.</p>

new_to_dl: <p>got it, thank you so much!</p>

sytelus: <p>Oh wait… are you saying we don’t really need to do those painful installs of NVidia CUDA and cuDNN on Windows or Linux for PyTorch? PyTorch works without those installs?</p>

ptrblck: <p>Yes, this was always the case since we shipped binaries.</p>

Vedant_Gannu: <p>So if you don’t actually need to install Cuda toolkit since Pytorch already has install cuda for its installation, then when is Pytorch support for cuda 12 going to be available for download? The Pytorch website still only has cuda 11.7 and cuda 11.8.</p>

ptrblck: <p>The nightlies are already built, but not public yet since they need more testing.<br>
If you don’t want to wait for the binaries you can always build from source using your locally installed CUDA toolkit.</p>

gmiliotis: <p>Sorry but I am still confused with this. I have cuda installed on my pc, this is the output of <code>nvidia-smi</code><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/7/27691ed64bf41400d5120888fe1c561414b67858.png" data-download-href="https://discuss.pytorch.org/uploads/default/27691ed64bf41400d5120888fe1c561414b67858" title="nvidia_smi"><img src="https://discuss.pytorch.org/uploads/default/original/3X/2/7/27691ed64bf41400d5120888fe1c561414b67858.png" alt="nvidia_smi" data-base62-sha1="5CDQ8nlzrdvqwaJ8RIH8jzdanBC" width="690" height="347" data-dominant-color="3C1831"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">nvidia_smi</span><span class="informations">808×407 48.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I tried to install pytorch by running:<br>
<code>conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia</code></p>
<p>but apparently there is no gpu available to pytorch:</p>
<pre><code class="lang-auto">Python 3.8.17 | packaged by conda-forge | (default, Jun 16 2023, 07:06:00) 
[GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
&gt;&gt;&gt; 
&gt;&gt;&gt; 
&gt;&gt;&gt; import torch
&gt;&gt;&gt; torch.cuda.is_available()
False
&gt;&gt;&gt; torch.cuda.device_count()
0
&gt;&gt;&gt; 
&gt;&gt;&gt; 
&gt;&gt;&gt; torch.__version__
'2.0.0'
&gt;&gt;&gt;
</code></pre>
<p>I know I have cuda 12.2 and I try to install pytorch with cuda 11.8 but if I understand well, that is not relevant since pytorch comes with its own binaries for cuda etc…What am I doing wrong please?</p>

ptrblck: <p>What does <code>torch.version.cuda</code> return? If it’s <code>None</code> you’ve installed the CPU-only binary.</p>

gmiliotis: <p>Yes,<code> torch.version.cuda</code> doesnt return anything, so yes it is <code>None</code>. This is the command I need to run, right? <code>conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia</code></p>
<p>Ok, I will try again, I am almost certain I ran the command above, but who knows, maybe I did some mistake…I will remove the environment and start from a fresh one. Anyways, thanks for your help!</p>

gmiliotis: <p>Ok, for some strange reason pytorch with cuda activated doesnt work for me when I try the <code>conda</code> command. This is the package plan,<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/0/80596beaf4c65524baef7180ac6923cd0d3cbe73.png" data-download-href="https://discuss.pytorch.org/uploads/default/80596beaf4c65524baef7180ac6923cd0d3cbe73" title="package_plan"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/8/0/80596beaf4c65524baef7180ac6923cd0d3cbe73_2_317x500.png" alt="package_plan" data-base62-sha1="ijqHsH3qP1LAfQkHsLn8udxsGfp" width="317" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/8/0/80596beaf4c65524baef7180ac6923cd0d3cbe73_2_317x500.png, https://discuss.pytorch.org/uploads/default/optimized/3X/8/0/80596beaf4c65524baef7180ac6923cd0d3cbe73_2_475x750.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/8/0/80596beaf4c65524baef7180ac6923cd0d3cbe73_2_634x1000.png 2x" data-dominant-color="441B37"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">package_plan</span><span class="informations">1037×1631 282 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>but still  this what I get</p>
<pre><code class="lang-auto">Python 3.8.0 | packaged by conda-forge | (default, Nov 22 2019, 19:11:38) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
&gt;&gt;&gt; 
&gt;&gt;&gt; 
&gt;&gt;&gt; import torch
&gt;&gt;&gt; 
&gt;&gt;&gt; 
&gt;&gt;&gt; torch.version.cuda
&gt;&gt;&gt; torch.cuda.is_available()
False
</code></pre>
<p>However when I try the pip command, then everything looks fine! Below is my pytorch that got installed with pip</p>
<pre><code class="lang-auto">Python 3.8.0 | packaged by conda-forge | (default, Nov 22 2019, 19:11:38) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
&gt;&gt;&gt; 
&gt;&gt;&gt; 
&gt;&gt;&gt; 
&gt;&gt;&gt; import torch
&gt;&gt;&gt; 
&gt;&gt;&gt; 
&gt;&gt;&gt; torch.cuda.is_available()
True
&gt;&gt;&gt; 
&gt;&gt;&gt; torch.version.cuda
'11.8'
&gt;&gt;&gt;
</code></pre>
<p>I dont know why that happens, I have to say however that I am using the <a href="https://mamba.readthedocs.io/en/latest/mamba-installation.html" rel="noopener nofollow ugc">Mambaforge distribution</a> and something might be happening with the package manager which presumably is compatible with conda but maybe it is not in this particular case.</p>
<p>I think I will remove Mambaforge entirely, install anaconda and try again,</p>

gmiliotis: <p>Im back here again. Removed mambaforge, installed minicoda and then the gpu version of pyrorch. everything is fine,</p>
<pre><code class="lang-auto">Python 3.8.17 (default, Jul  5 2023, 21:04:15)                                                                                       
[GCC 11.2.0] :: Anaconda, Inc. on linux                                                                                              
Type "help", "copyright", "credits" or "license" for more information.                                                               
&gt;&gt;&gt;                                                                                                                                  
&gt;&gt;&gt; import torch                                                                                                                     
&gt;&gt;&gt;                                                                                                                                  
&gt;&gt;&gt; torch.version.cuda                                                                                                               
'11.8'                                                                                                                               
&gt;&gt;&gt; torch.cuda.is_available()                                                                                                        
True                                                                                                                                 
&gt;&gt;&gt;
</code></pre>
<p>I guess mamba is to blame then…</p>