PyTorch 2.3 with Cuda 12.4 wont download GPU version

Steffan_Jones: <p>Hi, I cant get pytorch to download the GPU version. I have tried multiple times and am coming up with the same issue.</p>
<p>±----------------------------------------------------------------------------------------+<br>
| NVIDIA-SMI 552.44                 Driver Version: 552.44         CUDA Version: 12.4     |<br>
|-----------------------------------------±-----------------------±---------------------+<br>
| GPU  Name                     TCC/WDDM  | Bus-Id          Disp.A | Volatile Uncorr. ECC |<br>
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |<br>
|                                         |                        |               MIG M. |<br>
|=========================================+========================+======================|<br>
|   0  NVIDIA GeForce RTX 2060      WDDM  |   00000000:01:00.0  On |                  N/A |<br>
| N/A   41C    P8              7W /   80W |      62MiB /   6144MiB |      0%      Default |<br>
|                                         |                        |                  N/A |<br>
±----------------------------------------±-----------------------±---------------------+</p>
<p>±----------------------------------------------------------------------------------------+<br>
| Processes:                                                                              |<br>
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |<br>
|        ID   ID                                                               Usage      |<br>
|=========================================================================================|<br>
±----------------------------------------------------------------------------------------+</p>
<p>nvcc: NVIDIA (R) Cuda compiler driver<br>
Copyright (c) 2005-2024 NVIDIA Corporation<br>
Built on Thu_Mar_28_02:30:10_Pacific_Daylight_Time_2024<br>
Cuda compilation tools, release 12.4, V12.4.131<br>
Build cuda_12.4.r12.4/compiler.34097967_0</p>
<p>and in python I run a script and get</p>
<p>AssertionError: Torch not compiled with CUDA enabled<br>
CUDA Available: False<br>
PyTorch version: 2.3.0+cpu<br>
CUDA Version: None<br>
Number of GPUs: 0</p>
<p>I tried installing:  pip install torch torchvision torchaudio --extra-index-url <a href="https://download.pytorch.org/whl/cu124" rel="noopener nofollow ugc">https://download.pytorch.org/whl/cu124</a></p>
<p>pip install torch torchvision torchaudio --extra-index-url <a href="https://download.pytorch.org/whl/cu121" rel="noopener nofollow ugc">https://download.pytorch.org/whl/cu121</a></p>
<p>All with the same outcome</p>

Steffan_Jones: <p>also tried: pip install torch==2.3.0+cu121 torchvision==0.18.0+cu121 torchaudio==2.3.0+cu121 -f <a href="https://download.pytorch.org/whl/torch_stable.html" rel="noopener nofollow ugc">https://download.pytorch.org/whl/torch_stable.html</a></p>

ptrblck: <p>Assuming you are using a supported Python version, just run <code>pip install torch</code> and it’ll install the latest PyTorch release with CUDA dependencies.</p>

blair: <p>Oh, how I wish that were true for aarch64–although admittedly I have not actually tried simply ‘pip install torch’.  I’ve looked for the wheels on your site, but I suppose there is always the possibility that I missed them somehow.  If that’s the case, please simply point me in the right direction and disregard the rest of this message, heh.  A 2.3.0+cu121 aarch64 container appeared briefly on Docker Hub, and I was elated.  Then, it disappeared.  I know that the NGC containers exist, but they don’t use pytorch releases, have some non-standard names for other packages, and a whole bunch of packages I don’t need.  I presume that you guys removed that container for a reason, but if it should indeed be there, please put it back! ;-). The on-demand pricing for GH200s is just too good for me to pass up.</p>
<p>Thanks.</p>

ptrblck: <p>The ARM+CUDA wheels are not published yet, but if you don’t want to use the NGC containers, you can download and install the wheels from the builder jobs, e.g. <a href="https://github.com/pytorch/pytorch/actions/runs/9115630888?pr=126174">this one</a> (scroll down and download the wheel for the Python version you are using).</p>
<p>Note that only <code>sm_90</code> is currently supported in these wheels.</p>

mariam-162: <p>Hi, I am using NVIDIA 3090 on Windows 11 with driver version 551.86 and CUDA 12.4. I created a new environment using conda create -n newEnv python=3.10, and activated. The default packages are:</p>
<p>Package    Version</p>
<hr>
<p>pip        24.0<br>
setuptools 69.5.1<br>
wheel      0.43.0</p>
<p>I installed:<br>
py -m pip install nvidia-cudnn-cu12<br>
conda install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia</p>
<p>Input:<br>
import torch; print(torch.cuda.is_available()); print(torch.<strong>version</strong>)<br>
Output:<br>
True<br>
‘2.3.0’</p>
<p>I want to know why my pytroch version is not CUDA compiled? I expect the pytorch version to be something like: 2.3.0+cu121.</p>
<p>Thanks for your support in advance.</p>

ptrblck: <aside class="quote no-group" data-username="mariam-162" data-post="8" data-topic="202757">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mariam-162/48/69635_2.png" class="avatar"> mariam-162:</div>
<blockquote>
<p>I expect the pytorch version to be something like: 2.3.0+cu121</p>
</blockquote>
</aside>
<p>This would be the case for pip wheels, but not conda binaries. Check the CUDA runtime version via <code>torch.version.cuda</code> and keep in mind your locally installed CUDA toolkit won’t be used since PyTorch ships with its own runtime dependencies.</p>