Not detecting GPU RTX 4000

username: ricksant2003
cooked: <p>Hello<br>
I am trying to install pytorch in Ubunut Mint 21 and use it with RTX 4000. First I’ve installed all drivers and cuda (from cuda_12.2.1_535.86.10_linux.run). Here are some outputs (local user - not root):</p>
<p>$ nividia-smi<br>
Mon Aug  7 08:50:44 2023<br>
±--------------------------------------------------------------------------------------+<br>
| NVIDIA-SMI 535.86.05              Driver Version: 535.86.05    CUDA Version: 12.2     |<br>
|-----------------------------------------±---------------------±---------------------+<br>
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |<br>
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |<br>
|                                         |                      |               MIG M. |<br>
|=========================================+======================+======================|<br>
|   0  NVIDIA RTX A4000               Off | 00000000:01:00.0 Off |                  Off |<br>
| 41%   40C    P8              15W / 140W |    103MiB / 16376MiB |      0%      Default |<br>
|                                         |                      |                  N/A |<br>
±----------------------------------------±---------------------±---------------------+</p>
<p>±--------------------------------------------------------------------------------------+<br>
| Processes:                                                                            |<br>
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |<br>
|        ID   ID                                                             Usage      |<br>
|=======================================================================================|<br>
|    0   N/A  N/A      1143      G   /usr/lib/xorg/Xorg                           97MiB |<br>
±--------------------------------------------------------------------------------------+</p>
<p>$ /usr/local/cuda/extras/demo_suite/deviceQuery<br>
Starting…</p>
<p>CUDA Device Query (Runtime API) version (CUDART static linking)</p>
<p>Detected 1 CUDA Capable device(s)</p>
<p>Device 0: “NVIDIA RTX A4000”<br>
CUDA Driver Version / Runtime Version          12.2 / 12.2<br>
CUDA Capability Major/Minor version number:    8.6<br>
Total amount of global memory:                 16101 MBytes (16882663424 bytes)<br>
(48) Multiprocessors, (128) CUDA Cores/MP:     6144 CUDA Cores<br>
GPU Max Clock rate:                            1560 MHz (1.56 GHz)<br>
Memory Clock rate:                             7001 Mhz<br>
Memory Bus Width:                              256-bit<br>
L2 Cache Size:                                 4194304 bytes<br>
Maximum Texture Dimension Size (x,y,z)         1D=(131072), 2D=(131072, 65536), 3D=(16384, 16384, 16384)<br>
Maximum Layered 1D Texture Size, (num) layers  1D=(32768), 2048 layers<br>
Maximum Layered 2D Texture Size, (num) layers  2D=(32768, 32768), 2048 layers<br>
Total amount of constant memory:               65536 bytes<br>
Total amount of shared memory per block:       49152 bytes<br>
Total number of registers available per block: 65536<br>
Warp size:                                     32<br>
Maximum number of threads per multiprocessor:  1536<br>
Maximum number of threads per block:           1024<br>
Max dimension size of a thread block (x,y,z): (1024, 1024, 64)<br>
Max dimension size of a grid size    (x,y,z): (2147483647, 65535, 65535)<br>
Maximum memory pitch:                          2147483647 bytes<br>
Texture alignment:                             512 bytes<br>
Concurrent copy and kernel execution:          Yes with 2 copy engine(s)<br>
Run time limit on kernels:                     Yes<br>
Integrated GPU sharing Host Memory:            No<br>
Support host page-locked memory mapping:       Yes<br>
Alignment requirement for Surfaces:            Yes<br>
Device has ECC support:                        Disabled<br>
Device supports Unified Addressing (UVA):      Yes<br>
Device supports Compute Preemption:            Yes<br>
Supports Cooperative Kernel Launch:            Yes<br>
Supports MultiDevice Co-op Kernel Launch:      Yes<br>
Device PCI Domain ID / Bus ID / location ID:   0 / 1 / 0<br>
Compute Mode:<br>
&lt; Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) &gt;</p>
<p>deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 12.2, CUDA Runtime Version = 12.2, NumDevs = 1, Device0 = NVIDIA RTX A4000<br>
Result = PASS</p>
<p>So, its seems for me drivers from nvidia are ok and cuda installation is ol<br>
I’ve installed Anaconda and then I ’ ve created a dl_pytorch environment (with all anaconda packages). To install pytorch I used the following command:</p>
<p>(dl_pytorch) $ conda install pytorch torchvision torchaudio pytorch-cuda=11.8 cudatoolkit -c pytorch -c nvidia</p>
<p>(I am not sure I have to install cudatoolkit, so  i 've tried to run the above command with and without cudatoolkit. None of them worlked out)<br>
After installation, I tried to run the following command:<br>
(dl_pytorch) $ python -c “import torch; print(torch.cuda.is_available())”<br>
False</p>
<p>I am non-user in all above commands. I am using Ubuntu Mint. Any clues ?</p>
<p>Thanks</p>

username: ptrblck
cooked: <p>Check if you’ve installed the right PyTorch binaries shipping with CUDA by checking <code>torch.version.cuda</code>, which should return the selected version. If it returns <code>None</code> you’ve installed the CPU-only binary.</p>

username: ricksant2003
cooked: <aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="185867">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>torch.version.cuda</p>
</blockquote>
</aside>
<p>Ok. I am using cpu version. but how can I change it to force it to install gpu. When I use conda search pytorch -c pytorch, I got:</p>
<p>pytorch                        2.0.1    py3.11_cpu_0  pytorch<br>
pytorch                        2.0.1 py3.11_cuda11.7_cudnn8.5.0_0  pytorch<br>
pytorch                        2.0.1 py3.11_cuda11.8_cudnn8.7.0_0  pytorch</p>
<p>so, I think i have to install pytorch=2.0.1=py3.11_cuda11.8_cudnn8.7.0_0<br>
is it correct ?</p>
<p>BTW, Now I was wondering: do I need to install cudnn ?? because I did not install it (just cuda).<br>
thanks in advance</p>

username: ptrblck
cooked: <p>No, neither your locally installed CUDA toolkit nor cuDNN will be used as PyTorch ships with its own CUDA dependencies.<br>
Use the posted install commands from the install matrix and it should work.</p>

username: ricksant2003
cooked: <aside class="quote no-group" data-username="ricksant2003" data-post="3" data-topic="185867">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/r/a9a28c/48.png" class="avatar"> ricksant2003:</div>
<blockquote>
<p>py3.11_cuda11.8_cudnn8.7.0_0 pytorch</p>
</blockquote>
</aside>
<p>$ conda install  pytorch=2.0.1=py3.11_cuda11.8_cudnn8.7.0_0 torchvision torchaudio pytorch-cuda=11.8 cudatoolkit -c pytorch -c nvidia</p>
<p>it worked perfectly ! thanks</p>