[No CUDA GPUs are available] CUDA 12.2 w/ PyTorch 2.1.0+cu121

James_Jang: <p>Hello,</p>
<p>I repeatedly face this “No CUDA GPUs are available” error, although it seems that all the library versions are compatible.</p>
<p>My torch is <code>2.1.0+cu121</code> and CUDA is <code>12.2</code>. Could anyone check the issue I’m facing with?</p>
<pre><code class="lang-auto">Mon Nov 11 17:46:36 2024
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.161.07             Driver Version: 535.161.07   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA RTX 6000 Ada Gene...    On  | 00000000:01:00.0 Off |                  Off |
| 30%   26C    P8              24W / 300W |      1MiB / 49140MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
|   1  NVIDIA RTX 6000 Ada Gene...    On  | 00000000:25:00.0 Off |                  Off |
| 30%   25C    P8              22W / 300W |      1MiB / 49140MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
|   2  NVIDIA RTX 6000 Ada Gene...    On  | 00000000:41:00.0 Off |                  Off |
| 30%   26C    P8              20W / 300W |      1MiB / 49140MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
|   3  NVIDIA RTX 6000 Ada Gene...    On  | 00000000:61:00.0 Off |                  Off |
| 30%   25C    P8              19W / 300W |      1MiB / 49140MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
|   4  NVIDIA RTX 6000 Ada Gene...    On  | 00000000:81:00.0 Off |                  Off |
| 30%   25C    P8              19W / 300W |      1MiB / 49140MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
|   5  NVIDIA RTX 6000 Ada Gene...    On  | 00000000:A1:00.0 Off |                  Off |
| 30%   24C    P8              19W / 300W |      1MiB / 49140MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
|   6  NVIDIA RTX 6000 Ada Gene...    On  | 00000000:C1:00.0 Off |                  Off |
| 30%   25C    P8              21W / 300W |      1MiB / 49140MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
|   7  NVIDIA RTX 6000 Ada Gene...    On  | 00000000:E1:00.0 Off |                  Off |
| 30%   24C    P8              25W / 300W |      1MiB / 49140MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|  No running processes found                                                           |
+---------------------------------------------------------------------------------------+
</code></pre>
<pre><code class="lang-auto">(myenv) myname@myserver:~$ pip list
Package                   Version
------------------------- --------------
anyio                     4.6.2.post1
argon2-cffi               23.1.0
argon2-cffi-bindings      21.2.0
arrow                     1.3.0
asttokens                 2.4.1
async-lru                 2.0.4
attrs                     24.2.0
babel                     2.16.0
beautifulsoup4            4.12.3
bleach                    6.2.0
Brotli                    1.0.9
certifi                   2024.8.30
cffi                      1.17.1
charset-normalizer        3.4.0
comm                      0.2.2
contourpy                 1.3.0
cycler                    0.12.1
debugpy                   1.8.7
decorator                 5.1.1
defusedxml                0.7.1
exceptiongroup            1.2.2
executing                 2.1.0
fastjsonschema            2.20.0
filelock                  3.13.1
fonttools                 4.54.1
fqdn                      1.5.1
fsspec                    2024.10.0
gmpy2                     2.1.2
h11                       0.14.0
httpcore                  1.0.6
httpx                     0.27.2
huggingface-hub           0.26.2
idna                      3.10
ipykernel                 6.29.5
ipython                   8.29.0
ipywidgets                8.1.5
isoduration               20.11.0
jedi                      0.19.2
Jinja2                    3.1.4
json5                     0.9.25
jsonpointer               3.0.0
jsonschema                4.23.0
jsonschema-specifications 2024.10.1
jupyter                   1.1.1
jupyter_client            8.6.3
jupyter-console           6.6.3
jupyter_core              5.7.2
jupyter-events            0.10.0
jupyter-lsp               2.2.5
jupyter_server            2.14.2
jupyter_server_terminals  0.5.3
jupyterlab                4.2.5
jupyterlab_pygments       0.3.0
jupyterlab_server         2.27.3
jupyterlab_widgets        3.0.13
kagglehub                 0.3.4
kiwisolver                1.4.7
MarkupSafe                2.1.3
matplotlib                3.9.2
matplotlib-inline         0.1.7
mistune                   3.0.2
mkl_fft                   1.3.11
mkl_random                1.2.8
mkl-service               2.4.0
mpmath                    1.3.0
nbclient                  0.10.0
nbconvert                 7.16.4
nbformat                  5.10.4
nest-asyncio              1.6.0
networkx                  3.2.1
notebook                  7.2.2
notebook_shim             0.2.4
numpy                     1.23.5
nvidia-cublas-cu12        12.4.5.8
nvidia-cuda-cupti-cu12    12.4.127
nvidia-cuda-nvrtc-cu12    12.4.127
nvidia-cuda-runtime-cu12  12.4.127
nvidia-cudnn-cu12         9.1.0.70
nvidia-cufft-cu12         11.2.1.3
nvidia-curand-cu12        10.3.5.147
nvidia-cusolver-cu12      11.6.1.9
nvidia-cusparse-cu12      12.3.1.170
nvidia-nccl-cu12          2.21.5
nvidia-nvjitlink-cu12     12.4.127
nvidia-nvtx-cu12          12.4.127
overrides                 7.7.0
packaging                 24.1
pandas                    2.2.3
pandocfilters             1.5.1
parso                     0.8.4
pexpect                   4.9.0
pillow                    10.4.0
pip                       24.2
platformdirs              4.3.6
prometheus_client         0.21.0
prompt_toolkit            3.0.48
psutil                    6.1.0
ptyprocess                0.7.0
pure_eval                 0.2.3
pycparser                 2.22
Pygments                  2.18.0
pyparsing                 3.2.0
PySocks                   1.7.1
python-dateutil           2.9.0.post0
python-json-logger        2.0.7
pytz                      2024.2
PyYAML                    6.0.2
pyzmq                     26.2.0
referencing               0.35.1
regex                     2024.11.6
requests                  2.32.3
rfc3339-validator         0.1.4
rfc3986-validator         0.1.1
rpds-py                   0.21.0
safetensors               0.4.5
Send2Trash                1.8.3
sentencepiece             0.2.0
setuptools                75.1.0
six                       1.16.0
sniffio                   1.3.1
soupsieve                 2.6
stack-data                0.6.3
sympy                     1.13.1
terminado                 0.18.1
tinycss2                  1.4.0
tokenizers                0.20.3
tomli                     2.0.2
torch                     2.1.0+cu121
torchaudio                2.1.0+cu121
torchvision               0.16.0+cu121
tornado                   6.4.1
tqdm                      4.67.0
traitlets                 5.14.3
transformers              4.46.2
triton                    2.1.0
types-python-dateutil     2.9.0.20241003
typing_extensions         4.12.2
tzdata                    2024.2
uri-template              1.3.0
urllib3                   2.2.3
wcwidth                   0.2.13
webcolors                 24.8.0
webencodings              0.5.1
websocket-client          1.8.0
wheel                     0.44.0
widgetsnbextension        4.0.13
</code></pre>

ptrblck: <p>Most likely your NVIDIA drivers aren’t properly installed and thus PyTorch fails to communicate with your GPUs.</p>