CUDA initialization: CUDA driver initialization failed

vasigorc: <p>Hello! I have a 1 year old laptop from System 76 that shipped with POP!_OS 22.04 which is a fork of the same Ubuntu version:</p>
<pre data-code-wrap="bash"><code class="lang-bash">cat /etc/os-release
NAME="Pop!_OS"
VERSION="22.04 LTS"
ID=pop
ID_LIKE="ubuntu debian"
PRETTY_NAME="Pop!_OS 22.04 LTS"
VERSION_ID="22.04"
HOME_URL="https://pop.system76.com"
SUPPORT_URL="https://support.system76.com"
BUG_REPORT_URL="https://github.com/pop-os/pop/issues"
PRIVACY_POLICY_URL="https://system76.com/privacy"
VERSION_CODENAME=jammy
UBUNTU_CODENAME=jammy
LOGO=distributor-logo-pop-os
</code></pre>
<p>It does have Nvidia GPU:</p>
<pre data-code-wrap="bash"><code class="lang-bash"> nvidia-smi
Thu Jun 26 09:18:42 2025       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 570.153.02             Driver Version: 570.153.02     CUDA Version: 12.8     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4080 ...    Off |   00000000:02:00.0 Off |                  N/A |
| N/A   43C    P0            593W /  175W |      15MiB /  12282MiB |     15%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
                                                                                         
+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A            3886      G   /usr/lib/xorg/Xorg                        4MiB |
+-----------------------------------------------------------------------------------------+
</code></pre>
<p>The default OS CUDA toolkit was of a far inferior version:</p>
<pre data-code-wrap="bash"><code class="lang-bash">apt-cache show nvidia-cuda-toolkit | grep Version
Version: 11.5.1-1ubuntu1
</code></pre>
<p>So I had to remove it and followed the <a href="https://docs.nvidia.com/cuda/archive/12.8.0/cuda-installation-guide-linux/index.html#network-repo-installation-for-ubuntu" rel="noopener nofollow ugc">official Nvidia installation guide for CUDA 12.8 version</a>:</p>
<pre data-code-wrap="bash"><code class="lang-bash"> 9576* sudo apt remove nvidia-cuda-toolkit
 9577* sudo apt autoremove
 9578* wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.0-1_all.deb
 9579* sudo dpkg -i cuda-keyring_1.0-1_all.deb
 9580* sudo apt update
 9581* sudo apt install cuda-toolkit-12-8

 9608  sudo apt-get install nvidia-gds-12-8
</code></pre>
<p>I’ve also followed the <a href="https://docs.nvidia.com/cuda/archive/12.8.0/cuda-installation-guide-linux/index.html#post-installation-actions" rel="noopener nofollow ugc">post installation guide</a> to make sure that <code>nvcc</code> and all is on the path:</p>
<pre data-code-wrap="bash"><code class="lang-bash">echo $PATH | grep -i cuda
/usr/local/cuda-12.8/bin:/home/vasilegorcinschi/go/bin:/usr/local/go/bin:/usr/local/cuda-12.8/bin:...
...
echo $LD_LIBRARY_PATH
/usr/local/cuda-12.6/lib64
...
nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2025 NVIDIA Corporation
Built on Fri_Feb_21_20:23:50_PST_2025
Cuda compilation tools, release 12.8, V12.8.93
Build cuda_12.8.r12.8/compiler.35583870_0
</code></pre>
<p>Finally, I’ve re-intssalled <code>torch</code> for <code>cu128</code> (I had previously also installed it when I had 11.5 cudatoolkit) with the command prompted on PyTorch’s home page:</p>
<pre data-code-wrap="bash"><code class="lang-bash"> 9638  pip3 uninstall torch torchvision torchaudio -y
 9639  pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128
  pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128
  Defaulting to user installation because normal site-packages is not writeable
  Looking in indexes: https://download.pytorch.org/whl/cu128
  Collecting torch
    Using cached https://download.pytorch.org/whl/cu128/torch-2.7.1%2Bcu128-cp310-cp310-manylinux_2_28_x86_64.whl (1039.4 MB)
  Collecting torchvision
    Using cached https://download.pytorch.org/whl/cu128/torchvision-0.22.1%2Bcu128-cp310-cp310-manylinux_2_28_x86_64.whl (8.7 MB)
  Collecting torchaudio
    Using cached https://download.pytorch.org/whl/cu128/torchaudio-2.7.1%2Bcu128-cp310-cp310-manylinux_2_28_x86_64.whl (3.9 MB)
  Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.8.61 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (12.8.61)
  Requirement already satisfied: typing-extensions&gt;=4.10.0 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (4.12.2)
  Requirement already satisfied: nvidia-cufile-cu12==1.13.0.11 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (1.13.0.11)
  Requirement already satisfied: sympy&gt;=1.13.3 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (1.13.3)
  Requirement already satisfied: nvidia-cufft-cu12==11.3.3.41 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (11.3.3.41)
  Requirement already satisfied: nvidia-cusparse-cu12==12.5.7.53 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (12.5.7.53)
  Requirement already satisfied: filelock in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (3.16.1)
  Requirement already satisfied: nvidia-nvjitlink-cu12==12.8.61 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (12.8.61)
  Requirement already satisfied: networkx in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (3.3)
  Requirement already satisfied: triton==3.3.1 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (3.3.1)
  Requirement already satisfied: fsspec in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (2024.9.0)
  Requirement already satisfied: nvidia-curand-cu12==10.3.9.55 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (10.3.9.55)
  Requirement already satisfied: nvidia-nvtx-cu12==12.8.55 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (12.8.55)
  Requirement already satisfied: jinja2 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (3.1.4)
  Requirement already satisfied: nvidia-cublas-cu12==12.8.3.14 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (12.8.3.14)
  Requirement already satisfied: nvidia-cusparselt-cu12==0.6.3 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (0.6.3)
  Requirement already satisfied: nvidia-cudnn-cu12==9.7.1.26 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (9.7.1.26)
  Requirement already satisfied: nvidia-cusolver-cu12==11.7.2.55 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (11.7.2.55)
  Requirement already satisfied: nvidia-cuda-cupti-cu12==12.8.57 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (12.8.57)
  Requirement already satisfied: nvidia-nccl-cu12==2.26.2 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (2.26.2)
  Requirement already satisfied: nvidia-cuda-runtime-cu12==12.8.57 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (12.8.57)
  Requirement already satisfied: setuptools&gt;=40.8.0 in /usr/lib/python3/dist-packages (from triton==3.3.1-&gt;torch) (59.6.0)
  Requirement already satisfied: numpy in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torchvision) (1.26.4)
  Requirement already satisfied: pillow!=8.3.*,&gt;=5.3.0 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torchvision) (10.1.0)
  Requirement already satisfied: mpmath&lt;1.4,&gt;=1.1.0 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from sympy&gt;=1.13.3-&gt;torch) (1.3.0)
  Requirement already satisfied: MarkupSafe&gt;=2.0 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from jinja2-&gt;torch) (3.0.2)
  Installing collected packages: torch, torchvision, torchaudio
  Successfully installed torch-2.7.1+cu128 torchaudio-2.7.1+cu128 torchvision-0.22.1+cu128
</code></pre>
<p>But when I try to check for CUDA availability I get a failure (just I had it when using the default toolkit):</p>
<pre data-code-wrap="bash"><code class="lang-bash">python -c 'import torch; print(torch.__version__, torch.version.cuda, torch.cuda.is_available())'
/home/vasilegorcinschi/.local/lib/python3.10/site-packages/torch/cuda/__init__.py:174: UserWarning: CUDA initialization: CUDA driver initialization failed, you might not have a CUDA gpu. (Triggered internally at /pytorch/c10/cuda/CUDAFunctions.cpp:109.)
  return torch._C._cuda_getDeviceCount() &gt; 0
  2.7.1+cu128 12.8 False
</code></pre>
<p>Could you please kindly help me? Any help will be greatly appreciated!</p>

ptrblck: <p>Your NVIDIA driver is not properly installed and you might need to reinstall it. We have also seen other topics here where users ignored the request to reboot the system after upgrading the driver, so you might also want to make sure your laptop was rebooted already.</p>

vasigorc: <p>Hm I installed the driver as <a href="https://support.system76.com/articles/system76-driver/" rel="noopener nofollow ugc">instructed by Ssytem76</a>:</p>
<pre data-code-wrap="bash"><code class="lang-bash">sudo apt-get install system76-driver-nvidia
[sudo] password for vasilegorcinschi: 
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
system76-driver-nvidia is already the newest version (24.04.5~1749231807~22.04~6e6b806).
The following package was automatically installed and is no longer required:
  dctrl-tools
Use 'sudo apt autoremove' to remove it.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
</code></pre>
<p>Maybe I need to reach out to them.</p>

vasigorc: <p>To be fair, I’ve downloaded and installed <a href="https://github.com/nvidia/cuda-samples" rel="noopener nofollow ugc">cuda-samples</a> and the tests fail for me, e.g.:</p>
<pre data-code-wrap="bash"><code class="lang-bash">./build/Samples/1_Utilities/deviceQuery/deviceQuery
./build/Samples/1_Utilities/deviceQuery/deviceQuery Starting...

 CUDA Device Query (Runtime API) version (CUDART static linking)

cudaGetDeviceCount returned 3
-&gt; initialization error
Result = FAIL
</code></pre>
<p>FWIW, some people also advised to assure that <code>sudo systemctl status nvidia-fabricmanager</code> is running and enabled, and I don’t have it installed, but I am not sure if I try to install if it won’t make the things even more complicated.</p>

ptrblck: <p>Thanks for confirming the same errors using plain CUDA samples!</p>
<p>Fabric Manager is used for NVSwitch systems. Since your system has a single GPU it won’t be needed and is unrelated to your errors.</p>

vasigorc: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> could you please confirm/refute whether <code>cuda-toolkit</code> is required for <code>PyTorch</code> to be able to leverage GPU?</p>

ptrblck: <p>The PyTorch binaries ship with their own CUDA runtime dependencies and a locally installed CUDA toolkit is not required unless you build PyTorch from source or a custom CUDA extension.<br>
Since the CUDA Samples already fail, the issue is unrelated to PyTorch and you should check your driver installation.</p>

vasigorc: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> I think that we’re good to close those. As guided by a System76’s support engineer I’ve downgraded and re-installed Nvidia driver and Pytorch now is able to recognize CUDA on my machine. Thank you for your help!</p>
<pre data-code-wrap="bash"><code class="lang-bash">python3 -c 'import torch; print(torch.__version__, torch.version.cuda, torch.cuda.is_available())'
2.7.1+cu128 12.8 True
</code></pre>

ptrblck: <p>Great, thank you for confirming!</p>