CUDA driver initialization failed. torch.cuda.is_available()False

username: open2smu
cooked: <p>I am facing an issue with CUDA initialization on my EC2 instance running Amazon Linux. My environment has an NVIDIA A10G GPU, and the <code>nvidia-smi</code> and <code>nvcc</code> commands show that the GPU and CUDA are correctly installed. However, when I try to use PyTorch to access the GPU, I get the error message indicating that the CUDA driver initialization failed.</p>
<p><strong>CUDA and Driver Info</strong>:</p>
<pre data-code-wrap="bash"><code class="lang-bash">$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Tue_Oct_29_23:50:19_PDT_2024
Cuda compilation tools, release 12.6, V12.6.85
Build cuda_12.6.r12.6/compiler.35059454_0
</code></pre>
<p><strong>NVIDIA-SMI</strong>:</p>
<pre data-code-wrap="bash"><code class="lang-bash">$ nvidia-smi
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 565.57.01              Driver Version: 565.57.01      CUDA Version: 12.7     |
|-----------------------------------------+------------------------+----------------------+   
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
|-----------------------------------------+------------------------+----------------------+  
|   0  NVIDIA A10G                    Off |   00000000:00:1E.0 Off |                    0 |
|  0%   23C    P0             56W /  300W |       1MiB /  23028MiB |      5%      Default |
+-----------------------------------------+------------------------+----------------------+
</code></pre>
<p><strong>PyTorch Script</strong>:</p>
<pre data-code-wrap="python"><code class="lang-python">import torch
print(torch.__version__)  # 2.5.1+cu124
print(torch.cuda.is_available())  # False
print(torch.cuda.device_count())  # 1
print(torch.cuda.get_device_name(0))  # Error: CUDA driver initialization failed
</code></pre>
<p>Output:</p>
<pre data-code-wrap="bash"><code class="lang-bash">False
Traceback (most recent call last):
  File "/home/ec2-user/LLM/test_cuda.py", line 5, in &lt;module&gt;
    print(torch.cuda.get_device_name(0))
  File "/home/ec2-user/py39/lib64/python3.9/site-packages/torch/cuda/__init__.py", line 493, in get_device_name
    return get_device_properties(device).name
  File "/home/ec2-user/py39/lib64/python3.9/site-packages/torch/cuda/__init__.py", line 523, in get_device_properties
    _lazy_init()  # will define _get_device_properties
  File "/home/ec2-user/py39/lib64/python3.9/site-packages/torch/cuda/__init__.py", line 319, in _lazy_init
    torch._C._cuda_init()
RuntimeError: CUDA driver initialization failed, you might not have a CUDA gpu.
</code></pre>
<p>Is there a conflict between the CUDA version installed (12.6) and the PyTorch version (2.5.1+cu124)? Why the CUDA driver initialization failed?</p>

username: Aknw_Fen
cooked: <p>Not an expert, just a suggestion. I think torch relies on you installing <a href="https://pytorch.org/get-started/locally/" rel="noopener nofollow ugc">from the installation matrix</a>, so it’ll work but use latest in the matrix (once you install it.)</p>
<p>See this</p>
<aside class="quote" data-post="1" data-topic="203718">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/juanouo/48/70414_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/does-pytorch-work-with-cuda-12-5/203718">Does Pytorch work with CUDA 12.5?</a> 
  </div>
  <blockquote>
    Hello! I’m new to PyTorch with CUDA and I’m trying to set it up on WSL. My CUDA version is 12.5, but the version matrix goes up to 12.4. Is this outdated or should I downgrade my CUDA for Pytorch to work? 
Thanks a lot
  </blockquote>
</aside>


username: open2smu
cooked: <p>The issue is resolved. It seems to be related to Amazon Linux requiring a special version of the GPU driver.<br>
I followed the instructions provided in the AWS documentation:</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/install-nvidia-driver.html">
  <header class="source">
      <img src="https://docs.aws.amazon.com/assets/images/favicon.ico" class="site-icon" width="48" height="48">

      <a href="https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/install-nvidia-driver.html" target="_blank" rel="noopener nofollow ugc">docs.aws.amazon.com</a>
  </header>

  <article class="onebox-body">
    

<h3><a href="https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/install-nvidia-driver.html" target="_blank" rel="noopener nofollow ugc">NVIDIA drivers for your Amazon EC2 instance - Amazon Elastic Compute Cloud</a></h3>

  <p>NVIDIA driver information and install instructions for EC2 instances.</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>
<p>
After downloading and installing the driver as described, everything is working fine now, and I can run torch without any issues.</p>