Torch.cuda.is_available() is False for CUDA version 11.4

username: gauravraul
cooked: <p>Hi All,</p>
<p>We are stuck with pytorch installation on server. Below are the collect.py details:<br>
(collect.py reference : <a href="https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py" rel="noopener nofollow ugc">https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py</a>)</p>
<p>(gpu_env) python collect.py</p>
<p>Collecting environment information…<br>
/opt/platformx/sentiment_analysis/gpu_env/lib64/python3.8/site-packages/torch/cuda/<strong>init</strong>.py:82: UserWarning: CUDA initialization: CUDA driver initialization failed, you might not have a CUDA gpu. (Triggered internally at  …/c10/cuda/CUDAFunctions.cpp:112.)<br>
return torch._C._cuda_getDeviceCount() &gt; 0<br>
PyTorch version: 1.11.0+cu113<br>
Is debug build: False<br>
CUDA used to build PyTorch: 11.3<br>
ROCM used to build PyTorch: N/A</p>
<p>OS: Red Hat Enterprise Linux 8.6 (Ootpa) (x86_64)<br>
GCC version: (GCC) 8.5.0 20210514 (Red Hat 8.5.0-10)<br>
Clang version: Could not collect<br>
CMake version: Could not collect<br>
Libc version: glibc-2.28</p>
<p>Python version: 3.8.12 (default, Sep 16 2021, 10:46:05)  [GCC 8.5.0 20210514 (Red Hat 8.5.0-3)] (64-bit runtime)<br>
Python platform: Linux-4.18.0-372.13.1.el8_6.x86_64-x86_64-with-glibc2.2.5<br>
Is CUDA available: False<br>
CUDA runtime version: 11.4.48<br>
GPU models and configuration: GPU 0: GRID M6-4Q<br>
Nvidia driver version: 470.82.01<br>
cuDNN version: Could not collect<br>
HIP runtime version: N/A<br>
MIOpen runtime version: N/A<br>
Is XNNPACK available: True</p>
<p>Versions of relevant libraries:<br>
[pip3] numpy==1.23.1<br>
[pip3] torch==1.11.0+cu113<br>
[conda] Could not collect</p>
<p>What all we have tried:<br>
-Installing torch==1.11.0+cu113, torch==1.12.0+cu113, torch==1.11.0+cu102, torch==1.12.0+cu102.<br>
-Installing from .whl files for python 3.8 and cu113<br>
-Upgrading pip and pip3<br>
-Tried a fresh virtual enviroenment.</p>
<p>We know two other ways, but not sure if it would work:</p>
<ol>
<li>Downgrading CUDA version from 11.4 to 11.3</li>
<li>Building pytroch for CUDA 11.4 from source.</li>
</ol>
<p><strong>We cannot use Anaconda as well, only pip is allowed.</strong></p>
<p><strong>The above methods require sudo permissions that we don’t have</strong>. So, it would be better if anyone can suggest alternatives or better solutions.</p>
<p>Thanks</p>
<p>nvidia-smi output :<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/d/7/d7bb2024eae68dcd2d98b748f367bdab46d4d03e.png" alt="nvidia_smi" data-base62-sha1="uMrzodF88aSP7cxYyXwRVS32rnw" width="639" height="304"></p>

username: ptrblck
cooked: <p>PyTorch is unable to communicate with the GPU as its initialization is failing:</p>
<pre><code class="lang-python">UserWarning: CUDA initialization: CUDA driver initialization failed, you might not have a CUDA gpu. 
</code></pre>
<p>In case you have installed the drivers recently, make sure to reboot the node.</p>

username: gauravraul
cooked: <p>Thanks for the reply,<br>
We tried restarting the server, but it still shows the same issue.<br>
Any other way?</p>

username: Mohammed_Yaseen
cooked: <p>I had the same issue with CUDA version 11.4. torch.cuda.is_available() gives True. But the gpu is not recognised when used with pytorch-lightning.</p>
<p>One thing different than the original question parameters was that I had access to conda. If somebody is looking for a possible solution:</p>
<ol>
<li>I created a fresh conda environment.</li>
<li>In that environment, I installed torch, torchvision and cudatoolkit for CUDA 11.3</li>
</ol>
<pre><code class="lang-auto"># CUDA 11.3
pip install torch==1.11.0+cu113 torchvision==0.12.0+cu113 torchaudio==0.11.0 --extra-index-url https://download.pytorch.org/whl/cu113
</code></pre>
<ol start="3">
<li>The rest of the packages, i installed via pip in the same environment.</li>
</ol>
<pre><code class="lang-auto">pip install -r requirements.txt
</code></pre>
<p>This solved the issue for me. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"> . The gpu is being used now.</p>

username: nobodybutyoulin
cooked: <p>I still encounter the torch.cuda.is_available() is False for CUDA version 11.4 on Jetson Orin NX 16GB platform<br>
Based on the instruction of <a href="http://pytorch.org" rel="noopener nofollow ugc">pytorch.org</a> <a href="https://pytorch.org/get-started/previous-versions/" rel="noopener nofollow ugc">https://pytorch.org/get-started/previous-versions/</a>, I use the following command to install torch and torchvision in miniconda virtual environment:</p>
<pre><code>conda install pytorch==1.12.1 torchvision==0.13.1 cudatoolkit=11.3  -c pytorch -c conda-forge
</code></pre>
<p>But from the result of collect_env.py, it seems that the installed version of torch and torchvision is CPU version. The env collected result as follows:</p>
<pre><code>PyTorch version: 1.12.1
Is debug build: False
CUDA used to build PyTorch: Could not collect
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.6 LTS (aarch64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Clang version: Could not collect
CMake version: version 3.16.3
Libc version: glibc-2.31

Python version: 3.8.16 (default, Mar  2 2023, 03:16:31)  [GCC 11.2.0] (64-bit 
runtime)
Python platform: Linux-5.10.104-tegra-aarch64-with-glibc2.26
Is CUDA available: False
CUDA runtime version: 11.4.315
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Probably one of the following:
/usr/lib/aarch64-linux-gnu/libcudnn.so.8.6.0
/usr/lib/aarch64-linux-gnu/libcudnn_adv_infer.so.8.6.0
/usr/lib/aarch64-linux-gnu/libcudnn_adv_train.so.8.6.0
/usr/lib/aarch64-linux-gnu/libcudnn_cnn_infer.so.8.6.0
/usr/lib/aarch64-linux-gnu/libcudnn_cnn_train.so.8.6.0
/usr/lib/aarch64-linux-gnu/libcudnn_ops_infer.so.8.6.0
/usr/lib/aarch64-linux-gnu/libcudnn_ops_train.so.8.6.0
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture:                    aarch64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
CPU(s):                          8
On-line CPU(s) list:             0-7
Thread(s) per core:              1
Core(s) per socket:              4
Socket(s):                       2
Vendor ID:                       ARM
Model:                           1
Model name:                      ARMv8 Processor rev 1 (v8l)
Stepping:                        r0p1
CPU max MHz:                     1984.0000
CPU min MHz:                     115.2000
BogoMIPS:                        62.50
L1d cache:                       512 KiB
L1i cache:                       512 KiB
L2 cache:                        2 MiB
L3 cache:                        4 MiB
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via 
prctl
Vulnerability Spectre v1:        Mitigation; __user pointer sanitization
Vulnerability Spectre v2:        Not affected
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected
Flags:                           fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp 
asimdhp cpuid asimdrdm lrcpc dcpop asimddp uscat ilrcpc flagm

Versions of relevant libraries:
[pip3] numpy==1.21.0
[pip3] torch==1.12.1
[pip3] torchvision==0.13.1a0
[conda] cudatoolkit               11.3.1              h89cd5c7_10    conda-forge
[conda] numpy                     1.21.3           py38hb9da153_0    conda-forge
[conda] pytorch                   1.12.1          cpu_py38ha09e9da_1  
[conda] torchvision               0.13.1          cpu_py38heb4ea19_0      
</code></pre>
<p>Is there any experience to resolve this issue?</p>

username: ptrblck
cooked: <p>For Jetson you would have to use the custom wheels from NVIDIA which you can find e.g. <a href="https://docs.nvidia.com/deeplearning/frameworks/install-pytorch-jetson-platform/index.html">here</a> and <a href="https://forums.developer.nvidia.com/t/pytorch-for-jetson/72048">here</a>.</p>

username: nobodybutyoulin
cooked: <p>Is there any success experience to porting Yolov5 on Jetson Orin NX 16GB with torch.cuda.is_available true?<br>
Actually what  make me to adopt the <a href="http://pytorch.org" rel="noopener nofollow ugc">pytorch.org</a> instruction is version mismatch problem between torch and torchvision after following the instruction from nvidia developer forums pytorch-for-jetson/72048 and nvidia doc install-pytorch-jetson-platform/index.html.  I also tried the pytorch docker in <a href="https://catalog.ngc.nvidia.com/orgs/nvidia/containers/l4t-pytorch" class="inline-onebox" rel="noopener nofollow ugc">NVIDIA L4T PyTorch | NVIDIA NGC</a> which can get torch.cuda.is_available() return true but encounter another trouble to use camera in docker.<br>
Before running the yolov5 code, I am not sure whether I still need to adjust the source code of torch module in upsampling.py of toch2.0 in l4t-pytorch docker image. Upsampling.py need some modification in torch 1.12.1 if in Yolov5 usage scenario.<br>
The code of my yolov5 project has been successfully run on Jetson Nano after following the pytorch and tochvison installation instruction from q-engineering qengineering.eu/install-pytorch-on-jetson-nano.html but it is not for Jetson Orin NX. The code can also run on Server with RTX3080Ti.</p>

username: ptrblck
cooked: <p>The issue with the binaries posted on <a href="http://PyTorch.org">PyTorch.org</a> is that no Jetson binaries are available as the ARM build should be for Macs and thus does not support CUDA. I don’t know if modifications are needed to run Yolo.</p>

username: nobodybutyoulin
cooked: <p>Update for test result of yolov5 on jetson Orin NX 16GB with Jetpack 5.1.1.<br>
Following the instruction of <a href="https://forums.developer.nvidia.com/t/pytorch-for-jetson/72048" rel="noopener nofollow ugc">nvidia developer forums</a>, I select Pytorch 1.13.0 and tochvision 0.13.0 build wheel. Except small modification of torch upsampling.py and torchvision qnms_kernel.cpp, yolov5 is run as expected performance on the Orin NX 16GB with torch.cuda.is_available true.<br>
Thanks <a class="mention" href="/u/ptrblck">@ptrblck</a></p>

username: ptrblck
cooked: <p>Great, thanks for sharing the outcome!</p>