Nvidia Xavier AGX torch.cuda.is_available() returns false

Thomas_James: <p>On the Nvidia Xavier AGX A false is returned with the command torch.cuda.is_available().</p>
<p>Here are the python commands</p>
<blockquote>
<blockquote>
<blockquote>
<p>import torch<br>
torch.<strong>version</strong><br>
‘1.9.0’<br>
x = torch.rand(5, 3)<br>
print(x)<br>
tensor([[0.2251, 0.7957, 0.5024],<br>
[0.0310, 0.7917, 0.1989],<br>
[0.5037, 0.4068, 0.3340],<br>
[0.0275, 0.4699, 0.5500],<br>
[0.8979, 0.1096, 0.2719]])<br>
torch.version.cuda<br>
torch.cuda.is_available()<br>
False</p>
</blockquote>
</blockquote>
</blockquote>
<p>*********** At the Ubuntu prompt I get this for Cuda<br>
Command: cat /usr/local/cuda/version.txt<br>
Returns:     CUDA Version 10.2.300</p>
<hr>
<p>Command: ./deviceQuery<br>
Returns:</p>
<p>CUDA Device Query (Runtime API) version (CUDART static linking)</p>
<p>Detected 1 CUDA Capable device(s)</p>
<p>Device 0: “Xavier”<br>
CUDA Driver Version / Runtime Version          10.2 / 10.2<br>
CUDA Capability Major/Minor version number:    7.2<br>
Total amount of global memory:                 15817 MBytes (16584876032 bytes)<br>
( 8) Multiprocessors, ( 64) CUDA Cores/MP:     512 CUDA Cores<br>
GPU Max Clock rate:                            1377 MHz (1.38 GHz)<br>
Memory Clock rate:                             1377 Mhz<br>
Memory Bus Width:                              256-bit<br>
L2 Cache Size:                                 524288 bytes<br>
Maximum Texture Dimension Size (x,y,z)         1D=(131072), 2D=(131072, 65536), 3D=(16384, 16384, 16384)<br>
Maximum Layered 1D Texture Size, (num) layers  1D=(32768), 2048 layers<br>
Maximum Layered 2D Texture Size, (num) layers  2D=(32768, 32768), 2048 layers<br>
Total amount of constant memory:               65536 bytes<br>
Total amount of shared memory per block:       49152 bytes<br>
Total number of registers available per block: 65536<br>
Warp size:                                     32<br>
Maximum number of threads per multiprocessor:  2048<br>
Maximum number of threads per block:           1024<br>
Max dimension size of a thread block (x,y,z): (1024, 1024, 64)<br>
Max dimension size of a grid size    (x,y,z): (2147483647, 65535, 65535)<br>
Maximum memory pitch:                          2147483647 bytes<br>
Texture alignment:                             512 bytes<br>
Concurrent copy and kernel execution:          Yes with 1 copy engine(s)<br>
Run time limit on kernels:                     No<br>
Integrated GPU sharing Host Memory:            Yes<br>
Support host page-locked memory mapping:       Yes<br>
Alignment requirement for Surfaces:            Yes<br>
Device has ECC support:                        Disabled<br>
Device supports Unified Addressing (UVA):      Yes<br>
Device supports Compute Preemption:            Yes<br>
Supports Cooperative Kernel Launch:            Yes<br>
Supports MultiDevice Co-op Kernel Launch:      Yes<br>
Device PCI Domain ID / Bus ID / location ID:   0 / 0 / 0<br>
Compute Mode:<br>
&lt; Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) &gt;</p>
<p>deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 10.2, CUDA Runtime Version = 10.2, NumDevs = 1<br>
Result = PASS</p>
<hr>
<p>I am using Python 3.6<br>
Torch 1.9.0<br>
torchaudio   0.10.0<br>
torchvision   0.10.0<br>
Cuda 10.2</p>
<p>pandas                        1.1.5<br>
numpy                         1.19.4</p>
<p>Any help would be greatly appreciated</p>

tom: <p>Where did you get the PyTorch from and what does collect_env.py say?<br>
You can take it from your PyTorch distribution or use these instructions from the bug report template:</p>
<pre><code class="lang-auto">wget https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py
# For security purposes, please check the contents of collect_env.py before running it.
python collect_env.py
</code></pre>
<p>Best regards</p>
<p>Thomas</p>

Thomas_James: <p>First let me say I really appreciate you responding back hopefully you can help</p>
<p>I got pytorch from the pythorch website <a href="https://pytorch.org/get-started/locally/" class="inline-onebox" rel="noopener nofollow ugc">Start Locally | PyTorch</a><br>
This what I used<br>
<a href="https://pytorch.org/get-started/locally" class="inline-onebox" rel="noopener nofollow ugc">Start Locally | PyTorch</a></p>
<p>pip3 install torch torchvision torchaudio<br>
PyTorch Build<br>
Your OS   Linux<br>
Package   Pip<br>
Language  Python<br>
Compute Platform  CUDA 10.2<br>
Run this Command:<br>
pip3 install torch torchvision torchaudio</p>
<p>This is what I get form collect</p>
<p>Collecting environment information…<br>
PyTorch version: 1.9.0<br>
Is debug build: False<br>
CUDA used to build PyTorch: Could not collect<br>
ROCM used to build PyTorch: N/A</p>
<p>OS: Ubuntu 18.04.5 LTS (aarch64)<br>
GCC version: (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04) 7.5.0<br>
Clang version: Could not collect<br>
CMake version: version 3.10.2<br>
Libc version: glibc-2.25</p>
<p>Python version: 3.6.9 (default, Jan 26 2021, 15:33:00)  [GCC 8.4.0] (64-bit runtime)<br>
Python platform: Linux-4.9.253-tegra-aarch64-with-Ubuntu-18.04-bionic<br>
Is CUDA available: False<br>
CUDA runtime version: 10.2.300<br>
GPU models and configuration: Could not collect<br>
Nvidia driver version: Could not collect<br>
cuDNN version: Probably one of the following:<br>
/usr/lib/aarch64-linux-gnu/libcudnn.so.8.2.1<br>
/usr/lib/aarch64-linux-gnu/libcudnn_adv_infer.so.8.2.1<br>
/usr/lib/aarch64-linux-gnu/libcudnn_adv_train.so.8.2.1<br>
/usr/lib/aarch64-linux-gnu/libcudnn_cnn_infer.so.8.2.1<br>
/usr/lib/aarch64-linux-gnu/libcudnn_cnn_train.so.8.2.1<br>
/usr/lib/aarch64-linux-gnu/libcudnn_ops_infer.so.8.2.1<br>
/usr/lib/aarch64-linux-gnu/libcudnn_ops_train.so.8.2.1<br>
HIP runtime version: N/A<br>
MIOpen runtime version: N/A</p>
<p>Versions of relevant libraries:<br>
[pip3] mypy-extensions==0.4.3<br>
[pip3] numpy==1.19.4<br>
[pip3] torch==1.9.0<br>
[pip3] torchaudio==0.10.0<br>
[pip3] torchvision==0.10.0<br>
[conda] Could not collect</p>

tom: <p>I think the ARM64 builds from <a href="http://PyTorch.org" rel="noopener nofollow ugc">PyTorch.org</a> don’t include CUDA, so you would need the libraries from NVIDIA<br>
<a href="https://forums.developer.nvidia.com/t/pytorch-for-jetson-version-1-10-now-available/72048" class="inline-onebox" rel="noopener nofollow ugc">PyTorch for Jetson - version 1.10 now available - Jetson Nano - NVIDIA Developer Forums</a></p>
<p>(I also built <a href="http://mathinf.com/pytorch/arm64/2021-04/" rel="noopener nofollow ugc">packages for it</a>, but you’ll likely be happier with the ones from NVIDIA.)</p>
<p>Best regards</p>
<p>Thomas</p>

Thomas_James: <p>Thank you very very much Thomas when I do these commands I know get</p>
<blockquote>
<blockquote>
<blockquote>
<p>torch.cuda.is_available()<br>
True<br>
torch.zeros(1).cuda()<br>
tensor([0.], device=‘cuda:0’)</p>
</blockquote>
</blockquote>
</blockquote>
<p>I think the problem is I was using torch 1.9 when I used what you did they downloaded version 1.8.<br>
Not sure but it works. Thanks</p>
<p>Collecting environment information…<br>
PyTorch version: 1.8.0<br>
Is debug build: False<br>
CUDA used to build PyTorch: 10.2<br>
ROCM used to build PyTorch: N/A</p>
<p>OS: Ubuntu 18.04.5 LTS (aarch64)<br>
GCC version: (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04) 7.5.0<br>
Clang version: Could not collect<br>
CMake version: version 3.10.2<br>
Libc version: glibc-2.25</p>
<p>Python version: 3.6.9 (default, Jan 26 2021, 15:33:00)  [GCC 8.4.0] (64-bit runtime)<br>
Python platform: Linux-4.9.253-tegra-aarch64-with-Ubuntu-18.04-bionic<br>
Is CUDA available: True<br>
CUDA runtime version: 10.2.300<br>
GPU models and configuration: Could not collect<br>
Nvidia driver version: Could not collect<br>
cuDNN version: Probably one of the following:<br>
/usr/lib/aarch64-linux-gnu/libcudnn.so.8.2.1<br>
/usr/lib/aarch64-linux-gnu/libcudnn_adv_infer.so.8.2.1<br>
/usr/lib/aarch64-linux-gnu/libcudnn_adv_train.so.8.2.1<br>
/usr/lib/aarch64-linux-gnu/libcudnn_cnn_infer.so.8.2.1<br>
/usr/lib/aarch64-linux-gnu/libcudnn_cnn_train.so.8.2.1<br>
/usr/lib/aarch64-linux-gnu/libcudnn_ops_infer.so.8.2.1<br>
/usr/lib/aarch64-linux-gnu/libcudnn_ops_train.so.8.2.1<br>
HIP runtime version: N/A<br>
MIOpen runtime version: N/A</p>
<p>Versions of relevant libraries:<br>
[pip3] mypy-extensions==0.4.3<br>
[pip3] numpy==1.19.4<br>
[pip3] torch==1.8.0<br>
[pip3] torchaudio==0.10.0<br>
[pip3] torchvision==0.10.0<br>
[conda] Could not collect</p>