Unable to get pytorch working with CUDA 11.3

username: Michael_Lucy
cooked: <p>Hi,</p>
<p>When I install either pytorch 1.11 or the nightly version with CUDA 11.3, <code>torch.cuda.is_available()</code> returns false.  (I can’t use CUDA version 10.2 because I’m trying to use a 3090.)</p>
<p>Any idea what might be causing this?  I posted the output of <code>torch.utils.collect_env</code> below.</p>
<p>(Also, I’m trying to install on wsl2 if that’s relevant.)</p>
<pre><code class="lang-auto">Collecting environment information...
PyTorch version: 1.13.0.dev20220603+cu113
Is debug build: False
CUDA used to build PyTorch: 11.3
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Clang version: 10.0.0-4ubuntu1
CMake version: version 3.16.3
Libc version: glibc-2.31

Python version: 3.8.10 (default, Mar 15 2022, 12:22:08)  [GCC 9.4.0] (64-bit runtime)
Python platform: Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29
Is CUDA available: False
CUDA runtime version: 11.3.58
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3090
Nvidia driver version: 512.95
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.4.1
/usr/local/cuda-11.7/targets/x86_64-linux/lib/libcudnn.so.8.4.1
/usr/local/cuda-11.7/targets/x86_64-linux/lib/libcudnn_adv_infer.so.8.4.1
/usr/local/cuda-11.7/targets/x86_64-linux/lib/libcudnn_adv_train.so.8.4.1
/usr/local/cuda-11.7/targets/x86_64-linux/lib/libcudnn_cnn_infer.so.8.4.1
/usr/local/cuda-11.7/targets/x86_64-linux/lib/libcudnn_cnn_train.so.8.4.1
/usr/local/cuda-11.7/targets/x86_64-linux/lib/libcudnn_ops_infer.so.8.4.1
/usr/local/cuda-11.7/targets/x86_64-linux/lib/libcudnn_ops_train.so.8.4.1
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.22.4
[pip3] torch==1.13.0.dev20220603+cu113
[pip3] torchaudio==0.12.0.dev20220603+cu113
[pip3] torchvision==0.14.0.dev20220603+cu113
[conda] Could not collect
</code></pre>

username: ptrblck
cooked: <p>Are you able to run any CUDA applications (e.g. the CUDA samples) at all in your current setup?<br>
I assume when running <code>python -c "import torch; print(torch.randn(1).cuda())" you'll see an error complaining about the driver? If so, do you see any issues in </code>nvidia-smi<code>or</code>dmesg` as the driver might be in a “bad state”.<br>
E.g. it’s a common issue of the driver and Linux interaction to see this behavior after waking Linux from the “suspend” mode and in my setup running:</p>
<pre><code class="lang-auto">sudo rmmod nvidia_uvm
sudo modprobe nvidia_uvm
</code></pre>
<p>usually helps.</p>

username: Michael_Lucy
cooked: <p>I do seem to be able to run the CUDA examples:</p>
<pre><code class="lang-auto">mlucy@DESKTOP-MIBK3CH:~/cuda-samples/bin/x86_64/linux/release$ ./eigenvalues
Starting eigenvalues
GPU Device 0: "Ampere" with compute capability 8.6

Matrix size: 2048 x 2048
Precision: 0.000010
Iterations to be timed: 100
Result filename: 'eigenvalues.dat'
Gerschgorin interval: -2.894310 / 2.923303
Average time step 1: 0.920770 ms
Average time step 2, one intervals: 1.084130 ms
Average time step 2, mult intervals: 2.393340 ms
Average time TOTAL: 4.448770 ms
Test Succeeded!
</code></pre>
<p>When I run the command you gave, I get this, not a complaint about the driver:</p>
<pre><code class="lang-auto">mlucy@DESKTOP-MIBK3CH:~/cuda-samples/bin/x86_64/linux/release$ python -c "import torch; print(torch.randn(1).cuda())"
Traceback (most recent call last):
  File "&lt;string&gt;", line 1, in &lt;module&gt;
  File "/home/mlucy/.local/lib/python3.8/site-packages/torch/cuda/__init__.py", line 217, in _lazy_init
    torch._C._cuda_init()
RuntimeError: No CUDA GPUs are available
</code></pre>
<p>I followed the instructions at <a href="https://docs.nvidia.com/cuda/wsl-user-guide/index.html" class="inline-onebox" rel="noopener nofollow ugc">CUDA on WSL :: CUDA Toolkit Documentation</a> for setting up CUDA with wsl2, and they said to install a display driver on the Windows side but not the Linux side.</p>

username: Michael_Lucy
cooked: <p>Actually, I think I just found the fix.  The problem is that installing the NVIDIA driver on the Windows side put a new libcuda.so into the Linux side in an odd place.  If I add <code>LD_LIBRARY_PATH=/usr/lib/wsl/lib/</code> to force pytorch to use <code>/usr/lib/wsl/lib/libcuda.so</code> preferentially, everything runs fine.</p>
<p>Thanks for the help!  I was stuck on that for a long time, and putting me on to looking at the driver documentation again was what got me unstuck.</p>