No CUDA GPUs are available

NullPointer: <p>Hi, I’m trying to run a project within a conda env. I have a rtx 3070ti installed in my machine and it seems that the initialization function is causing issues in the program.</p>
<p>Error:<br>
<code>File "sTrain.py", line 37, in &lt;module&gt;                                                                   torch.cuda.set_device(gpuid)                                                                                       File "/home/user/miniconda3/envs/deeplearningenv/lib/python3.6/site-packages/torch/cuda/__init__.py", line 263, in set_device                                                                                                                  torch._C._cuda_setDevice(device)                                                                                   File "/home/user/miniconda3/envs/deeplearningenv/lib/python3.6/site-packages/torch/cuda/__init__.py", line 172, in _lazy_init                                                                                                                  torch._C._cuda_init()                                                                                            RuntimeError: No CUDA GPUs are available</code></p>
<p>Any guidance would be very helpful. I tried uninstalling cudatookit, pytorch, and torchvision and reinstalling with <code>conda install pytorch torchvision cudatoolkit=10.1</code> but I get the same error. I also the GPU appears in the device manager menu.</p>
<p>I’m not sure if this will aid in finding a solution but, trying <code>torch.cuda.is_available()</code> prints false.</p>
<p>Thank you in advance.</p>

ptrblck: <p>The error points to a missing NVIDIA driver, so you might want to reinstall it.<br>
While the PyTorch pip wheels and conda binaries ship with the CUDA runtime, an NVIDIA driver would still be needed to be able to execute workloads on the GPU.</p>

NullPointer: <p>I’ve been trying that but I’m not having any luck. My card says it’s on cuda 11.4. Is it possible to downgrade to 11.1?</p>

NullPointer: <p><img src="https://discuss.pytorch.org/uploads/default/original/3X/f/a/fa88ddea768094c307efe04f87751f5e7fc9cb68.png" alt="image" data-base62-sha1="zKkBpS8Hahluweegpjs01TKVAdW" width="458" height="87"></p>
<p>It seems the version is correct here, but when I run nvidia-smi I see the cuda version is 11.4</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/0/b/0b99af3e798563ffa63a88b2bcb01322ca56cdad.png" alt="image" data-base62-sha1="1ECwe6vQQaYz0RgQ41mpf5xofPD" width="601" height="18"></p>

ptrblck: <p>It seems you’ve installed a new driver while an older CUDA toolkit was still installed.<br>
Try to compile CUDA examples and execute them to make sure your setup is working fine.<br>
If that’s not the case, uninstall the driver and CUDA toolkit and reinstall it.</p>

SM2023: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a> , sorry I run my code for different times, now I change a parameter ( number of epoch) and wana to rerun it again. it gave m ethe error . I called the code as , why it happen suddenly? before it worked before! my system is linux.</p>
<p>UDA_VISIBLE_DEVICES=“1,2,3” python casesummary_resolution_GPT_Neo_GPU_V5-125M-Trainer_v22.py</p>
<p>in jupyter notebook it will be run with device=1 but if I call it as above it gave me he following error.<br>
and the error is :</p>
<p>torch._C._cuda_init()<br>
RuntimeError: No CUDA GPUs are available</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/a/4a2992063537357eec3c8627c1cbcec92bbdf43a.png" data-download-href="https://discuss.pytorch.org/uploads/default/4a2992063537357eec3c8627c1cbcec92bbdf43a" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/4/a/4a2992063537357eec3c8627c1cbcec92bbdf43a.png" alt="image" data-base62-sha1="aA4oC4cDE3ZFcjwEZMm7VOqyTSW" width="690" height="232" data-dominant-color="EDECEC"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">776×262 7.45 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>If your system only has a single valid GPU, you are masking it via the <code>CUDA_VISIBLE_DEVICES</code>. If that’s not the case, check if any drivers etc. were updated which might have broken your setup.</p>

SM2023: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> Many thanks for your help and reply. Sorry, do you have any idea about my post here?</p>
<p><a href="https://discuss.pytorch.org/t/help-cuda-error-out-of-memory/145937/17">Help CUDA error: out of memory - PyTorch Live - PyTorch Forums</a></p>
<p>The model can’t be fine tuned properly by using ddp and it generate nothing for me. I would appreciate if you can have a look. many thanks.</p>