AssertionError: Torch not compiled with CUDA enabled, torch.cuda.is_available()= False

username: Nouf
cooked: <p>Hello everyone, I hope all are doing well<br>
I am writing this topic after trying all possible solutions for my issue.</p>
<p>I am trying to run my deep-learning model (building based on PyTorch) on the Jupyter notebook, however, I faced this error: AssertionError: Torch not compiled with CUDA enabled</p>
<p>I have installed Cuda toolkit 10.2 and cuDNN v8.7.0</p>
<p>Additional info:</p>
<pre><code class="lang-auto">(gputorch) C:\Users\dell&gt;python -m torch.utils.collect_env
Collecting environment information...
PyTorch version: 1.13.1
Is debug build: False
CUDA used to build PyTorch: Could not collect
ROCM used to build PyTorch: N/A

OS: ??Microsoft Windows 10 Pro
GCC version: Could not collect
Clang version: Could not collect
CMake version: Could not collect
Libc version: N/A

Python version: 3.9.13 (main, Aug 25 2022, 23:51:50) [MSC v.1916 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-10-10.0.19044-SP0
Is CUDA available: False
CUDA runtime version: 10.2.89
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: GPU 0: GeForce MX130
Nvidia driver version: 441.22
cuDNN version: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\bin\cudnn_ops_train64_8.dll
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.21.5
[pip3] numpydoc==1.4.0
[pip3] torch==1.13.1
[pip3] torchaudio==0.13.1+cu116
[pip3] torchvision==0.14.1
[conda] blas                      1.0                         mkl
[conda] cudatoolkit               10.2.89              h74a9793_1
[conda] mkl                       2021.4.0           haa95532_640
[conda] mkl-service               2.4.0           py310h2bbff1b_0
[conda] mkl_fft                   1.3.1           py310ha0764ea_0
[conda] mkl_random                1.2.2           py310h4ed8f06_0
[conda] numpy                     1.21.5          py310h6d2d95c_3
[conda] numpy-base                1.21.5          py310h206c741_3
[conda] numpydoc                  1.5.0           py310haa95532_0
[conda] pytorch                   1.12.1             py3.10_cpu_0    pytorch
[conda] pytorch-mutex             1.0                         cpu    pytorch
[conda] torchaudio                0.12.1                py310_cpu    pytorch
[conda] torchvision               0.13.1                py310_cpu    pytorch
</code></pre>
<pre><code class="lang-auto">(gputorch) C:\Users\dell&gt;nvidia-smi
Sun Jan 22 18:35:22 2023
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 441.22       Driver Version: 441.22       CUDA Version: 10.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name            TCC/WDDM | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce MX130      WDDM  | 00000000:01:00.0 Off |                  N/A |
| N/A   42C    P8    N/A /  N/A |     40MiB /  4096MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+```</code></pre>

username: ptrblck
cooked: <aside class="quote no-group" data-username="Nouf" data-post="1" data-topic="170827">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/nouf/48/62538_2.png" class="avatar"> Nouf:</div>
<blockquote>
<p>I have installed Cuda toolkit 10.2 and cuDNN v8.7.0</p>
</blockquote>
</aside>
<p>Note that your local CUDA toolkit (and cuDNN) won’t be used unless you build from source or a custom CUDA extension when you are installing the binaries as these ship with their own CUDA runtime, cuDNN, NCCL. etc.<br>
Also, CUDA 10.2 is officially not supported anymore (and source builds might start breaking soon or might already be broken).</p>
<p>Install the current stable or nightly PyTorch pip wheel or conda binary using <a href="https://pytorch.org/get-started/locally/">these install commands</a> and select the CUDA version you want.<br>
Then update your driver since <code>441.22</code> is too old for CUDA 11 and you would need <code>&gt;= 450.80.02</code> for minor version compatibility as described <a href="https://docs.nvidia.com/deploy/cuda-compatibility/index.html#default-to-minor-version">here</a>.</p>

username: Nouf
cooked: <p>Thank you <a class="mention" href="/u/ptrblck">@ptrblck</a> for your reply,</p>
<p>kindly can you explain more what you mean by " Note that your local CUDA toolkit (and cuDNN) won’t be used unless you build from source or a custom CUDA extension when you are installing the binaries as these ship with their own CUDA runtime, cuDNN, NCCL. etc. " ?</p>
<p>Sorry but it is my first time working with Pytorch I am not familiar with it</p>

username: ptrblck
cooked: <p>You would only need to install a new NVIDIA driver without the full CUDA toolkit, which ships with e.g. the CUDA compiler (<code>nvcc</code>), CUDA Math libraries (e.g. <code>cuBLAS</code>, <code>cuSOLVER</code>, etc.).<br>
The pip wheels and conda binaries which we are building will already ship directly with these required libraries or will install them as a dependency in your environment.</p>
<p>If you want to use a specific CUDA compiler or cuBLAS version etc. you would need to build PyTorch from source locally.<br>
Custom CUDA extensions as described <a href="https://pytorch.org/tutorials/advanced/cpp_extension.html">here</a> will also use the locally installed CUDA toolkit.</p>
<p>In the current nightly releases, <code>torch.compile</code> would also depend on your local <code>ptxas</code> binary to JIT compile the OpenAI/Triton kernels if I’m not mistaken.</p>

username: Michel777
cooked: <p>Hi,<br>
I have a similar issue:</p>
<blockquote>
<pre><code>    if not hasattr(torch._C, '_cuda_getDeviceCount'):
        raise AssertionError("Torch not compiled with CUDA enabled")
</code></pre>
</blockquote>
<p>raises the error<br>
Torch not compiled with CUDA enabled</p>
<p>torch: 2.0.0<br>
pytorch: 2.0.0<br>
cuda-python 12.1.0</p>
<p>A simple question: What should I do ?</p>
<p>Thanks in advance,</p>
<p>Michel</p>

username: ptrblck
cooked: <p>Install the PyTorch binary shipping with CUDA support using the install commands linked in my previous post.</p>

username: Michel777
cooked: <p>Hi <span class="mention">@ptrblck_de</span>,</p>
<p>thanks a lot, I did it - it works !</p>
<p>Warm regards,</p>
<p>Michel</p>

username: Yusuf_Kemal_Demir
cooked: <p>Hi,<br>
I have M2 max chip MacOS with 12 CPU, 38 GPU.<br>
for this code:"<span class="mention">@click.command</span>()<br>
<span class="mention">@click.option</span>(‘–device_type’, default=‘gpu’, help=‘device to run on, select gpu or cpu’)<br>
def main(device_type, ):<br>
# load the instructorEmbeddings<br>
if device_type in [‘cpu’, ‘CPU’]:<br>
device=‘cpu’<br>
else:<br>
device=‘cuda’<br>
",</p>
<p>I am getting this error :“Torch not compiled with CUDA enabled”. May I ask your help please?</p>
<p>Thanks</p>

username: eqy
cooked: <p>For M2, you would likely want <code>device='mps'</code> rather than <code>'cuda'</code>: <a href="https://pytorch.org/docs/stable/notes/mps.html" class="inline-onebox" rel="noopener nofollow ugc">MPS backend — PyTorch 2.0 documentation</a></p>

username: Yusuf_Kemal_Demir
cooked: <p>Thanks for the reply. but still getting the same issue.</p>
<p>Here is what I type based on your post.</p>
<p><span class="mention">@click.command</span>()<br>
<span class="mention">@click.option</span>(‘–device_type’, default=‘mps’, help=‘device to run on, select gpu or cpu’)<br>
def main(device_type, ):<br>
# load the instructorEmbeddings<br>
if device_type in [‘mps’, ‘MPS’]:<br>
device=‘mps’</p>
<p>What I meant to say is I have tried all sort of “mps” typing in the code I posted, but still getting this issue. I am new to MacOS and PyTorch.</p>

username: eqy
cooked: <p>How did you install PyTorch? was it according to e.g., <a href="https://developer.apple.com/metal/pytorch/" class="inline-onebox" rel="noopener nofollow ugc">Accelerated PyTorch training on Mac - Metal - Apple Developer</a> ? I would check that the environment is properly set up and that the install version is current.</p>

username: Yusuf_Kemal_Demir
cooked: <p>Appreciated. it worked.<br>
Regards</p>