Libcudnn_cnn_infer.so.8. Error: libnvrtc.so and torch._inductor.utils: [WARNING] not enough cuda cores to use max_autotune mode

FurkanGozukara: <p>i am trying to run dreambooth on runpod</p>
<p>unfortunately pytorch team removed xformers older version<br>
i cant believe how smart they are<br>
now we have to use torch 2<br>
however it is not working on runpod</p>
<p>here the errors and steps i tried to solve the problem</p>
<p>I have installed Torch 2 via this command on RunPod io instance</p>
<p><code>    pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118</code></p>
<p>Everything installed perfectly fine</p>
<p>With Torch 1 and Cuda 11.7, I was not getting any error but with Torch 2 the below error produced</p>
<p><code> Could not load library libcudnn_cnn_infer.so.8. Error: libnvrtc.so: cannot open shared object file: No such file or directory</code></p>
<p>How to fix?</p>
<p>It is using unix</p>
<p>On Windows same prodecure working very well</p>
<p>Using Automatic1111 web UI to use Stable Diffusion</p>
<p>this above i couldnt solve</p>
<p>therefore i have done the following things</p>
<p>apt update<br>
apt install sudo<br>
sudo apt install nvidia-cudnn<br>
sudo apt-get install python3-dev</p>
<p>after installing all above</p>
<p>now i have this warning and training never progress</p>
<p><code>Steps:   0%|                                                  | 0/170 [00:00&lt;?, ?it/s][2023-03-29 18:50:26,163] torch._inductor.utils: [WARNING] not enough cuda cores to use max_autotune mode</code></p>
<p>now when i run below python code i see everything looking good</p>
<pre><code class="lang-auto">import torch

# Check if CUDA is available
if torch.cuda.is_available():
    print("CUDA is available")
    # Display the current GPU name
    print("GPU name: ", torch.cuda.get_device_name(torch.cuda.current_device()))
else:
    print("CUDA is not available")

# Verify the PyTorch version
print("PyTorch version: ", torch.__version__)

import torch
print(torch.cuda.get_device_properties(0).multi_processor_count)
</code></pre>
<p>test.py result</p>
<pre><code class="lang-auto">CUDA is available
GPU name:  NVIDIA RTX A4500
PyTorch version:  2.0.0+cu118
56

</code></pre>
<p>it is able to generate images with 15.58it which is very fast</p>
<p>any help appreciated very much</p>

ptrblck: <p>The first issue is related to <a href="https://discuss.pytorch.org/t/torch-2-installed-could-not-load-library-libcudnn-cnn-infer-so-8-error-libnvrtc-so-cannot-open-shared-object-file-no-such-file-or-directory/176167/3">this topic</a>.</p>
<p>The Inductor warning is raised from <a href="https://github.com/pytorch/pytorch/blob/db8abde9b6c4735d18d4681a1f70a55ff0b09f5b/torch/_inductor/utils.py#L567-L573">here</a>, which indicates that <code>max_autotune</code> mode requires GPUs with a min. sm count of 80, while your GPU seems to have less.</p>

FurkanGozukara: <p>RTX A4500 has lesser than 80?</p>

ptrblck: <p>You can check it via <code>print(torch.cuda.get_device_properties(index).multi_processor_count)</code> where <code>index</code> corresponds to the device index in your system.<br>
Based on a quick search for the specs of the RTX A4500 it seems its SM count is 56.</p>

FurkanGozukara: <p>ok now the issue is, with pytorch 1.13 - xformers compiled for 1.13, everything works perfect<br>
but with torch 2, we are getting errors and cant train on the same card</p>
<p>what do yo think about it?</p>