cuDNN version incompatibility

username: gram
cooked: <p>Hie, I built a simple CNN architecture when I try to run it, I get the error<br>
<code>RuntimeError: GET was unable to find an engine to execute this computation</code></p>
<p>Then I tried to find cudnn version using <code>torch.backends.cudnn.version()</code> which throwed me another error<br>
<code>RuntimeError: cuDNN version incompatibility: PyTorch was compiled  against (8, 5, 0) but found runtime version (8, 1, 0). PyTorch already comes bundled with cuDNN. One option to resolving this error is to ensure PyTorch can find the bundled cuDNN.Looks like your LD_LIBRARY_PATH contains incompatible version of cudnnPlease either remove it from the path or install cudnn (8, 5, 0)</code>.</p>
<p>But conda says my cudnn version is <code>8.5.0.96</code><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/3/e3c2997b3e493326a09c7d2ed6ebbf500f31c0a8.png" data-download-href="https://discuss.pytorch.org/uploads/default/e3c2997b3e493326a09c7d2ed6ebbf500f31c0a8" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/e/3/e3c2997b3e493326a09c7d2ed6ebbf500f31c0a8.png" alt="image" data-base62-sha1="wuRjqCqUo2qA61OEUERgY37eH5C" width="690" height="88" data-dominant-color="191A1A"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">833×107 4.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>A simple Linear Model works fine in GPU, but the CNN doesn’t. Below is the simple CNN I tried which throws me the error.</p>
<pre data-code-wrap="Python"><code class="lang-Python">layer =nn.Conv2d(in_channels=1,
                  out_channels=10,
                  kernel_size=3,
                  stride=1,
                  padding=1).to(device)

rand_image_tensor = torch.randn(size=(1, 1, 28, 28)).to(device)

layer(rand_image_tensor).shape
</code></pre>
<p>PyTorch Version - <code>2.0.1+cu117</code><br>
OS - WSL2<br>
Ubuntu Version in WSL2:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/4/a/4acfdeae35a825ecdb575ded9bc2616ad87a92fb.png" alt="image" data-base62-sha1="aFOGTG1WS55ZxTuwTQfwqHbbclZ" width="431" height="137"></p>

username: ptrblck
cooked: <p>As the error message explains an older locally installed cuDNN version is found and loaded, so you could:</p>
<ul>
<li>uninstall the locally installed cuDNN version</li>
<li>remove it from the <code>LD_LIBRARY_PATH</code>
</li>
<li>or preload the cuDNN version installed from PyPI.</li>
</ul>

username: gram
cooked: <p>I am not sure, but for WSL it was not recommended to install cudnn inside WSL so I had to install it in Windows(I’m using Windows 11). So I really don’t understand what you mean in the first point, if you could please explain the 2nd and 3rd point.</p>
<p>I am inside the .bashrc file, I see 3 lines<br>
<code>export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH export LD_LIBRARY_PATH=/usr/local/cuda/include:$LD_LIBRARY_PATH export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/extras/CUPTI/lib64</code><br>
are these the lines you’re talking about, if so, what should I do?</p>

username: gram
cooked: <p>Hie, <a class="mention" href="/u/ptrblck">@ptrblck</a></p>
<p>If I remove the line,</p>
<pre><code class="lang-auto">export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH export
</code></pre>
<p>from .bashrc file, the moment I run the Conv2d code the Jupyter kernel crashes.</p>

username: gram
cooked: <p>I simply uninstalled Cuda and re-installed it, that fixed it.</p>