CUDA suddenly always runs out of memory even with small models

clemo: <p>I’ve been building some models, some larger some smaller, but all of them could run on my laptop’s GPU (GeForce GTX 1650 Ti). I always ran them with a batch size of 32 but once try to increase it to 64 which led to my laptop basically freezing. I had to force it to shutdown. Ever since then, with even the smallest model and any batch size, I get a CUDA out of memory error.</p>
<p>I have done quite some googling and searched this forum a fair bit, and I assume I have some zombie process occupying a lot of memory. However, I cannot even localize it. The nvidia-smi command doesn’t show any processes. Btw I am on Windows, so I cannot use the “killall” or the “nvidia-smi --gpu-reset” commands.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/0/2078d2c1022a08b341de88299913177a8d71dfe3.png" data-download-href="https://discuss.pytorch.org/uploads/default/2078d2c1022a08b341de88299913177a8d71dfe3" title="Screenshot 2024-06-06 185517"><img src="https://discuss.pytorch.org/uploads/default/original/3X/2/0/2078d2c1022a08b341de88299913177a8d71dfe3.png" alt="Screenshot 2024-06-06 185517" data-base62-sha1="4Dg8MEF1wEHkHl7MRLGH7OxaU5d" width="690" height="337" data-dominant-color="151515"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot 2024-06-06 185517</span><span class="informations">812×397 9.03 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Via PowerShell, I have also inspected active processes using “Get-Process” but couldn’t find anything. That is, when Spyder (which I am using, but running my code via the command prompt leads to similar issues) is closed, there aren’t any Python-related processes (as far as I can tell). torch.cuda.memory_allocated() also indicates that 0 memory is allocated (on start up before) running a model.</p>

ptrblck: <aside class="quote no-group" data-username="clemo" data-post="1" data-topic="204183">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/clemo/48/65749_2.png" class="avatar"> clemo:</div>
<blockquote>
<p>I had to force it to shutdown. Ever since then, with even the smallest model and any batch size, I get a CUDA out of memory error.</p>
</blockquote>
</aside>
<aside class="quote no-group" data-username="clemo" data-post="1" data-topic="204183">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/clemo/48/65749_2.png" class="avatar"> clemo:</div>
<blockquote>
<p>nd I assume I have some zombie process occupying a lot of memory. However, I cannot even localize it.</p>
</blockquote>
</aside>
<p>After a restart no zombie process would be alive from the previous run which is also confirmed by checking <code>nvidia-smi</code> showing a 0MiB usage.</p>
<p>Are you able to use the GPU at all using any other CUDA application?</p>

clemo: <p>Oh boy, I guess I googled a bit too much and convinced myself that it must be a zombie process. All the posts about it I read seemed so similar to what I experienced. Turns out, I just created very large latent vectors by accident. Sorry about that.</p>
<p><a class="mention" href="/u/ptrblck">@ptrblck</a> Thank you for the quick response and in general all your replies on this forum. You have already helped me countless times.</p>