RuntimeError: CUDA out of memory.Tried to allocate 20.00 MiB

ky_Pa: <p>The error is reported as follows. My batchsize is set to 1. What is the problem and how to modify it?<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/3/73e420ce3e70d2f2d9aa66120aa98e99499271a1.png" data-download-href="https://discuss.pytorch.org/uploads/default/73e420ce3e70d2f2d9aa66120aa98e99499271a1" title="微信图片_20200410100858"><img src="https://discuss.pytorch.org/uploads/default/original/3X/7/3/73e420ce3e70d2f2d9aa66120aa98e99499271a1.png" alt="微信图片_20200410100858" data-base62-sha1="gxdFCS157PPgLFXdVIF7MfNWsq5" width="690" height="160" data-dominant-color="31322C"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">微信图片_20200410100858</span><span class="informations">1896×441 31.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div> <div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/7/b7651e6509d0d559d39500fbcb557f4f35f60590.png" data-download-href="https://discuss.pytorch.org/uploads/default/b7651e6509d0d559d39500fbcb557f4f35f60590" title="微信图片_20200410101038"><img src="https://discuss.pytorch.org/uploads/default/original/3X/b/7/b7651e6509d0d559d39500fbcb557f4f35f60590.png" alt="微信图片_20200410101038" data-base62-sha1="qao1qitZXdkTybiEcrPnxdLFZ3q" width="690" height="150" data-dominant-color="322F29"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">微信图片_20200410101038</span><span class="informations">1053×230 12.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div> <img src="https://discuss.pytorch.org/uploads/default/original/3X/e/e/ee30e3b612f3177c07fb9a1a71201686f1f6e018.png" alt="微信图片_20200410101124" data-base62-sha1="xZ8nL8H5Ap2LUYGRlrkhoEcC1RK" width="657" height="139"> <div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/0/c03e64fcf77db31ec071dd0d1338bf696f0b1c25.png" data-download-href="https://discuss.pytorch.org/uploads/default/c03e64fcf77db31ec071dd0d1338bf696f0b1c25" title="微信图片_20200410101205"><img src="https://discuss.pytorch.org/uploads/default/original/3X/c/0/c03e64fcf77db31ec071dd0d1338bf696f0b1c25.png" alt="微信图片_20200410101205" data-base62-sha1="rqFlWmMJT67FlUHQM4xu8ZZuFtX" width="690" height="304" data-dominant-color="38332D"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">微信图片_20200410101205</span><span class="informations">741×327 20.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div> <img src="https://discuss.pytorch.org/uploads/default/original/3X/1/0/10fe0cbef0950755c7cedebbab7cf2df55cdda3a.png" alt="微信图片_20200410101238" data-base62-sha1="2qjWcjOU1VfVrxqtplrR48fPOzw" width="674" height="247"></p>

ptrblck: <p>4GB total memory might not be enough to train CycleGAN, but I haven’t verified it.<br>
Could you check, if your code is working on Colab?<br>
If I’m not mistaken, they are providing GPUs with more memory.</p>
<p>PS: As always, it’s better to post code instead of pictures. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

braindotai: <aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="76119">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>If I’m not mistaken, they are providing GPUs with more memory.</p>
</blockquote>
</aside>
<p>Yes they do, normally they provide you about 12 or 13 gigs, if you can train with that only then no problem.</p>
<p>Otherwise you can increase your ram there upto 25 gigs. But there’s no option to manually do that, you need to run below script -</p>
<pre data-code-wrap="python"><code class="lang-python">a = []
while(1):
    a.append(‘gaufbsiznaoejwuiaaoke be uJwbdiajajdhziajjdbsians’)
</code></pre>
<p>And then wait till session crash prompt occurs, you’ll be asked “If you would like to switch to a high-RAM runtime” click yes and there you go.</p>

ky_Pa: <p>But my computer’s RAM is 8GB, why is 4GB displayed and used there?</p>

ptrblck: <p>Your GPU has apparently 4GB of memory.<br>
If you give me the product name of your GPU, I can double check it.</p>

ky_Pa: <p>Is such that.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/4/b4d3909301ae2e5d20946789d56405ff7336122c.png" data-download-href="https://discuss.pytorch.org/uploads/default/b4d3909301ae2e5d20946789d56405ff7336122c" title="微信图片_20200410134847"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/b/4/b4d3909301ae2e5d20946789d56405ff7336122c_2_690x116.png" alt="微信图片_20200410134847" data-base62-sha1="pNFdDSG5EXsVjNXa3oJrum8Ay04" width="690" height="116" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/b/4/b4d3909301ae2e5d20946789d56405ff7336122c_2_690x116.png, https://discuss.pytorch.org/uploads/default/original/3X/b/4/b4d3909301ae2e5d20946789d56405ff7336122c.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/b/4/b4d3909301ae2e5d20946789d56405ff7336122c.png 2x" data-dominant-color="EFF0F1"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">微信图片_20200410134847</span><span class="informations">1004×169 31.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

KamWoh_Ng: <p>pytorch is using your “GPU” memory instead of your RAM memory, try reduce the batch size</p>

ky_Pa: <p>Thank you. Batchsize is already 1. It should be another reason.</p>

KamWoh_Ng: <p>Slightly reduce input image size, might help</p>

ptrblck: <p>The 940MX has a max. memory of 4GB.</p>

ky_Pa: <p>Thank you. So can I only run this program on the cloud? Is this error related to "With torch.no_grad () : " ?</p>

ptrblck: <p><code>with torch.no_grad()</code> will save some memory during evaluation and testing, but you won’t be able to train the model.<br>
4GB won’t be enough for a lot of common models, so you would either have to use a cloud runtime (e.g. Colab, Kaggle, or any cloud provider) or buy a bigger device.</p>

ky_Pa: <p>Thank you very much!</p>

ky_Pa: <p>Can I run down if I only use the CPU to run the program?</p>

ptrblck: <p>That might be possible, but also 8GB might not be enough for a full workload.<br>
If you keep the batch size low, it could fit without using the swap.<br>
The CPU run will be slower than running it on the GPU. However, if you run out of your system RAM and use the swap, you will see an even bigger performance hit.</p>