I fine-tuned the PEGASUS model for abstractive Summarization on a virtual environment but the model had some problems so I created a new virtual environment to run the model again but the following error keeps popping

111495: <p><strong>RuntimeError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 2.00 GiB total capacity; 1.28 GiB already allocated; 4.55 MiB free; 1.28 GiB reserved in total by PyTorch)</strong><br>
<em>The Script that I run:</em> <a href="https://gist.github.com/jiahao87/50cec29725824da7ff6dd9314b53c4b3" class="inline-onebox" rel="noopener nofollow ugc">Pytorch script for fine-tuning Pegasus Large model · GitHub</a></p>
<p>Note: <em>The batch size in the script is 1</em> and I ran the following to see more details <strong>torch.cuda.memory_summary(device=‘cuda’, abbreviated=False)</strong> and this was the output:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/a/a/aa9e0f975688c6cdb83e9d8bc7fb7c57826a5d3b.png" data-download-href="https://discuss.pytorch.org/uploads/default/aa9e0f975688c6cdb83e9d8bc7fb7c57826a5d3b" title="Screenshot (13)"><img src="https://discuss.pytorch.org/uploads/default/original/3X/a/a/aa9e0f975688c6cdb83e9d8bc7fb7c57826a5d3b.png" alt="Screenshot (13)" data-base62-sha1="ollOsCXGs1EOYxDVeq9cJJLwYnF" width="690" height="268" data-dominant-color="EEEDED"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot (13)</span><span class="informations">845×329 20.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>Could you check, if the GPU is completely empty before running the script via <code>nvidia-smi</code>?<br>
2GB device memory is not a lot, but I assume you were able to run the script with a batch size of one before?</p>

111495: <p>Yes I was able to run the script once.<br>
I checked the memory before running the script but with these commands:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/8/98c7b22ca5bd1bd8fda68c55f486a02b72bb4a4b.png" data-download-href="https://discuss.pytorch.org/uploads/default/98c7b22ca5bd1bd8fda68c55f486a02b72bb4a4b" title="Screenshot (19)"><img src="https://discuss.pytorch.org/uploads/default/original/3X/9/8/98c7b22ca5bd1bd8fda68c55f486a02b72bb4a4b.png" alt="Screenshot (19)" data-base62-sha1="lNyqaWu2JNUkO9lWzdx1NFYQeOT" width="690" height="335" data-dominant-color="F1F1F1"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot (19)</span><span class="informations">870×423 21.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p><code>torch.cuda.memory_summary()</code> will report, what PyTorch is using and not if other processes might also use device memory, so you would have to use e.g. <code>nvidia-smi</code>.</p>

111495: <p>This is the result of running  nvidia-smi.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/4/e4c8e55ae9111c5910a22950e3aba2b3bdd66ae4.png" data-download-href="https://discuss.pytorch.org/uploads/default/e4c8e55ae9111c5910a22950e3aba2b3bdd66ae4" title="Screenshot (22)"><img src="https://discuss.pytorch.org/uploads/default/original/3X/e/4/e4c8e55ae9111c5910a22950e3aba2b3bdd66ae4.png" alt="Screenshot (22)" data-base62-sha1="wDVhpkltSVcEBRhoVj8YK7bfRKk" width="690" height="367" data-dominant-color="181818"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot (22)</span><span class="informations">747×398 10.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

omarfoq: <p>Hello <a class="mention" href="/u/111495">@111495</a>,</p>
<p>Tt shows that you are using 1.7GB.</p>
<p>I think you executed your script before launching<code>nvidia-smi</code>, do you confirm that ? If it’s not the case, kill the script that is consuming GPU and run your code.</p>

111495: <p>I found out that the problem was that my GPU memory is about 2GB while the recommended memory size is 16 GB.<br>
Upon this I subscribed to <em>Colab Pro</em>  as it provides 16 GB of GPU memory and enough disk space to run the script.<br>
<a href="https://github.com/google-research/pegasus/issues/180" rel="noopener nofollow ugc">Here</a> is the link to the issue on github which also contain some tips were mentioned by one of the contributors that might help others0</p>