How to print memory usage of each layer?

bosskwei: <p>Hi All,<br>
I am a beginner of pytorch, I tried to print memory cost (or variable shape/size) of each layer, not only the <code>model._params</code> and <code>model._buffers</code>, but also some mid-variable in <code>model.backward()</code>, how can I do it?<br>
Thanks</p>

anand.saha: <p>^bump^</p>
<p>I would also like to know the answer.</p>
<p>Specifically, how to get the memory footprint of a Tensor? sys.getsizeof() gives size of the metadata of the Tensor. I would like to know how much Memory is it actually occupying.</p>
<p>Thanks</p>

EKami: <p>Any updates on this? Did you find a way to do that <a class="mention" href="/u/anand.saha">@anand.saha</a> ?</p>

anand.saha: <p>Nope, <a class="mention" href="/u/ekami">@EKami</a>. Had moved on to other things …</p>

EKami: <p>okay np <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=5" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

jlquinn: <p>If you use master instead of 0.3.1, there is torch.cuda.memory_allocated().  This gives you all the allocated cuda memory, so you can instrument your code with it.  I found it very helpful.</p>

EKami: <p>That’s perfect thanks a lot!</p>

alessandro_mondin: <aside class="quote no-group" data-username="jlquinn" data-post="6" data-topic="7636">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/j/cab0a1/48.png" class="avatar"> jlquinn:</div>
<blockquote>
<p>torch.cuda.memory_allocated()</p>
</blockquote>
</aside>
<p>Hello,<br>
Anyone knows if there is any equivalent for CPU execution?<br>
I came across this picture in this great <a href="https://papers.nips.cc/paper/2021/file/1371bccec2447b5aa6d96d2a540fb401-Paper.pdf" rel="noopener nofollow ugc">paper</a> and I would like to replicate<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/e/9e178fb141447c5f719be0b716794b8dd8e693fd.png" data-download-href="https://discuss.pytorch.org/uploads/default/9e178fb141447c5f719be0b716794b8dd8e693fd" title="Screenshot 2023-01-12 at 15.31.26"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/9/e/9e178fb141447c5f719be0b716794b8dd8e693fd_2_690x162.png" alt="Screenshot 2023-01-12 at 15.31.26" data-base62-sha1="myxV3aFKsjYKwvZ4wJTCDvoMsfb" width="690" height="162" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/9/e/9e178fb141447c5f719be0b716794b8dd8e693fd_2_690x162.png, https://discuss.pytorch.org/uploads/default/optimized/3X/9/e/9e178fb141447c5f719be0b716794b8dd8e693fd_2_1035x243.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/9/e/9e178fb141447c5f719be0b716794b8dd8e693fd.png 2x" data-dominant-color="EAEBEC"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot 2023-01-12 at 15.31.26</span><span class="informations">1286×302 129 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

alessandro_mondin: <p>It could be done by simply printing sys.getsizeof() after each layer, right?</p>

rinkujadhav2013: <p>Have you guys looked at the profiler?<br>
<a href="https://pytorch.org/tutorials/recipes/recipes/profiler_recipe.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/tutorials/recipes/recipes/profiler_recipe.html</a></p>