How to determine memory used by a given computational graph?

pyscorcher: <p>Let’s say I have some computational graph - is it possible to determine the amount of memory it needs? I know there is <a href="https://pytorch.org/docs/stable/cuda.html#torch.cuda.memory_allocated" rel="nofollow noopener">https://pytorch.org/docs/stable/cuda.html#torch.cuda.memory_allocated</a> but that only allows you monitor the total memory used (and only if you use a GPU), so that is not very convenient if you have to monitor multiple graphs at once. I imagine it somehow must be possible to recurse down the graph to get the total size?</p>
<p>I created a small toy example:</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
net = torch.nn.Sequential(
    nn.Conv2d(1, 1, kernel_size=3, padding=1),
    nn.ReLU(),
    nn.Sigmoid(),
    nn.MaxPool2d(kernel_size=2, stride=1, padding=0),
    nn.Conv2d(1, 1, kernel_size=3, padding=1),
    nn.LeakyReLU()
)
optim = torch.optim.SGD(params=net.parameters(), lr=0.001)
x = torch.randn((1, 1, 64, 64))
y = net(x).sum() #how much memory?
y.backward() #how much memory?
</code></pre>

albanD: <p>Hi,</p>
<p>I’m afraid it’s not possible as some buffers are only saved on the cpp side and not accessible from python. You will need to check the global memory usage differences to see this <img src="https://discuss.pytorch.org/images/emoji/apple/confused.png?v=9" title=":confused:" class="emoji" alt=":confused:"></p>

pyscorcher: <p>I see, now it makes sense that I never found these buffers when I tried to explore the graph manually:) So I guess I’m gonna use hooks to read out the intermediate values. Thanks for your help!</p>

MasterScrat: <aside class="quote no-group" data-username="albanD" data-post="2" data-topic="62291">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alband/48/215_2.png" class="avatar"> albanD:</div>
<blockquote>
<p>I’m afraid it’s not possible as some buffers are only saved on the cpp side and not accessible from python.</p>
</blockquote>
</aside>
<p>Is that still the case?</p>
<p>If yes, would there be a way to persist the computational graph to disk? In that case, we could measure the size from there.</p>

albanD: <aside class="quote no-group" data-username="MasterScrat" data-post="4" data-topic="62291">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/masterscrat/48/16642_2.png" class="avatar"> MasterScrat:</div>
<blockquote>
<p>If yes, would there be a way to persist the computational graph to disk? In that case, we could measure the size from there.</p>
</blockquote>
</aside>
<p>No you cannot easily save that.</p>
<blockquote>
<p>Is that still the case?</p>
</blockquote>
<p>I’m afraid it is still the case right now.<br>
We are working on trying to provide a hacky way to access these attributes here that would allow you to check all the saved Tensors and their size though: <a href="https://github.com/pytorch/pytorch/pull/52451">https://github.com/pytorch/pytorch/pull/52451</a></p>