Keep getting the following error when trying to compute gradients with autograd "RuntimeError: Trying to backward through the graph a second time"

username: kirk86
cooked: <pre><code class="lang-auto">from torch.autograd import grad

def data(n=10000, d=2, eps=1):
    x = torch.randn(n, d) * eps
    y = x + torch.randn(n, d) * eps
    z = y + torch.randn(n, d)
    return torch.cat((x, z), dim=1), y.sum(dim=1, keepdim=True)

dummy_psi = torch.nn.Parameter(torch.ones(4, 1))
dummy_w = torch.nn.Parameter(torch.Tensor([1.0]))

opt = torch.optim.SGD([phi], lr=1e-3)
mse = torch.nn.MSELoss(reduction="none")

pairs = data(eps=0.1)

for i in range(50000):
    error = 0
    penalty_value = 0
    for x, y in [pairs]:
        error_e = mse(x @ dummy_psi * dummy_w, y)
        mean_error = error_e.mean()
        g = grad(mean_error, dummy_w)[0]
        penalty_value += g.sum()
        error += mean_error
    opt.zero_grad()
    (1e-3 * error + penalty_value).backward()
    opt.step()
    if i % 1000 == 0:
        print(dummy_psi)
</code></pre>

username: ptrblck
cooked: <p>Try to use <code>retain_graph=True</code> in <code>autograd.grad</code>.</p>

username: kirk86
cooked: <p>Thanks, should I be using <code>retain_graph=True</code> over <code>create_graph=True</code>?</p>
<p>My question which I cannot answer yet is why <code>.backward()</code> cannot backprop through the gradients? What exactly is it that makes it require <code>retain_graph=True</code> or <code>create_graph=True</code>?</p>

username: ptrblck
cooked: <p>Based on the provided code it seems <code>grad(mean_error, dummy_w)</code> might be freeing the intermediate tensors, which are still needed in <code>(error + penalty_value).backward()</code>.<br>
<code>retain_graph=True</code> will make sure to keep these intermediates alive.</p>

username: kirk86
cooked: <p>That’s very interesting to know. From reading the docs seems that I would have been unable to tell why that would happen. Are there any insights on why <code>grad</code> might freeze intermediate tensors?</p>

username: ptrblck
cooked: <p>Sorry for the typo. I meant the backward call <em>frees</em> (deletes) the intermediate tensors, not freezes. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"><br>
This is done to save memory, as the gradients are already calculated and these tensors are not needed anymore.<br>
However, if your use case needs these tensors again, you have to use <code>retain_graph=True</code>.</p>

username: kirk86
cooked: <p>Ah, now that explains a lot. Thanks for this insight.</p>
<p>Although it strikes as odd that <code>backward()</code> doesn’t understand that I’m still using the intermediate tensors because the values on which  I call <code>backward()</code> depend on those intermediate tensors.</p>
<p>For instance, I would have expected <code>backward()</code> to recognise that <code>penalty_value</code> depends on <code>g</code> which is the grad since I’m passing it explicitly in the call of <code>backward()</code>.</p>
<p>BTW, which is better recommended from these two options? Thanks!</p>
<ol>
<li><code>.backward(retain_graph=True)</code></li>
<li><code>grad(output, input, create_graph=True)</code></li>
</ol>

username: ptrblck
cooked: <aside class="quote no-group" data-username="kirk86" data-post="7" data-topic="73332">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/kirk86/48/151_2.png" class="avatar"> kirk86:</div>
<blockquote>
<p>For instance, I would have expected <code>backward()</code> to recognise that <code>penalty_value</code> depends on <code>g</code> which is the grad since I’m passing it explicitly in the call of <code>backward()</code> .</p>
</blockquote>
</aside>
<p><code>g = grad(mean_error, dummy_w)[0]</code> will already remove the intermediates, so that <code>backward</code> cannot do anything about it anymore.</p>
<aside class="quote no-group" data-username="kirk86" data-post="7" data-topic="73332">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/kirk86/48/151_2.png" class="avatar"> kirk86:</div>
<blockquote>
<p>BTW, which is better recommended from these two options?</p>
</blockquote>
</aside>
<p>It depends highly on your use case and I can’t give a general answer. <img src="https://discuss.pytorch.org/images/emoji/apple/confused.png?v=12" title=":confused:" class="emoji" alt=":confused:" loading="lazy" width="20" height="20"></p>

username: kirk86
cooked: <aside class="quote no-group" data-username="ptrblck" data-post="8" data-topic="73332">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>It depends highly on your use case and I can’t give a general answer.</p>
</blockquote>
</aside>
<p>Please correct me if I’m wrong but from the docs what I understand (unless it’s otherwise stated) is that we should avoid <code>retain_graph=True</code>?</p>
<p>Could you give a two cent summary on how 1. and 2. are dependent based on use case? What would be the major diff. in chosing one versus the other?</p>
<p>Thanks a lot!</p>

username: kirk86
cooked: <p>It seems like with <code>retain_graph=True</code> there’s not much change in the gradients compared to <code>create_graph=True</code> which is kind of strange?</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/a/7ac22ff31118f6772027e3b4522cac912d89d2b6.png" data-download-href="https://discuss.pytorch.org/uploads/default/7ac22ff31118f6772027e3b4522cac912d89d2b6" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/7/a/7ac22ff31118f6772027e3b4522cac912d89d2b6.png" alt="image" data-base62-sha1="hvYikWJEQ1f9UuCu89nMynZxlJk" width="690" height="153" data-dominant-color="DEDEDE"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">720×160 6.18 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: kirk86
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> Sorry to bother, do you know if there are any ways to capture memory leaks when using <code>grad()</code> from <code>autograd</code>.?</p>
<p>For instance in the MWE in my first post if I replace <code>mse(x @ dummy_psi * dummy_w)</code> with <code>mse(vgg_like(x) * dummy_w)</code> I get cuda error out of memory.</p>
<p>When I switch the whole example to cpu I can monitor the ram keep increasing on every iteration until it takes all my available ram. I’ve tracked down the issue to the <code>grad()</code> method because when I comment it out everything works fine but I don’t know why this is happening or where the leak commes from?</p>
<p>Also this leak happens regardless of using <code>retain-graph=True</code> or <code>create_graph=True</code>.</p>