Inplace operation breaks computation graph

username: zerodecoder
cooked: <p>Hi! I am trying to create a model that makes predictions and updates the current state based on this. However, when I try keep track of the state and updating this, the computation graph seems to break. How do I address this issue/is there something I’m doing incorrectly in my code:</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
from copy import deepcopy
import math
from torch.nn import Linear, ModuleList, Sequential, ReLU
torch.manual_seed(0)
import numpy as np

class FeedForward():
    def __init__(self, input_size, hidden_size):
        self.input_size = input_size
        self.hidden_size  = hidden_size
        self.mlp = Sequential(
            torch.nn.Linear(self.input_size, self.hidden_size),
            torch.nn.ReLU(),
            torch.nn.Linear(self.hidden_size, 1),
            torch.nn.Sigmoid()
        )

    def forward(self, x):
        output = self.mlp(x)
        return output

model = FeedForward(8,32)

z = torch.tensor([math.pi/2.3], dtype=torch.float64)
curr_state = torch.tensor([[0, 1, z]]) #state variables, (for ex pos, vel, accel)
curr_state = curr_state.repeat(5,1) #batch size of 5, all with same curr_state

opt = torch.optim.Adam(model.mlp.parameters(), lr=1e-3)

next_pos = curr_state[:,0] #assume this is current_pos
for ep in range(10):
    with torch.autograd.set_detect_anomaly(True):
        print(ep)
        opt.zero_grad()
        x = torch.rand(5, 8) 
        y = torch.rand(5, 1)

        delta_pos = model.mlp(x)
        #next_pos = next_pos.detach() --- note adding this fixes this issue. why is this needed? will this solve the issue?
        next_pos = next_pos + delta_pos

        loss = torch.sum((next_pos-y)**2)
        print(loss)
        loss.backward(retain_graph=True) 
        opt.step()
</code></pre>
<p>Thanks!</p>

username: mariosasko
cooked: <p>The code you provided works fine on my machine. Make sure you have the newest version of PyTorch.</p>
<p>There is an error if you change <code>retain_graph</code> to <code>False</code> in the <code>backward()</code> call and in that case I think this thread will be helpful.</p><aside class="quote quote-modified" data-post="29" data-topic="6795">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alband/48/215_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/runtimeerror-trying-to-backward-through-the-graph-a-second-time-but-the-buffers-have-already-been-freed-specify-retain-graph-true-when-calling-backward-the-first-time/6795/29">RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time</a> 
  </div>
  <blockquote>
    In pytorch, every time you perform a computation with Variables, your create a graph, then if you call backward on the last Variable, it will traverse this graph to compute the gradients for everything in it (and delete the graph as it goes through it if retain_graph=False). So in your command line, you created a single graph, trying to backprop twice through it (withour retain_graph) so it will fail. 
If now inside your forward loop you do redo the forward computation, then the Variable on whic…
  </blockquote>
</aside>

<p>In PyTorch, after the <code>backward</code> call, it is not allowed to call the <code>backward</code> again on the same graph (or part of it)  unless it is a leaf node of that graph (because the backward pass frees the buffers stored by the <code>autograd</code> engine). <code>next_pos</code> is a part of the graph that is build in the first iteration. By detaching it (so it’s no longer a part of the graph), <code>next_pos</code> becomes a leaf node and everything works fine. Otherwise, the error is raised since <code>next_pos</code> was used in the previous <code>backward</code> call.</p>
<p>Here is a snippet that makes the erorr more obvious</p>
<pre><code class="lang-auto"># dummy forward pass (the engine records it)
a = torch.tensor(10., requires_grad=True)
b = a * 3
c = b - 1.5

# now it works 
c.backward()

# this will work
# (a + b.detach()).backward()

# error
(a + b).backward()
</code></pre>