RuntimeError: Trying to backward through the graph a second time at my case like BPTT

Qmin-Lee: <pre><code class="lang-auto">class RNN(nn.Module):
    def __init__(self, input_size, hidden_size, output_size):
        super(RNN, self).__init__()

        self.hidden_size = hidden_size

        self.i2h = nn.Linear(input_size, hidden_size)
        self.h2h = nn.Linear(hidden_size, hidden_size)
        self.h2o = nn.Linear(hidden_size, output_size)
        self.hidden = torch.zeros(1, self.hidden_size)
        self.softmax = nn.LogSoftmax(dim=1)

    def forward(self, input):
        self.hidden = F.tanh(self.i2h(input) + self.h2h(self.hidden))
        output = self.h2o(self.hidden)
        output = self.softmax(output)
        return output

rnn = RNN(n_letters, n_hidden, n_categories)
temp_layer = nn.Linear(n_categories*2, n_categories)

for iter in range(1, n_iters + 1):
    category, line, category_tensor, line_tensor = randomTrainingExample()
    rnn.zero_grad()
    outputs = []
    for i in range(0,4): # T = 4  for BPTT
        output = forward_step(line_tensor, i)
        outputs.append(output)
        
    result = torch.cat(outputs, dim=-1)
    result_logit = temp_layer(result)
    loss = criterion(result_logit, category_tensor)
    
    loss.backward()
    optimizer.step()
</code></pre>
<p>This is a simple RNN example I created to reproduce an issue I encountered in another task, mimicking BPTT. The behavior may slightly differ from a real RNN because I defined <code>self.hidden</code> in the init function to recreate my problem. For simplicity, I omitted the dataloader part.</p>
<p>The issue is that when I run it, I encounter the following error:</p>
<blockquote>
<p>RuntimeError: Trying to backward through the graph a second time (or directly access saved tensors after they have already been freed). Saved intermediate values of the graph are freed when you call .backward() or autograd.grad(). Specify retain_graph=True if you need to backward through the graph a second time or if you need to access saved tensors after calling backward.</p>
</blockquote>
<p>I suspect this might be because <code>self.hidden</code> is being overwritten inside the <code>forward</code> function. However, the code works fine for the first couple of iterations but throws the above error around the third iteration.</p>
<p>Also, I thought the computational graph gets re-initialized in every iteration, so I don’t understand why it’s trying to reference something again.</p>

ptrblck: <aside class="quote no-group" data-username="Qmin-Lee" data-post="1" data-topic="212294">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/qmin-lee/48/50344_2.png" class="avatar"> Qmin-Lee:</div>
<blockquote>
<p>Also, I thought the computational graph gets re-initialized in every iteration, so I don’t understand why it’s trying to reference something again.</p>
</blockquote>
</aside>
<p>That’s correct if no references are stored to the previous computation graph.<br>
However, in your example <code>self.hidden</code> appends to a computation graph in each forward pass and thus keeps it alive.<br>
You would thus need to <code>.detach()</code> the tensor (or re-initialize <code>self.hidden</code>) after the backward is done.</p>

Qmin-Lee: <p>Thanks for replying!</p>
<p>I’m still confusing the reason…</p>
<p>In the following line inside the <code>forward</code> function:</p>
<pre><code class="lang-auto">self.hidden = F.tanh(self.i2h(input) + self.h2h(self.hidden))
</code></pre>
<p>Is <code>self.hidden</code> being appended to the computational graph at this line?</p>
<p>Also, the layers like <code>self.h2o</code>, <code>self.i2h</code>, and <code>self.h2h</code> are repeatedly used at each iteration.<br>
But why aren’t they considered “alive” in the computational graph?</p>

ptrblck: <aside class="quote no-group" data-username="Qmin-Lee" data-post="3" data-topic="212294">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/qmin-lee/48/50344_2.png" class="avatar"> Qmin-Lee:</div>
<blockquote>
<p>Is <code>self.hidden</code> being appended to the computational graph at this line?</p>
</blockquote>
</aside>
<p>Yes, since you are assigning the output to itself:</p>
<pre data-code-wrap="python"><code class="lang-python">self.hidden = differentiable_operations(self.hidden)
</code></pre>
<aside class="quote no-group" data-username="Qmin-Lee" data-post="3" data-topic="212294">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/qmin-lee/48/50344_2.png" class="avatar"> Qmin-Lee:</div>
<blockquote>
<p>Also, the layers like <code>self.h2o</code>, <code>self.i2h</code>, and <code>self.h2h</code> are repeatedly used at each iteration.</p>
</blockquote>
</aside>
<p>Yes, but their result is not assigned to themselves (which also wouldn’t make sense assuming these are layers, not state tensors such as <code>self.hidden</code>).</p>
<aside class="quote no-group" data-username="Qmin-Lee" data-post="3" data-topic="212294">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/qmin-lee/48/50344_2.png" class="avatar"> Qmin-Lee:</div>
<blockquote>
<p>But why aren’t they considered “alive” in the computational graph?</p>
</blockquote>
</aside>
<p>Their parameters are used in the computation graph.</p>