"backward through the graph a second time" runtime error in LSTM

username: ding
cooked: <p>I am trying to build an LSTM model, and use the predicted result as the input to predict the result multiple timesteps away. And got the following error: <strong>RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.</strong></p>
<p>I checked this thread <a href="https://discuss.pytorch.org/t/runtimeerror-trying-to-backward-through-the-graph-a-second-time-but-the-buffers-have-already-been-freed-specify-retain-graph-true-when-calling-backward-the-first-time/6795">link</a>, and it does not work for my case.</p>
<p>The model is built based on the tutorial TRANSLATION WITH A SEQUENCE TO SEQUENCE NETWORK AND ATTENTION <a href="https://pytorch.org/tutorials/intermediate/seq2seq_translation_tutorial.html" rel="nofollow noopener">link</a>, with “Teacher forcing: Feed the target as the next input”.</p>
<p>Basically, the pseudocode is the following:</p>
<pre><code class="lang-auto">for i in range(epoch):
   for j in range(0, X_train.size(0), batch_size):

        x, y = getXandY(j)
        optimizer.zero_grad()
        loss = 0
        output = None

        for k in range(timeframe_to_be_predicted):
           x_input = update_x(output, x, k)
           output = model(x_input)
           loss += F.mse_loss(output, y)
     
        loss.backward()
        optimizer.step()

</code></pre>
<p>The main difference between my code and the code on the tutorial is that I am using minibatch and Adam as the optimizer (the code in the tutorial update the parameter by every each epoch).</p>
<p>As my understanding, after <code>optimizer.step()</code> is called, the parameter was updated and we can start a new computing graph for the next minibatch, so we do not need to set <code>retain_graph=True</code> in <code>loss.backward()</code>. But it shows the RuntimeError and because of the memory limitation, <code>loss.backward(retain_graph=True)</code> is not a good idea as well.</p>
<p>I also tried another method that collects the loss of each timestep. The code is the following. It does not work as well.</p>
<pre><code class="lang-auto">        loss = []
        output = None

        for k in range(timeframe_to_be_predicted):
           x_input = update_x(output, x, k)
           output = model(x_input)
           loss.append(F.mse_loss(output, y))
     
        for k in range(len(loss)):
            if k == timeframe_to_be_predicted - 1:
                loss[k].backward()
            else:
                loss[k].backward(retain_graph=True)

        optimizer.step()

</code></pre>
<p>Do you guys have any suggestion on this issue?</p>

username: ding
cooked: <p>I solved this issue. The problem is the following function. When we update the next step input (using the output of the current step), we need to update the tensor NOT in-place. Create a new tensor helps.</p>
<aside class="quote no-group" data-username="ding" data-post="1" data-topic="45242">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/d/97f17d/48.png" class="avatar"> ding:</div>
<blockquote>
<p>x_input = update_x(output, x, k)</p>
</blockquote>
</aside>

username: lda16
cooked: <p>Thanks for your answer. I also came across this issue. The key is to detach the grad info of the previous tensor before sending the values to the next step. The h and c tensors have been auto-graded once, and if you don’t detach the grad info, the next auto-grad will raise an error:</p>
<p>input_tensor = output_tensor.detach()<br>
output_tensor = model(input_data,input_tensor)</p>