Recursive call to backward()

aclifton314: <p><strong>SETUP</strong><br>
pytorch: 1.5.1<br>
huggingface transformers: 3.0.2<br>
python: 3.7.6<br>
OS: Pop!_OS 20.04 on VM</p>
<p>I’m experiencing some strange behavior with <code>backward()</code>.  Below is some pseudocode that shows what I am doing:</p>
<pre><code class="lang-python">ZERO = sys.float_info.min
ZERO_PT = torch.tensor(ZERO)

def sentence_loss(sent: str):
    vec = torch.tensor(sentence_vec(sent), dtype=torch.float, requires_grad=True) # remove punct, lower case, split on space, prepend "&lt;s&gt;", postpend "&lt;/s&gt;" start and stop tokens. Returns tensor of ints of vocab.
    last_idx = min(max_ngram, len(vec)) #max_ngram is an int

    probs = [max(ZERO_PT, pkatz(vec[0:i])) for i in range(2, last_idx + 1)] #pkatz is katz backoff probability and returns a tensor with grad function set.
    for i in range(1, len(vec) - last_idx + 1):
        j = i + last_idx
        probs.append(max(ZERO_PT, pkatz(vec[i:j])))

    probs = torch.stack(probs)
    log_probs = torch.log(probs)
    log_prob = torch.sum(log_probs)
    len_tensor = torch.tensor(len(vec), dtype=float, requires_grad=True)
    final_prob = torch.true_divide(-log_prob, len_tensor)
    return final_prob

class MyModel(MyTextGenModel):
    def __init__(self):
        ...

    def forward(self, input_str: str):
        gen_ids = self.generate(input_str, max_length=25)
        decoded_gen_sample = self.tokenizer.decode(gen_ids, skip_special_tokens=True)
        return decoded_gen_sample


model = MyModel()

for epoch in range(2):
    for i, input_str in enumerate(MyDataLoader, 0):
        output = model(input_str)
        print(output)
        loss = sentence_loss(output)
        loss.backward()

</code></pre>
<p>In words, <code>MyModel</code> inherits from a text generation model (namely GPT2 from HuggingFace Transformers). The <code>forward</code> method takes an input string from the training set and generates tokens from GPT2 until <code>max_length</code> is hit then returns the generated portion. That output is given to <code>sentence_loss()</code> to calculate the <a href="https://en.wikipedia.org/wiki/Katz%27s_back-off_model" rel="noopener nofollow ugc">katz backoff</a> loss with respect to an ngram model I have trained previously.</p>
<p><strong>PROBLEM</strong><br>
If I put a breakpoint at <code>print(output)</code> and one at <code>loss.backward()</code>, the training loop will get through 2 examples just fine. On the third example, the breakpoint at <code>loss.backward()</code> stops the code. Then when I resume the program from there, instead of hitting the breakpoint at <code>print(output)</code>, it comes back to <code>loss.backward()</code>. Resuming the program from that point results in the following error:</p>
<pre><code class="lang-python">RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.
</code></pre>
<p>Does anyone know why this might happen? The error makes sense because the computation graph has been cleared by the time it “recursively” comes back to <code>loss.backward()</code>, but I can’t figure out why it would do that recursion.</p>
<p><strong>THINGS I’VE CHECKED</strong></p>
<ul>
<li>
<p>I’ve checked the <code>loss</code> tensor and there isn’t anything that stands out to me as problematic (<code>grad_fn</code> is set, <code>requires_grad=True</code>, etc.).</p>
</li>
<li>
<p>Technically there are no learned parameters given to <code>sentence_loss()</code>. <code>sentence_vec()</code> returns a list of <code>ints</code> over the vocabulary the ngrams model was trained on to represent <code>sent</code>. I tried removing the <code>requires_grad</code> flags in <code>sentence_loss()</code> and <code>pkatz()</code> but setting it to <code>True</code> in <code>vec</code>. That produces the following error on the first example:</p>
</li>
</ul>
<pre><code class="lang-python">RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn
</code></pre>
<ul>
<li>Lastly, I tried setting <code>requires_grad=False</code> in <code>sentence_loss()</code>, <code>pkatz()</code>, and <code>vec</code>. That produces the following error on the first example:</li>
</ul>
<pre><code class="lang-python">RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn
</code></pre>
<p>If it would be helpful to see the entirety of the code, please send me a PM and I’ll be happy to provide it (it’s a little messy). Thank you in advance for your help!</p>

albanD: <p>Hi,</p>
<p>The <code>print(output)</code> and <code>loss.backward()</code> you mention above are the ones in the last lines of your code sample right?</p>
<p>If you don’t set any breakpoint, is the code working fine?</p>
<p>If the structure of your code is:</p>
<pre><code class="lang-auto">    for i, input_str in enumerate(MyDataLoader, 0):
        output = model(input_str)
        print(output)
        loss = sentence_loss(output)
        loss.backward()
</code></pre>
<p>I don’t see how the last line can be called twice without the print being called.<br>
Or you have other calls to <code>.backward()</code> in your code?</p>

aclifton314: <p>Hi <a class="mention" href="/u/alband">@albanD</a>. Without any breakpoints, the code still returns the same errors. To illustrate the issue further, I setup my code as follows:</p>
<pre><code class="lang-python">for i, input_str in enumerate(MyDataLoader, 0):
        output = model(input_str)
        print(output)
        loss = sentence_loss(output)
        loss.backward()
        print('pytorch is fantastic!')

</code></pre>
<p>and set another breakpoint at <code>print('pytorch is fantastic!')</code>. On the first two examples, that breakpoint is hit. On the third example, the breakpoint at <code>print('pytorch is fantastic!')</code> is not hit and <code>loss.backward()</code> gets called twice, producing the error on the second call.</p>
<p>The structure of my code does follow what is above, but I am using the Transformers library to handle the finetuning and GPT2. I’ve simplified it in this post because it’s a bit more complicated. The issue might be related to Transformers so I will definitely check with them.</p>
<p>Is there anything I could print out for <code>loss</code> before and/or after the <code>loss.backward()</code> call that might be useful? I cal also see into it’s attributes when the breakpoints hit.</p>

albanD: <blockquote>
<p>On the third example, the breakpoint at <code>print('pytorch is fantastic!')</code> is not hit and <code>loss.backward()</code></p>
</blockquote>
<p>Does the <code>print(output)</code> gets called again?<br>
I don’t think this is possible that this one line gets called again <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20">  Unless something goes horribly wrong with the python interpreter haha<br>
The autograd most likely gets called from some other place in the code no?</p>

aclifton314: <p>`print(output) does not get called again. On the third example, the breakpoint hits are as follows:</p>
<pre><code class="lang-python">print(output)
loss.backward()
loss.backward()
*exit with error*
</code></pre>
<p>I’ll go check with the Transformer folks to see what they think and mention your question about the autograd. It very well could be the case that it’s being called somewhere else in the library. I agree also that I don’t see how <code>loss.backward()</code> could be called twice, but not cause problems on the other examples.</p>