How to modify gradients in-place?

Florian_Dietz: <p>I am working on an unusual research project and need to modify some gradients in-place.</p>
<p>This worked fine so far, but I just updated to a newer version of Pytorch, and now I am getting this error:</p>
<p>“RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [762, 5]], which is output 0 of TBackward, is at version 5; expected version 2 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!”</p>
<p>I get that this is an error for almost every usecase, but I actually want to do this on purpose.</p>
<p>Is there a way to turn this error off?</p>
<p>Alternatively, is there another way to modify gradients that is legal / preferred?</p>
<p>(My situation: I’m running a network where the architecture is not fixed but changes based on intermediate results. To backpropagate through this safely, I break the network into smaller parts, and manually retrieve and re-insert the gradients at the appropriate tensors.)</p>

ptrblck: <p>Would your workflow work with hooks (via <code>tensor.register_hook()</code>) to manipulate the gradients?<br>
If not, could you try to wrap the gradient manipulations into a <code>with torch.no_grad()</code> block, if that’s not already the case?</p>

Florian_Dietz: <p><code>tensor.register_hook()</code> would have worked, but I would have to refactor everything. It would take days, and I might lose some functionality, because the system I already built is more flexible for my purposes. (Hooks get executed immediately, but I build a prioritized queue of calculations that still need to be performed)</p>
<p>I’m not sure I understand how <code>torch.no_grad()</code> would help. I am performing normal gradient calculations through backpropagation. Then I apply modifications to the generated gradients.</p>

tom: <p>Note that</p>
<ol>
<li>the error message comes from modifying something in-place that has been used in the forward and is needed for the backward. You might check the backtrace mentioned here: (Note that it typically is in C++ stderr and not shown in Jupyter unless you use nightlies when it is in Python stderr.)</li>
</ol>
<blockquote>
<p>Hint: the backtrace further above shows the operation that failed to compute its gradient.</p>
</blockquote>
<ol start="2">
<li>If for the type of thing you are doing, I would probably not use <code>backward</code> for the intermediates but use <code>torch.autograd.grad</code>. Basically, <code>backward</code> is best thought of as existing for the very narrow case of “compute gradients then fed into the optimizer”.</li>
</ol>
<p>Best regards</p>
<p>Thomas</p>

Florian_Dietz: <p>That’s interesting. I didn’t know there was a difference between <code>torch.autograd.grad</code> and <code>torch.autograd.backward</code>. It looks like the first one is functional, while the second has side-effects. Is that correct? Is there anything you would recommend to read to understand this better? I’m worried I might break something if I make a switch at this point without fully understanding everything.</p>

tom: <p>Yeah, so basically, the difference is “only” functional vs. side-effects.<br>
For optimizers, you need to have <code>.backward</code> as .grad is what they consume.<br>
But for other purposes it would seem that <code>autograd.grad</code> makes is clear that the result is yours.<br>
Probably a very careful reading of the documentation (the Autograd Mechanics note and torch.autograd come to mind).</p>
<p>Best regards</p>
<p>Thomas</p>

Florian_Dietz: <p>Thanks! That’s good to know. Now I just need to figure out if that’s worth all the refactoring…</p>

Florian_Dietz: <p>Quick update: In the meantime, I have noticed that my code contained a bug. I tried to apply a loss to a calculation after some parameters used in it had already been modified by another calculation. This is what caused the error message. The question I asked is still valid, and the answers are still useful. But just in case somebody wonders, the reason for the error message was not the in-place modification of gradients.</p>