Sorting Triggers Versioning Errors

username: nimu
cooked: <p>Hello,</p>
<p>I would like to ask why am I getting following error message when sorting an array of tensors during enable_grad:</p>
<pre><code class="lang-auto">RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [192, 1000]], which is output 0 of TBackward, is at version 3; expected version 2 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).
</code></pre>
<p>I have confirmed that the error comes from sort operation. The sort operation is not in-place (Pythons sorted). I have also try to do argsort + gather, but the same error appears. Filter operation for example doesn’t trigger the versioning error. Could you guys give me some pointers on this?</p>
<p>Backstory: I am feeding triplets into my transformer network and I want to select n relevant triplets by sorting them by their loss.</p>
<p>The solution is I think to do two forward passes. One that selects triplets and the second one that constructs autograd graph. Am I right on this?</p>
<p>Thank you for your help. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>

username: KFrank
cooked: <p>Hi Nimu!</p>
<aside class="quote no-group quote-modified" data-username="nimu" data-post="1" data-topic="144998" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/n/58956e/48.png" class="avatar"> nimu:</div>
<blockquote>
<pre><code class="lang-auto">RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [192, 1000]], which is output 0 of TBackward, is at version 3; expected version 2 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).
</code></pre>
<p>The sort operation is not in-place (Pythons sorted). I have also try to do argsort + gather, but the same error appears. Filter operation for example doesn’t trigger the versioning error.</p>
</blockquote>
</aside>
<p>You have to find / deduce the inplace operation that is causing your<br>
problem and work around it.</p>
<p>You can – at the cost of additional storage – make a copy of the tensor<br>
in question which you can then modify inplace, e.g.:</p>
<pre data-code-wrap="python"><code class="lang-python">x = x.clone()   # assume that the original x will be used by backward()
x.copy_ (y)     # modify the cloned x inplace -- okay
...
z.backward()    # won't break
</code></pre>
<p>Some comments:</p>
<p>An inplace operation does not necessarily cause an error – it depends on<br>
whether the tensor being modified has been stored in the computation<br>
graph for use during <code>backward()</code>.  Some operations make use of their<br>
input tensors during <code>backward()</code>; others do not.</p>
<p>Consider:</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; import torch
&gt;&gt;&gt; print (torch.__version__)
1.10.2
&gt;&gt;&gt;
&gt;&gt;&gt; x0 = torch.ones (1, requires_grad = True)
&gt;&gt;&gt; x1 = 2 * x0
&gt;&gt;&gt; x2 = torch.exp (x1)   # backward() works after inplace operation
&gt;&gt;&gt; x1.copy_ (torch.ones (1))
tensor([1.], grad_fn=&lt;CopyBackwards&gt;)
&gt;&gt;&gt; x3 = x1 + x2
&gt;&gt;&gt; x3.backward()
&gt;&gt;&gt; x0.grad
tensor([14.7781])
&gt;&gt;&gt;
&gt;&gt;&gt; y0 = torch.ones (1, requires_grad = True)
&gt;&gt;&gt; y1 = 2 * y0
&gt;&gt;&gt; y2 = y1 ** 2   # inplace operation would break backward()
&gt;&gt;&gt; y1 = y1.clone()   # but clone() saves the day
&gt;&gt;&gt; y1.copy_ (torch.ones (1))
tensor([1.], grad_fn=&lt;CopyBackwards&gt;)
&gt;&gt;&gt; y3 = y1 + y2
&gt;&gt;&gt; y3.backward()
&gt;&gt;&gt; y0.grad
tensor([8.])
&gt;&gt;&gt;
&gt;&gt;&gt; z0 = torch.ones (1, requires_grad = True)
&gt;&gt;&gt; z1 = 2 * z0
&gt;&gt;&gt; z2 = z1 ** 2   # inplace operation breaks backward()
&gt;&gt;&gt; z1.copy_ (torch.ones (1))
tensor([1.], grad_fn=&lt;CopyBackwards&gt;)
&gt;&gt;&gt; z3 = z1 + z2
&gt;&gt;&gt; z3.backward()
Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in &lt;module&gt;
  File "&lt;path_to_pytorch_install&gt;\torch\_tensor.py", line 307, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "&lt;path_to_pytorch_install&gt;\torch\autograd\__init__.py", line 156, in backward
    allow_unreachable=True, accumulate_grad=True)  # allow_unreachable flag
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [1]], which is output 0 of struct torch::autograd::CopyBackwards, is at version 1; expected version 0 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).
&gt;&gt;&gt; z0.grad   # grad not populated because backward() broke
</code></pre>
<p>Best.</p>
<p>K. Frank</p>

username: nimu
cooked: <p>Thank you very much for help and the example. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>

username: nimu
cooked: <p>I couldn’t figure out the reason why Pythons sorted did trigger the error. However I was able to solve it by doing two forward passes.</p>