In-place operation error in backward call

aa08: <p>Hello,</p>
<p>I am trying to run the below code but I am getting an in place operation error for loss computation. Is there a way to resolve this without cloning the “target” tensor? Maybe a PyTorch function to directly subtract the specific elements from the x,y,z indices?</p>
<pre><code class="lang-auto">import torch

batch_size = 64
data = (torch.rand(batch_size, 50, 100) &lt; 0.01).float().to_sparse()
target = torch.rand(batch_size, 50, 100) - 0.5
target.requires_grad = True

x, y, z = data.indices()
losses = torch.exp(target)
losses[x, y, z] = losses[x,y,z] -  data.values() * target[x, y, z]
loss = losses.sum().backward()
</code></pre>
<p>Error:<br>
RuntimeError                              Traceback (most recent call last)<br>
Cell In[18], line 4<br>
2 losses = torch.exp(target)<br>
3 losses[x, y, z] = losses[x,y,z] - data.values() * target[x, y, z]<br>
----&gt; 4 loss = losses.sum().backward()</p>
<p>File ~/miniconda3/envs/sbtt-demo/lib/python3.9/site-packages/torch/_tensor.py:488, in Tensor.backward(self, gradient, retain_graph, create_graph, inputs)<br>
478 if has_torch_function_unary(self):<br>
479     return handle_torch_function(<br>
480         Tensor.backward,<br>
481         (self,),<br>
(…)<br>
486         inputs=inputs,<br>
487     )<br>
 → 488 torch.autograd.backward(<br>
489     self, gradient, retain_graph, create_graph, inputs=inputs<br>
490 )</p>
<p>File ~/miniconda3/envs/sbtt-demo/lib/python3.9/site-packages/torch/autograd/<strong>init</strong>.py:197, in backward(tensors, grad_tensors, retain_graph, create_graph, grad_variables, inputs)<br>
192     retain_graph = create_graph<br>
194 # The reason we repeat same the comment below is that<br>
195 # some Python versions print out the first line of a multi-line function<br>
196 # calls in the traceback and some print out the last line<br>
…<br>
 → 197 Variable.<em>execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass<br>
198     tensors, grad_tensors</em>, retain_graph, create_graph, inputs,<br>
199     allow_unreachable=True, accumulate_grad=True)</p>
<p>RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [64, 50, 100]], which is output 0 of ExpBackward0, is at version 1; expected version 0 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).</p>

KFrank: <p>Hi Aa!</p>
<aside class="quote no-group quote-modified" data-username="aa08" data-post="1" data-topic="171058" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/d2c977/48.png" class="avatar"> aa08:</div>
<blockquote>
<p>I am getting an in place operation error for loss computation. Is there a way to resolve this without cloning the “target” tensor?</p>
<pre><code class="lang-auto">losses = torch.exp(target)
losses[x, y, z] = losses[x,y,z] -  data.values() * target[x, y, z]
loss = losses.sum().backward()
</code></pre>
</blockquote>
</aside>
<p>When you assign into <code>losses</code> using indices you modify <code>losses</code> inplace,<br>
hence the error.  There is no way to avoid this without somehow creating<br>
a new tensor (not necessarily using <code>.clone()</code>).</p>
<p>The simplest fix might be <code>losses = torch.exp (target).clone()</code>.</p>
<p>In a little more detail, the gradient (derivative) of <code>exp (x)</code> is <code>exp (x)</code> itself.<br>
Therefore autograd keeps a reference to <code>exp (x)</code> during the forward pass<br>
so that it doesn’t have to recompute it during the backward pass.  If you<br>
modify <code>exp (x)</code> inplace, autograd can’t complete the backward pass (It<br>
isn’t willing to recompute <code>exp (x)</code>.), so it complains.</p>
<p>If you want to modify <code>losses</code>, you actually have to modify a copy of it.  In<br>
your use case <code>.clone()</code> is probably the most straightforward way to make<br>
that copy.</p>
<p>Best.</p>
<p>K. Frank</p>

aa08: <p>Hey Frank,<br>
Thanks for the response and explanation. It’s much clearer now.<br>
Is there any way I can do this without creating a copy of the target tensor? The only reason I was trying this out was to reduce the memory usage and creating a copy of the tensor would increase the memory usage.</p>

KFrank: <p>Hi Aa!</p>
<aside class="quote no-group" data-username="aa08" data-post="3" data-topic="171058" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/d2c977/48.png" class="avatar"> aa08:</div>
<blockquote>
<p>Is there any way I can do this without creating a copy of the target tensor?</p>
</blockquote>
</aside>
<p>First note that you are not “creating a copy of the target tensor.”  You are<br>
creating a copy of the <code>losses</code> tensor so that you can modify that copy<br>
inplace.</p>
<p>Having said that, in the context of the specific code you posted, you could<br>
trade computation for memory, as follows:</p>
<p>When you backpropagate through <code>exp (target)</code>, you need access<br>
to <code>exp (target)</code> because it is, itself, the gradient of <code>exp (target)</code>.<br>
<code>torch.exp()</code> does this in concert with autograd by saving a reference<br>
to <code>exp (target)</code> for use in the backward pass (and then complains<br>
when you modify it inplace).</p>
<p>At the cost of recomputing <code>exp (target)</code>, your forward pass could save<br>
a reference to <code>target</code> itself (and then recompute <code>exp (target)</code> during<br>
the backward pass).  In your specific case you are not freeing <code>target</code>, so<br>
holding on to (a reference to) <code>target</code> costs no additional memory, and<br>
you don’t modify <code>target</code>, so you don’t need a second copy of it.</p>
<p><code>torch.exp()</code> won’t do this for you, but you can write your own <a href="https://pytorch.org/docs/1.13/notes/extending.html#extending-torch-autograd" rel="noopener nofollow ugc">custom<br>
autograd function</a> that works this way.</p>
<p>However, I would recommend instead that you make your peace with the<br>
fact that neural networks are typically memory intensive, especially when<br>
using autograd to carry out the forward / backward pass process.  If you are,<br>
in fact, running out of memory, you could buy more memory, run a simpler,<br>
smaller model, or use a smaller batch size, depending on what you can make<br>
work for your use case.</p>
<p>Good luck.</p>
<p>K. Frank</p>