RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [1]] is at version 2; expected version 1 instead

username: banikr
cooked: <p>Error:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "/home/banikr/.config/JetBrains/PyCharm2022.1/scratches/scratch_8.py", line 124, in &lt;module&gt;
    loss_dec.backward(retain_graph=True)
  File "/home/banikr/miniconda3/envs/ims37/lib/python3.7/site-packages/torch/_tensor.py", line 396, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "/home/banikr/miniconda3/envs/ims37/lib/python3.7/site-packages/torch/autograd/__init__.py", line 175, in backward
    allow_unreachable=True, accumulate_grad=True)  # Calls into the C++ engine to run the backward pass
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [1]] is at version 2; expected version 1 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).
</code></pre>
<p>The error comes from:</p>
<pre><code class="lang-auto">for epoch in range(max_epochs):
    for batch_idx, data in tqdm(enumerate(train_loader), total=len(train_loader),
                    desc='Train epoch=%d' % epoch, ncols=100, leave=False):  # epoch will be updtd by train_epoch()
        x = Variable(data, requires_grad=False).float().to(device)
        x_avg = x_avg + torch.mean(x, axis=0)
        # +------------------------------+
        # |        generator loss        |
        # +------------------------------+
        x_hat, elbo_loss = net_g(x)
        x_hat_avg = x_hat_avg + torch.mean(x_hat, axis=0)
        # z = net_g.decoder()
        _, z_p, _, _ = net_g.encoder(x)
        x_p = net_g.decoder(z_p)
        # +----------------------------------+
        # |        discriminator loss        |
        # +----------------------------------+
        d = net_D(x)
        d_hat = net_D(x_hat)
        d_p = net_D(x_p)
        real_label = Variable(Tensor(x.size(0), 1).fill_(1.0), requires_grad=False).to(device)
        fake_label = Variable(Tensor(x.size(0), 1).fill_(0.0), requires_grad=False).to(device)
        loss_D_real = adversarial_loss(d, real_label)
        loss_D_fake = adversarial_loss(d_hat, fake_label)
        loss_D_prior = adversarial_loss(d_p, fake_label)
        loss_gan = loss_D_real + loss_D_fake + loss_D_prior
        # print(loss_gan)
        optimizer_D.zero_grad()
        loss_gan.backward(retain_graph=True)
        optimizer_D.step()
        # +----------------------------+
        # |        decoder loss        |
        # +----------------------------+
        rec_loss = ((net_D(x_hat) - net_D(x)) ** 2).mean()
        print(rec_loss)
        loss_dec = gamma * rec_loss - loss_gan # &lt;&lt;&lt;&lt; error here
        optimizer_d.zero_grad()
        loss_dec.backward(retain_graph=True)
        optimizer_d.step()
</code></pre>
<p>Some of the solutions to similar errors are changing the dropout layer <code>inplace</code>(which I am not using here).<br>
<code>adversarial_loss</code> is:</p>
<pre><code class="lang-auto">adversarial_loss = torch.nn.BCELoss().to(device)
</code></pre>
<p>Any help in troubleshooting is much appreciated.</p>

username: ptrblck
cooked: <p>Might be similar to:<br>
<a href="https://discuss.pytorch.org/t/getting-this-error-message-runtimeerror-one-of-the-variables-needed-for-gradient-computation-has-been-modified-by-an-inplace-operation/164470/2" class="inline-onebox">Getting this Error Message: "RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation" - #2 by ptrblck</a></p>

username: banikr
cooked: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a><br>
removing <code>retain_graph</code> generates the following error from the very same line:</p>
<pre><code class="lang-auto">loss_dec.backward()#retain_graph=True)
  File "/home/banikr/miniconda3/envs/ims37/lib/python3.7/site-packages/torch/_tensor.py", line 396, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "/home/banikr/miniconda3/envs/ims37/lib/python3.7/site-packages/torch/autograd/__init__.py", line 175, in backward
    allow_unreachable=True, accumulate_grad=True)  # Calls into the C++ engine to run the backward pass
RuntimeError: Trying to backward through the graph a second time (or directly access saved tensors after they have already been freed). Saved intermediate values of the graph are freed when you call .backward() or autograd.grad(). Specify retain_graph=True if you need to backward through the graph a second time or if you need to access saved tensors after calling backward.
</code></pre>
<p>Do you think using the <code>loss_gan</code> again after backpropagating it could be the reason?</p>

username: AlphaBetaGamma96
cooked: <p>Hi <a class="mention" href="/u/banikr">@banikr</a>,</p>
<p>Have you tried running your code within a <code>torch.autograd.set_detect_anomaly(True)</code> context manager? It might give a more detailed stacktrace.</p>
<p>Also, check your models aren’t using an in-place ReLU by ensuring <code>ReLU(inpalce=False)</code> etc.</p>

username: banikr
cooked: <p>Hi,<br>
I used the anomaly part as follow:</p>
<pre><code class="lang-auto">torch.autograd.set_detect_anomaly(True)
for epoch in range(max_epochs):
.
.
</code></pre>
<p>And added <code>inplace=False</code> in  <code>ReLU</code>.</p>
<p>The following error generates:</p>
<pre><code class="lang-auto">loss_D_prior = adversarial_loss(d_p, fake_label)
  File "/home/banikr/miniconda3/envs/ims37/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/banikr/miniconda3/envs/ims37/lib/python3.7/site-packages/torch/nn/modules/loss.py", line 613, in forward
    return F.binary_cross_entropy(input, target, weight=self.weight, reduction=self.reduction)
  File "/home/banikr/miniconda3/envs/ims37/lib/python3.7/site-packages/torch/nn/functional.py", line 3083, in binary_cross_entropy
    return torch._C._nn.binary_cross_entropy(input, target, weight, reduction_enum)
 (Triggered internally at  /opt/conda/conda-bld/pytorch_1659484809535/work/torch/csrc/autograd/python_anomaly_mode.cpp:102.)
  allow_unreachable=True, accumulate_grad=True)  # Calls into the C++ engine to run the backward pass
Traceback (most recent call last):
  File "/home/banikr/.config/JetBrains/PyCharm2022.1/scratches/scratch_8.py", line 124, in &lt;module&gt;
    loss_dec.backward()#retain_graph=True)
  File "/home/banikr/miniconda3/envs/ims37/lib/python3.7/site-packages/torch/_tensor.py", line 396, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "/home/banikr/miniconda3/envs/ims37/lib/python3.7/site-packages/torch/autograd/__init__.py", line 175, in backward
    allow_unreachable=True, accumulate_grad=True)  # Calls into the C++ engine to run the backward pass
RuntimeError: Trying to backward through the graph a second time (or directly access saved tensors after they have already been freed). Saved intermediate values of the graph are freed when you call .backward() or autograd.grad(). Specify retain_graph=True if you need to backward through the graph a second time or if you need to access saved tensors after calling backward.

</code></pre>

username: AlphaBetaGamma96
cooked: <aside class="quote no-group" data-username="banikr" data-post="1" data-topic="164474">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/banikr/48/54353_2.png" class="avatar"> banikr:</div>
<blockquote>
<pre><code class="lang-auto">        loss_dec.backward(retain_graph=True)
</code></pre>
</blockquote>
</aside>
<p>Try putting the <code>retain_graph=True</code> for the decoder, and re-run it (with the <code>torch.autograd.set_detect_anamoly()</code> context manager). It should find where the in-place error is occurring.</p>

username: soulitzer
cooked: <p>The in-place may be happening from the optimizer step, you should move it until after your perform the second backward (which doesn’t need retain_graph=True). Alternatively if you are able to modify the model itself, you can clone the parameters before using them in any operation. It is likely that you are directly using your parameters in an operation that saves those inputs for backward. If you perform in-place optimizer update and then backward again through that same graph, it would’ve produce incorrect gradients (which the error protects against).</p>
<p>(For more context, retain_graph=True is useful when you plan to backward through the same graph another time. This is true for the first time you execute backward, but not the second)</p>

username: banikr
cooked: <p>Hello <a class="mention" href="/u/alphabetagamma96">@AlphaBetaGamma96</a>,<br>
This is the error I am getting following your advice:</p>
<pre><code class="lang-auto">torch.autograd.set_detect_anomaly(True)
for epoch in range(max_epochs):
    epoch_loss = 0
    ...
</code></pre>
<pre><code class="lang-auto">loss_D_prior = adversarial_loss(d_p, fake_label)
  File "/home/banikr/miniconda3/envs/ims37/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/banikr/miniconda3/envs/ims37/lib/python3.7/site-packages/torch/nn/modules/loss.py", line 613, in forward
    return F.binary_cross_entropy(input, target, weight=self.weight, reduction=self.reduction)
  File "/home/banikr/miniconda3/envs/ims37/lib/python3.7/site-packages/torch/nn/functional.py", line 3083, in binary_cross_entropy
    return torch._C._nn.binary_cross_entropy(input, target, weight, reduction_enum)
 (Triggered internally at  /opt/conda/conda-bld/pytorch_1659484809535/work/torch/csrc/autograd/python_anomaly_mode.cpp:102.)
  allow_unreachable=True, accumulate_grad=True)  # Calls into the C++ engine to run the backward pass
Traceback (most recent call last):
  File "/home/banikr/.config/JetBrains/PyCharm2022.1/scratches/scratch_8.py", line 125, in &lt;module&gt;
    loss_dec.backward(retain_graph=True)
  File "/home/banikr/miniconda3/envs/ims37/lib/python3.7/site-packages/torch/_tensor.py", line 396, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "/home/banikr/miniconda3/envs/ims37/lib/python3.7/site-packages/torch/autograd/__init__.py", line 175, in backward
    allow_unreachable=True, accumulate_grad=True)  # Calls into the C++ engine to run the backward pass
RuntimeError: Trying to backward through the graph a second time (or directly access saved tensors after they have already been freed). Saved intermediate values of the graph are freed when you call .backward() or autograd.grad(). Specify retain_graph=True if you need to backward through the graph a second time or if you need to access saved tensors after calling backward.

</code></pre>

username: banikr
cooked: <p><a class="mention" href="/u/soulitzer">@soulitzer</a>,<br>
I thought <code>optimizer_D</code> step and <code>optimizer_d</code> are optimizing different networks. Though they share the <code>loss_gan</code>.<br>
I took the first optimizer step after optimizer D but still got the errors.</p>
<p>Could you give me more hints with some demo code on how to clone parameters?</p>

username: soulitzer
cooked: <blockquote>
<p>I thought <code>optimizer_D</code> step and <code>optimizer_d</code> are optimizing different networks. Though they share the <code>loss_gan</code> .</p>
</blockquote>
<p>Yes because the loss-gan part is shared, parameters (which are saved for backward) modified in-place by the first optimizer step will be used in the second backward. I’m not sure your code includes the part where the optimizers are defined (I doubt that that is relevant though)</p>
<blockquote>
<p>Could you give me more hints with some demo code on how to clone parameters?</p>
</blockquote>
<p>There’s a way to apply this clone automatically to any model: <a href="https://discuss.pytorch.org/t/runtimeerror-one-of-the-variables-needed-for-gradient-computation-has-been-modified-by-an-inplace-operation-meta-learning/161783/12" class="inline-onebox">RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation (Meta Learning) - #12 by soulitzer</a></p>
<blockquote>
<p>I took the first optimizer step after optimizer D but still got the errors.</p>
</blockquote>
<p>Even though the cloning should fix this issue, I’d suggest looking into this error more to see if we can avoid the extra overhead of cloning. Was this the same error? Could you post the stack trace. (Or if possible, post a short runnable snippet to demonstrate the issue)</p>

username: banikr
cooked: <p><a class="mention" href="/u/soulitzer">@soulitzer</a><br>
I have shared demo code to this colab file: <a href="https://colab.research.google.com/drive/1EBS-Q99wx7-F-VqS8Z0WHCENArVFX3T2?usp=sharing" class="inline-onebox" rel="noopener nofollow ugc">Google Colab</a></p>
<p>It also generates the same error with demo data:</p>
<pre><code class="lang-auto">RuntimeError                              Traceback (most recent call last)
&lt;ipython-input-8-cdfe813ffd22&gt; in &lt;module&gt;
     98         loss_dec = gamma * rec_loss - loss_gan
     99         optimizer_d.zero_grad()
--&gt; 100         loss_dec.backward(retain_graph=True)
    101         optimizer_d.step()
    102         # +----------------------------+

1 frames
/usr/local/lib/python3.7/dist-packages/torch/autograd/__init__.py in backward(tensors, grad_tensors, retain_graph, create_graph, grad_variables, inputs)
    173     Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
    174         tensors, grad_tensors_, retain_graph, create_graph, inputs,
--&gt; 175         allow_unreachable=True, accumulate_grad=True)  # Calls into the C++ engine to run the backward pass
    176 
    177 def grad(

RuntimeError: Trying to backward through the graph a second time (or directly access saved tensors after they have already been freed). Saved intermediate values of the graph are freed when you call .backward() or autograd.grad(). Specify retain_graph=True if you need to backward through the graph a second time or if you need to access saved tensors after calling backward.
</code></pre>
<p>Will appreciate if you take a look.</p>

username: soulitzer
cooked: <p>Thanks for the repro, I’m taking a look.</p>

username: soulitzer
cooked: <p>Sorry for the delay. I’ve tried it now, and it looks like it runs without error on the latest nightly. Let me know if upgrading works for you.</p>

username: soulitzer
cooked: <aside class="quote no-group" data-username="soulitzer" data-post="7" data-topic="164474">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/s/839c29/48.png" class="avatar"> soulitzer:</div>
<blockquote>
<p>The in-place may be happening from the optimizer step, you should move it until after your perform the second backward</p>
</blockquote>
</aside>
<p>Ahhh I also just realized that I wrote the comment above, I didn’t see that you had three losses to backward in total, you’ll need to make sure all the optimizer updates are done at the very end.</p>
<p>So, modifying the code in the a way is probably the preferred solution here:</p>
<pre data-code-wrap="python"><code class="lang-python"># +----------------------------------+
# |        discriminator loss        |
# +----------------------------------+
d = net_D(x)
d_hat = net_D(x_hat)
d_p = net_D(x_p)
real_label = Variable(Tensor(x.size(0), 1).fill_(1.0), requires_grad=False).to(device)
fake_label = Variable(Tensor(x.size(0), 1).fill_(0.0), requires_grad=False).to(device)
loss_D_real = adversarial_loss(d, real_label)
loss_D_fake = adversarial_loss(d_hat, fake_label)
loss_D_prior = adversarial_loss(d_p, fake_label)
loss_gan = loss_D_real + loss_D_fake + loss_D_prior
optimizer_D.zero_grad()
loss_gan.backward(retain_graph=True)
  
# +----------------------------+
# |        decoder loss        |
# +----------------------------+
rec_loss = ((net_D(x_hat) - net_D(x)) ** 2).mean()
print(rec_loss)
loss_dec = gamma * rec_loss - loss_gan
optimizer_d.zero_grad()
loss_dec.backward(retain_graph=True)
  
# +----------------------------+
# |        encoder loss        |
# +----------------------------+
loss_enc = elbo_loss + rec_loss
optimizer_e.zero_grad()
loss_enc.backward()#retain_graph=True)
optimizer_D.step()
optimizer_d.step()
optimizer_e.step()
</code></pre>