Error running multiple models in Torch 1.7.1 but works in Torch 1.0

mancunian1792: <p>I am trying to implement the cluster GAN architecture in Pytorch. The following steps work in Pytorch 1.0 but not in torch 1.7.0+cu101</p>
<pre><code class="lang-auto">optimizer_ge = Adam(itertools.chain(encoder.parameters(), generator.parameters()) ....)
opt_disc = Adam(discriminator.parameters() .....) 
</code></pre>
<p>The generator and the encoder are updated together and the discriminator is updated separately.</p>
<p>The following is done for each batch of images</p>
<pre><code class="lang-auto">generator.train()
encoder.train()

generator.zero_grad()
encoder.zero_grad()
discriminator.zero_grad()

optimizer_ge.zero_grad()

fake_image = generator(random_z)
fake_op = discriminator(fake_image)
real_op = discriminator(real_image)
zn, zc, zc_idx = encoder(fake_image)

ge_loss = (Cross_entropy loss) + (Clustering_loss) 
ge_loss.backward(retain_graph=True)
optimizer_ge.step()

opt_disc.zero_grad()
# Compute vannila gan discriminator loss disc_loss using bce loss function
disc_loss.backward()
opt_disc.step()
</code></pre>
<p>The above code works fine in torch 1.0 but torch 1.7 throws the following error.</p>
<pre><code class="lang-auto">one of the variables needed for gradient computation has been modified by an inplace operation: 
[torch.cuda.FloatTensor [64, 1, 4, 4]] is at version 2; expected version 1 instead. 
Hint: enable anomaly detection to find the operation that failed to
 compute its gradient, with torch.autograd.set_detect_anomaly(True).
</code></pre>
<p>The error seems to be resolved when I do</p>
<pre><code class="lang-auto">fake_op = discriminator(fake_image.detach())
</code></pre>
<p>or</p>
<pre><code class="lang-auto">ge_loss.backward(retain_graph=True)
disc_loss.backward()
optimizer_ge.step()
opt_disc.step()
</code></pre>
<p>However, the results after doing the above changes aren’t matching up with the results of the code run in torch 1.0</p>
<p>Can someone help me in debugging this?<br>
Thanks.</p>

ptrblck: <p>Could you check, if you might be facing a similar issue as described <a href="https://discuss.pytorch.org/t/struggling-with-a-runtime-error-related-to-in-place-operations/102222/6">here</a>?</p>

mancunian1792: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a>. Thanks for the reply. I took a look at the thread.<br>
I actually want to implement the way you suggested in that thread but currently failing to do that.</p>
<pre><code class="lang-auto">opt1.zero_grad()
loss1.backward(retain_graph=True)
opt1.step()

opt2.zero_grad()
loss2.backward()
opt2.step()
</code></pre>
<p>The above fails in torch 1.7 but works in torch 1.0</p>

ptrblck: <p>If you call <code>loss2.backward</code> after <code>opt1.step()</code>, the parameters used to calculate <code>loss2</code> were already updated and thus <code>loss2</code> would be stale.<br>
The proper way would be to execute a new forward pass to compute <code>loss2</code> and call <code>loss2.backward()</code> afterwards.</p>

mancunian1792: <pre><code class="lang-auto">opt1.zero_grad()
opt2.zero_grad()
loss1.backward(retain_graph=True)
loss2.backward()
opt1.step()
opt2.step()
</code></pre>
<p>Does it need to be modified like this?</p>

ptrblck: <p>Yes, this approach should work fine.</p>

mancunian1792: <p>Thanks. Just to understand a few things better, Did something underlying with the way autograd works change between torch 1.0 and torch 1.7?</p>
<p>The PyTorch implementation of cluster GAN architecture (torch 1.0) uses the following way to update their networks.</p>
<pre><code class="lang-auto">opt1.zero_grad()
loss1.backward(retain_graph=True)
opt1.step()

opt2.zero_grad()
loss2.backward()
opt2.step()
</code></pre>
<p>The above doesn’t work in torch 1.7<br>
I am <strong>not able to reproduce the results</strong> using the following suggested change.</p>
<pre><code class="lang-auto">opt1.zero_grad()
opt2.zero_grad()
loss1.backward(retain_graph=True)
loss2.backward()
opt1.step()
opt2.step()
</code></pre>
<p>Do you have any insights as to why this could happen?</p>

ptrblck: <p>Yes, the inplace updates of parameters are raising an error now, if you are using stale gradients as described in the <a href="https://github.com/pytorch/pytorch/releases/tag/v1.5.0">1.5 release notes</a> (described in the <code>torch.optim optimizers changed to fix in-place checks for the changes made by the optimizer</code> section).</p>
<p>The reason is that the gradient computation would be incorrect. In your example you would calculate <code>loss1</code> and <code>loss2</code> using the model parameters in the initial state <code>s0</code>. <code>loss1.backward()</code> calculates the gradients and <code>opt1.step()</code> updates the parameters to state <code>s1</code>.<br>
<code>loss2.backward()</code> was computed using the model in state <code>s0</code> and would thus calculate the gradients of <code>loss2</code> w.r.t. parameters <code>s0</code>, while the model is already updated to <code>s1</code>. These gradients would thus be wrong and the error is raised.</p>

jsktt01: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a>, I have a similar but different problem. I want to calculate the <code>loss2</code> based on params <code>s1</code>, and get the gradients w.r.t <code>s0</code>, then update <code>s0</code> with the gradients. But the there is a problem when calculating the gradients, which was expressed in <a href="https://discuss.pytorch.org/t/meta-learning-gradient-computation-error-runtimeerror-one-of-the-variables-needed-for-gradient-computation-has-been-modified-by-an-inplace-operation/181454">the link</a>. Could you please give me some suggestions? Thank you.</p>

ptrblck: <p>I’ve answered in the linked topic.</p>