Trying to backward through the graph a second time even when create graph and retain graph is true

username: Sherine_Brahma
cooked: <p>Hi, I am trying to implement WGAN-GP (in a conditional probability setting). When I use inplace=True in the ReLU activation layers, I get the error “RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.” during my Critic training.</p>
<p>I have come across a similar post here in <a href="https://discuss.pytorch.org/t/freeing-buffer-strange-behavior/31955" class="inline-onebox">Freeing buffer strange behavior</a> where replacing the inplace=False in the ReLU activation solved the problem. It “solved” my problem too but my model became very very slow and the model output values dropped drastically (I do not think they are correct).</p>
<p>At the time when the aforementioned post was asked, the poster was using pytorch 0.4.1. But I am using pytorch 1.1.0. Strange thing is this exact code was working fine for a few days and then this error started popping up. Could I please have some suggestions about what I can do or any leads to what might trigger this behaviour? Please let me know if I need to provide more information.</p>
<p>This is my critic training code-</p>
<h1>def train_disc_net(bx, by, gen_net, disc_net, disc_net_optmzr):</h1>
<pre><code># Zeroing out Gradients
disc_net_optmzr.zero_grad()  
gen_net.zero_grad()     
disc_net.zero_grad()

# Reset requires_grad
for p in disc_net.parameters():
    p.requires_grad = True
	
# Training sign convention
one = torch.ones(bx.shape[0],1).cuda()
neg_one = -1*torch.ones(bx.shape[0],1).cuda()

# True data
dval_true = disc_net(by, bx)
dval_true.backward(neg_one)

# Generated data
by_gen = gen_net(bx)  # Generated data
dval_gen = disc_net(by_gen.detach(), bx)
dval_gen.backward(one)

# Wasserstein distance
was_dist = dval_true.mean() - dval_gen.mean()

Get drift regularization
d_drift_reg = dval_true**2 + dval_gen**2
d_drift_reg = 10e-9 * d_drift_reg.mean()
d_drift_reg.backward(one)

# Train with gradient regularization to ensure lipschitz 1 constraint
grad_reg = 10 * get_grad_reg(by, by_gen.detach(), bx, disc_net)
grad_reg.backward() &lt;----- Giving the error

# Objective function
d_cost = -was_dist + grad_reg #+ d_drift_reg

# Update the networks
disc_net_optmzr.step()

return d_cost, was_dist, grad_reg
</code></pre>
<p>This is my lipschitz constraining code-</p>
<h1>def get_grad_reg(by, by_gen, bx, disc_net):</h1>
<pre><code># Mixing real and fake inputs in a random fashion
epsilon = torch.FloatTensor(by.shape[0], 1, 1, 1, 1).uniform_(0.0, 1.0).cuda()
by_hat = epsilon * by + (1-epsilon) * by_gen
by_hat = torch.autograd.Variable(by_hat, requires_grad=True)

# Get output
d_hat = disc_net(by_hat, bx.detach())

# I concatenate the inputs of disc_net inside the disc_net function and I want to compute the gradients with respect to this concatenated input
d_in = disc_net.x_xyt 

# Getting gradient regularization
grad = torch.autograd.grad(outputs=d_hat, inputs=d_in, grad_outputs=torch.ones(d_hat.size()).cuda(), retain_graph=True, create_graph=True)[0]
grad_norm = torch.sqrt(1e-8+torch.sum(grad**2, dim=(1,2, 3, 4)))
one = torch.ones(grad_norm.shape).cuda()
grad_reg =  (grad_norm-one)**2
grad_reg =  grad_reg.mean()

return grad_reg</code></pre>

username: Sherine_Brahma
cooked: <p>I am sorry to bump up this question. I believe I have sufficiently searched for similar issues on the internet but I was unsucessful. It is currently posing a blockage for me. I share resources with other people and I wanted to know if an upgrade of software requirements is needed. I do not have a computer science background, so I do not know if I might unintentionally break others code in the server trying to upgrade (or if I cannot upgrade by myself, I will have to convince the server admin that this is my problem say).</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="Sherine_Brahma" data-post="1" data-topic="104624">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sherine_brahma/48/31736_2.png" class="avatar"> Sherine_Brahma:</div>
<blockquote>
<p>At the time when the aforementioned post was asked, the poster was using pytorch 0.4.1. But I am using pytorch 1.1.0.</p>
</blockquote>
</aside>
<p>I would recommend to update to the latest released version (<code>1.7.0</code>) as it would ship with bug fixes and new features.</p>
<aside class="quote no-group" data-username="Sherine_Brahma" data-post="1" data-topic="104624">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sherine_brahma/48/31736_2.png" class="avatar"> Sherine_Brahma:</div>
<blockquote>
<p>Strange thing is this exact code was working fine for a few days and then this error started popping up.</p>
</blockquote>
</aside>
<p>You might have accidentally changes something in the code, as it shouldn’t break “by itself” <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=12" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20"></p>
<aside class="quote no-group" data-username="Sherine_Brahma" data-post="1" data-topic="104624">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sherine_brahma/48/31736_2.png" class="avatar"> Sherine_Brahma:</div>
<blockquote>
<p>Could I please have some suggestions about what I can do or any leads to what might trigger this behaviour?</p>
</blockquote>
</aside>
<p>The inplace error is raised, since a tensor was overridden, which would be needed for the gradient calculation as described <a href="https://discuss.pytorch.org/t/gru-model-one-of-the-variables-needed-for-gradient-computation-has-been-modified-by-an-inplace-operation/57312/9">here</a>.</p>
<p>That being said, the model behavior and convergence shouldn’t change by using the inplace or out-of-place relu.</p>

username: Sherine_Brahma
cooked: <p>Thank you so much. I debugged the problem and the output was different because of the layer normalization layers I coincidently started using when I put inplace=false. When I found that there was a bug related to this before, I started attributing the change in output to this. Thank you for clearing that up.</p>