When do I use `create_graph` in autograd.grad()

username: klory
cooked: <p>Hi, Im trying to understand the code of <a href="https://github.com/caogang/wgan-gp/blob/ae47a185ed2e938c39cf3eb2f06b32dc1b6a2064/gan_mnist.py#L146" rel="noopener nofollow ugc">WGAN-gp</a>.<br>
<code>... create_graph=True, retain_graph=True,...</code><br>
However, I did not understand what the argument <code>create_graph</code> is doing?</p>
<p>I’ve checked the <a href="https://pytorch.org/docs/stable/autograd.html#torch.autograd.grad" rel="noopener nofollow ugc">doc</a> however it’s not clear for me. <strong>Where did it calculate the second-order gradient?</strong></p>
<blockquote>
<p>If  <code>True</code> , graph of the derivative will be constructed, allowing to compute higher order derivative products. Default:  <code>False</code> .</p>
</blockquote>
<p>When I change it to<br>
<code>... create_graph=True, retain_graph=True,...</code><br>
or<br>
<code>... create_graph=True, retain_graph=None,...</code><br>
the code still works, but the model generates same image (model collapse?)</p>

username: smth
cooked: <p>in WGAN-gp, you want to calculate the gradient wrt the norm of your gradient, because you want to optimize that your norm of gradient is constant (this is the Lipschitz constraint that you apply).</p>
<p>With <code>create_graph=True</code>, we are declaring that we want to do further operations on gradients, so that the autograd engine can create a backpropable graph for operations done on gradients.</p>
<p><code>retain_graph=True</code> declares that we will want to reuse the overall graph multiple times, so do not delete it after someone called <code>.backward()</code>. From looking at the code, we do not call <code>.backward()</code> on the same graph again, so <code>retain_graph=True</code> is not needed in this case.</p>
<p>When you changed <code>retain_graph=None</code>, a model mode collapse occuring is likely something that’s happening coincidentally, but the reason for the mode collapse is not because of changing <code>retain_graph</code> (from the 2 minutes I spend reading the code, I think so).</p>

username: klory
cooked: <p>Thanks!</p>
<p>I actually made a mistake in the original post, what I changed the code to is</p>
<p><code>... create_graph=False, retain_graph=True,...</code><br>
or<br>
<code>... create_graph=False, retain_graph=None,...</code><br>
the code still works, but the model generates same image (model collapse?)</p>

username: smth
cooked: <p>if <code>create_graph=False</code>, technically it should error out when you try to compute the gradient of gradient. It is a bit strange that it is silently ignoring it.</p>

username: klory
cooked: <p>I see… I made a small demo and yes you’re correct it errors out. Thanks</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/1/13d8964855f04bf806a7bce478adee928f74ff5e.png" data-download-href="https://discuss.pytorch.org/uploads/default/13d8964855f04bf806a7bce478adee928f74ff5e" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/1/13d8964855f04bf806a7bce478adee928f74ff5e_2_690x203.png" alt="image" data-base62-sha1="2Pz6Fmr7LxBtnWTGTiWbdsSw3fg" width="690" height="203" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/1/13d8964855f04bf806a7bce478adee928f74ff5e_2_690x203.png, https://discuss.pytorch.org/uploads/default/optimized/2X/1/13d8964855f04bf806a7bce478adee928f74ff5e_2_1035x304.png 1.5x, https://discuss.pytorch.org/uploads/default/original/2X/1/13d8964855f04bf806a7bce478adee928f74ff5e.png 2x" data-dominant-color="3A152E"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1039×307 40.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<pre><code class="lang-auto">import torch
from torch.autograd import grad

torch.manual_seed(0)
G = torch.nn.Linear(2,4)
D = torch.nn.Linear(4,1)

real = torch.ones(5,4, requires_grad=False)
z = torch.rand(5,2, requires_grad=False)


D.zero_grad()
y = D(real)
loss = y.mean()
loss.backward()
print(G.weight.grad)
print(D.weight.grad)
print()

fake = G(z)
y = D(fake)
loss = y.mean()
(-loss).backward()
print(G.weight.grad)
print(D.weight.grad)
print()

alpha = torch.rand(5,1)
alpha = alpha.expand(real.size())
interp = alpha*real + (1-alpha)*fake
y = D(interp)
gradients = grad(outputs=y, inputs=interp, grad_outputs=torch.ones(y.size()),
create_graph=False, retain_graph=None, only_inputs=True)[0]
gp = ((gradients.norm(dim=1)-1)**2).mean()
gp.backward()
print(G.weight.grad)
print(D.weight.grad)
print()
</code></pre>

username: ngoyal2707
cooked: <p>That’s interesting, I do think here <code>retain_graph=False</code> doesn’t work even if <code>create_graph=True</code> with normal discriminator loss i.e. <code>gradient_penalty - d_real_out + d_fake_out</code><br>
First of all, as per Pytorch docs, <code>retain_graph=None</code> defaults to <code>create_graph</code>.</p>
<p>So when we give following:</p>
<pre><code class="lang-auto">... create_graph=True, retain_graph=False,...
</code></pre>
<p>and use above test code with slight changes to only one line:</p>
<pre><code class="lang-auto">gp = ((gradients.norm(dim=1)-1)**2).mean() + y.mean()
</code></pre>
<p>it fails with:</p>
<pre><code class="lang-auto">RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.
</code></pre>
<p>So it does seem like <code>retain_graph=True</code> (or <code>None</code> if <code>create_graph=True</code>) is necessary for normal discriminator loss.</p>

username: JTiC
cooked: <p>Hi. If <code>create_graph=True</code>, does that mean you can get the previous parameters after <code>optimizer.step()</code>, by using backward the (-1) * gradient?</p>