How to understand autograd.grad function

username: kkew3
cooked: <p>Hi,</p>
<p><a href="https://pytorch.org/docs/stable/autograd.html#torch.autograd.grad" rel="noopener nofollow ugc">The grad function</a> can be used to calculate partial derivative according to <a href="https://discuss.pytorch.org/t/using-the-gradients-in-a-loss-function/10821/2">this post</a>. As per documentation,</p>
<blockquote>
<p>If  <code>create_graph=True</code> , graph of the derivative will be constructed, allowing to compute higher order derivative products.</p>
</blockquote>
<p>and <code>retain_graph</code> will be automatically set to <code>True</code>. However, I’m a bit confused how it works, or in other words, what should the computation graph look like by using this option.</p>
<p>As a tractable example, let <em>x</em> be a known vector, <em>f</em> be a <code>torch.nn.Module</code> instance <em>that returns a scalar</em>, and <em>θ</em> be the parameters of <em>f</em>:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/2X/4/4d8d31aff8a22fbe33150c017370b1df93c79f7c.png" alt="partial" data-base62-sha1="b43gSdLNdzuC2pr1txtW5k3vygs" width="544" height="103"></p>
<p>To implement this loss, which will be subsequently differentiated against <em>θ</em>, I think a good implementation should:</p>
<ul>
<li>not compute gradient with respect to <em>θ</em> when evaluating the partial gradient with respect to <em>x</em></li>
<li>have the two occurrences of <em>x</em> sharing the same data</li>
<li>contain only one graph for <em>f(x;θ)</em></li>
<li>treat the partial derivative in the second term as a new function with respect to <em>θ</em></li>
</ul>
<p>The computation graph I’d like to achieve should appear as: (legend: node without border → requires_grad=False; node in oval border → requires_grad=True; node in rectangle border → <code>torch.nn.Module</code> instance or similar entity in a computation graph; node in diamond border → the scalar output i.e. the loss)<br>
<img src="https://discuss.pytorch.org/uploads/default/original/2X/a/a4ed99fe1bd24d7690229edc9c6e58bcfeacd2be.png" alt="dot" data-base62-sha1="nx1mzzfwdpmzfcTJ3V6kc58GuYK" width="179" height="251"></p>
<p>To attain the desired computation graph I may write the following code (version: <code>PyTorch-0.4.1</code>):</p>
<pre data-code-wrap="python"><code class="lang-python"># `x` is the input such that x.requires_grad is False;
# `f` is the network
# As a specific example, assume that:
# x = torch.rand(3)
# f = torch.nn.Linear(3, 1)
import torch
from torch.autograd import grad
# build graph for f(x;theta) without derivative
loss1 = f(x)
# only_inputs=True: so that gradient of f(x) with respect to theta is not evaluated here
# create_graph=True: so that `dfdx` can be used as intermediate node in other graph
# retain_graph=False: so that we won't have a duplicate graph for f(x)
x_ = x.detach().requires_grad_()
dfdx, = grad(f(x_), x_, only_inputs=True, create_graph=True, retain_graph=False)
loss2 = torch.norm(dfdx, 1)
loss = loss1 + loss2
loss.backward()
</code></pre>
<p>This code snippet certainly compiles. However, it’s quite difficult for me to directly verify the result is correct. <strong>My questions here are:</strong></p>
<ul>
<li>How to visualize the computation graph created by <code>create_graph=True</code> in the above simple code?</li>
<li>Does <code>retain_graph=False</code> do what I intend to do as shown in comment? What if I change to <code>retain_graph=True</code>?</li>
<li>Is there a way to not compute <code>f(x)</code> twice while maintaining the same computation graph?</li>
</ul>
<p>Thanks for reading till here! Thanks in advance for your help!</p>

username: Yao_Xuan
cooked: <p>Hi,</p>
<p>I am coding on the similar los function. Did you finally figure out if this is correct? Thank you very much.</p>

username: Robin_Ranabhat
cooked: <p>Adding to this question,</p>
<p>I am hopelessly trying to figure out how the computational graph would look like with second derivatives involved. I understand how auto diff works in 1st order derivative case and can work my way with drawing the computational graph and calculate derivatives. But for higher-order derivatives, I simply cannot do so.</p>
<p>I can only find the code on how this can be done with autograd.grad but not an exact figurative answer. <a href="https://lucainiaoge.github.io/download/PyTorch-create_graph-is-true_Tutorial_and_Example.pdf" rel="noopener nofollow ugc">This</a> is the closest answer I could find, but still, didn’t quite get it.</p>
<p>And help would be really really appreciated</p>