Why cant I see .grad of an intermediate variable?

Kalamaya: <p>Hi! I am loving this framework…</p>
<p>Since Im a noob, I am probably not getting something, but I am wondering why I cant get the gradient of an intermediate variable with .grad? Here is an example of what I mean:</p>
<pre><code>xx = Variable(torch.randn(1,1), requires_grad = True)
yy = 3*xx
zz = yy**2
zz.backward()
xx.grad # This is ok
yy.grad # This gives 0! 
zz.grad # This should give 1! 
</code></pre>
<p>So I get the correct result for xx.grad, but why does yy.grad show 0, as does zz.grad? How can I get the yy.grad value in this case?</p>
<p>Thanks!</p>

smth: <p>Hi Kalamaya,</p>
<p>By default, gradients are only retained for leaf variables. non-leaf variables’ gradients are not retained to be inspected later. This was done by design, to save memory.</p>
<p>However, you can inspect and extract the gradients of the intermediate variables via <code>hooks</code>.<br>
You can register a function on a <code>Variable</code> that will be called when the backward of the variable is being processed.</p>
<p>More documentation on hooks is here: <a href="http://pytorch.org/docs/autograd.html#torch.autograd.Variable.register_hook">http://pytorch.org/docs/autograd.html#torch.autograd.Variable.register_hook</a></p>
<p>Here’s an example of calling the <code>print</code> function on the variable <code>yy</code> to print out it’s gradient (you can also define your own function that copies the gradient over else-where or modifies the gradient, for example.</p>
<pre><code class="lang-python">from __future__ import print_function
from torch.autograd import Variable
import torch

xx = Variable(torch.randn(1,1), requires_grad = True)
yy = 3*xx
zz = yy**2

yy.register_hook(print)
zz.backward()
</code></pre>
<p>Output:</p>
<pre><code class="lang-bash">Variable containing:
-3.2480
[torch.FloatTensor of size 1x1]
</code></pre>

Kalamaya: <p>Thanks <a class="mention" href="/u/smth">@smth</a></p>
<p>The only way I have been able to really extract the gradient however is via a global variable at the moment. This is because the function I pass in (apparently) only allows me to pass in one argument, and that is reserved for the yy.grad. What I mean is given here:</p>
<pre><code>yGrad = torch.zeros(1,1)
def extract(xVar):
	global yGrad
	yGrad = xVar	

xx = Variable(torch.randn(1,1), requires_grad = True)
yy = 3*xx
zz = yy**2

yy.register_hook(extract)

#### Run the backprop:
print (yGrad) # Shows 0.
zz.backward()
print (yGrad) # Show the correct dzdy
</code></pre>
<p>So here, I am able to extract the yy.grad, BUT, I can only do so with a global variable, which I would rather not do. Is there a simpler way? Many thanks.</p>

mrdrozdov: <p>Might help to take a look at how optimizers update parameters using the gradient. For instance, this line / block of code in SGD. <a href="https://github.com/pytorch/pytorch/blob/master/torch/optim/sgd.py#L45" rel="nofollow noopener">https://github.com/pytorch/pytorch/blob/master/torch/optim/sgd.py#L45</a></p>

apaszke: <p><a class="mention" href="/u/mrdrozdov">@mrdrozdov</a> I don’t think this applies to this use case, because optimizers always work with leaf Variables.</p>
<p><a class="mention" href="/u/kalamaya">@Kalamaya</a> Is there any reason why using a closure is not acceptable? If you can give me some more details about your use case, and why do you need the intermediate gradient, I could probably suggest some other way.</p>

Kalamaya: <p><a class="mention" href="/u/apaszke">@apaszke</a> Ok - I am not familiar with closures, (learning python still), but from the googling I just did, sounds like it is acceptable for my solution: How would we use closures in this case? The examples I saw all have nested functions and I am not seeing the connection still… many thanks!!</p>

apaszke: <p>You can think of a function that also keeps some additional variables from the outer scope. For example in here <code>hook</code> is a closure that remembers a <code>name</code> given to the outer function:</p>
<pre><code class="lang-python">grads = {}
def save_grad(name):
    def hook(grad):
        grads[name] = grad
    return hook

x = Variable(torch.randn(1,1), requires_grad=True)
y = 3*x
z = y**2

# In here, save_grad('y') returns a hook (a function) that keeps 'y' as name
y.register_hook(save_grad('y'))
z.register_hook(save_grad('z'))
z.backward()

print(grads['y'])
print(grads['z'])
</code></pre>

Kalamaya: <p>Many thanks! I will process this and let you know how it goes! <img src="//discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=5" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
<p>Just for my own knowledge, am I to understand that, given what I am trying to do, the only ways we have are i) global variables, and ii) closures?</p>
<p>Thanks again.</p>

apaszke: <p>I’d say that these are the most obvious ways, but you could probably come up with more sophisticated solutions too. As I said, the best one depends on the specific use case, and it’s hard to provide a one that fits all. I find using closures like above to be ok, others will find something else better.</p>

Kalamaya: <p>Thanks again. I will process it tonight and reply back here for my exact use case. Thanks again!</p>

EvanZ: <p>Aren’t the gradients of internal nodes necessary for doing backprop?</p>

fmassa: <p>Yes, they are, but as soon as they have been used and are not necessary anymore, they are freed to save memory</p>

vitchyr: <p>While I understand why this design decision was made, are there any plans to make it easier to save the gradients of intermediate variables? For example, it’d be nice if something like this was supported:</p>
<pre><code class="lang-auto">from torch.autograd import Variable
import torch

xx = Variable(torch.randn(1,1), requires_grad = True)
yy = 3*xx
yy.require_grad = True  # &lt;-- Override default behavior
zz = yy**2

zz.backward()

# do something with yy.grad
</code></pre>
<p>It seems like it’d be easier to let variables keep track of their own gradients rather than having to keep track of them with my own closures. Then if I want to analyze the gradients of my variables (leaf or not), I can do something like</p>
<pre><code class="lang-auto">do_something_with_data_and_grad_of(xx)
do_something_with_data_and_grad_of(yy)
</code></pre>
<p>Also, it might be useful to be able to set <code>require_gradients</code> for intermediate variables. For example, I might want to plot a histogram of intermediate variable gradients while not needing gradients for upstream variables. Right now, I’d have to set the<code>require_gradients</code> flag True to upstream nodes just to make sure that the gradients for this intermediate node are computed, but that seems a bit wasteful.</p>

chenjus: <p>Is it possible to get the gradients of a torch.nn.Linear module using the way you suggested or am I limited to capturing gradients by defining Variables? Would this work for convolutions or recurrent layers?</p>

miguelvr: <p>Is it possible to create a (torch.autograd) flag in order to save all the variable’s gradients?</p>

yusaku: <p>Looks like PyTorch 0.2.0 now has Variable.retain_grad(): <a href="http://pytorch.org/docs/master/autograd.html?highlight=retain_grad#torch.autograd.Variable.retain_grad" rel="nofollow noopener">http://pytorch.org/docs/master/autograd.html?highlight=retain_grad#torch.autograd.Variable.retain_grad</a></p>
<p>The above could now be done via<br>
<code>yy.retain_grad()</code></p>

blackyang: <p>Hi <a class="mention" href="/u/smth">@smth</a>, thanks for your reply. I have another question, suppose there are two heads on top of yy, how can we get grad_output from one of them, instead of the addition?</p>
<p>For example, how to get yy’s grad_output from zz1 part?</p>
<pre><code class="lang-auto">xx = Variable(torch.randn(1,1), requires_grad = True)
yy = 3*xx
zz1 = yy**2
zz2 = yy**2

yy.register_hook(print)
(zz1+zz2).backward()
</code></pre>

SimonW: <p>Ha, I recently did exactly this. Not sure if its the best way, but I did:</p>
<ol>
<li>detach <code>yy</code> before feeding to get <code>zz</code>s, e.g. <code>yyy = y.detach()</code>
</li>
<li>Manually call <code>autograd.grad</code> to get each of <code>zz</code>s grad w.r.t. <code>yyy</code>.</li>
<li>Save the one you want</li>
<li>call <code>yy.backward(grad_to_yyy_1 + grad_to_yyy_2)</code>.</li>
</ol>

blackyang: <p>Great, thanks! That’s also what in my mind, basically we need a dummy variable.</p>
<p>BTW, is there something like nn.Identity() in torch? I didn’t find it</p>

SimonW: <p>AFAIK, there is not. You can write one yourself though, although it won’t<br>
be helpful in this case. The important thing is to detach yy from the graph<br>
so you don’t backward through it to the part before it twice.</p>