Weight gradient in backward propagation

username: yzz
cooked: <p>Hi,</p>
<p>I know that .backward() can dynamically calculate the gradient. I wonder how we can obtain the weight’s gradient layer by layer during the .backward() calculation. Hope someone could help</p>
<ol>
<li>
<p>I know we can monitor the input and output of each layer during the .backward() by using .register_backward_hook(). I wonder if <strong>layer.register_backward_hook(module, grad_out, grad_in)</strong> is the right way to get the weights’ gradient of each layer.  The reason I am asking is that I don’t see the output difference between <strong>layer.register_backward_hook(module, grad_out, grad_in)</strong> and <strong>layer.register_backward_hook(module, input, output)</strong> in my own example.</p>
</li>
<li>
<p>If I am able to manually do the backward propagation layer by layer (see code below, followed by an example from <a href="https://discuss.pytorch.org/t/how-to-split-backward-process-wrt-each-layer-of-neural-network/7190/10" class="inline-onebox">How to split backward process wrt each layer of neural network?</a>). I wonder if this is correct that I get weight gradient of each layer via  <strong>self.layers[i].weight.grad</strong> during the backward execution (check the last line of backward() function).</p>
</li>
</ol>
<pre><code class="lang-auto">import torch.nn as nn
from torch.autograd import Variable

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.layers = nn.ModuleList([
            nn.Linear(10, 10),
            nn.Linear(10, 10),
            nn.Linear(10, 10),
            nn.Linear(10, 10),
        ])

    def forward(self, x):
        self.output = []
        self.input = []
        for layer in self.layers:
            # detach from previous history
            x = Variable(x.data, requires_grad=True)
            self.input.append(x)

            # compute output
            x = layer(x)

            # add to list of outputs
            self.output.append(x)
        return x

    def backward(self, g):
        for i, output in reversed(list(enumerate(self.output))):
            if i == (len(self.output) - 1):
                # for last node, use g
                output.backward(g)
                print(self.input[i].grad.shape)
            else:
               output.backward(self.input[i+1].grad.data)
               print(self.layers[i].weight.grad)



model = Net()
inp = Variable(torch.randn(4, 10))
output = model(inp)
gradients = torch.randn(*output.size())
model.backward(gradients)
</code></pre>

username: albanD
cooked: <p>Hi,</p>
<p>You should not rely on <code>regiter_backward_hook</code> as, as mentionned in the doc, they are mostly broken at the moment <img src="https://discuss.pytorch.org/images/emoji/apple/confused.png?v=9" title=":confused:" class="emoji" alt=":confused:"></p>
<p>Also for such apporaches, I would recommend using <code>autograd.grad</code> so that you can directly provide the Tensors you want the gradients for.</p>

username: yzz
cooked: <p>Hi,</p>
<p>Thanks for the response. I got a little bit confused on <code>torch.autograd.grad </code> and <code>torch.autograd.backward</code>. So for a given y = f(x), <code>torch.autograd.grad </code> returns dy/dx.  For <code>torch.autograd.backward</code>, based on my understanding, <code>torch.autograd.backward</code> uses the  chain rule to calculate the gradient. However, I am not very clear about what <code>torch.autograd.backward</code> returns ?  If possible, I wonder if you could provide some hints for the difference between these two.</p>
<p>Thank you in advance!</p>

username: albanD
cooked: <p>Hi,</p>
<p>The difference is that <code>out.backward()</code> will compute the gradient for all the leaf Tensors that were used to compute <code>out</code> and accumulate these gradients in their <code>.grad </code> field.<br>
<code>autograd.grad(out, inp)</code> will compute the gradient of out wrt inp and return that gradient directly.</p>
<p>One way to see it is that backward() is just a nice wrapper around autograd.grad to work nicely with torch.nn</p>
<pre><code class="lang-auto"># This is not the actual implementation
def backward(out, grad_out, *args):
  inp = find_all_leafs(out)
  grads = autograd.grad(out, inp, grad_out, *args)
  for i, g in zip(inp, grads):
    i.grad += g
</code></pre>

username: yzz
cooked: <p>Many thanks for the good explanation. Problem solved.</p>