Difference between gradients from network.parameters() and register_backward_hook()?

chenjus: <p>What’s the difference between the <code>grad_inputs</code> from <code>register_backward_hook()</code> and the gradients from <code>[x.grad.data for x in network.parameters()]</code>?</p>
<pre><code>class Feedforward(nn.Module):
	def __init__(self, topology):
		super(Feedforward, self).__init__()
		self.input_layer  = nn.Linear(topology['features'], topology['hidden_dim'])
		self.hidden_layer = nn.Linear(topology['hidden_dim'], topology['hidden_dim'])
		self.output_layer = nn.Linear(topology['hidden_dim'], topology['output_dim'])
		self.num_hidden   = topology['hidden_layers']


	def forward(self, x):
		hidden = self.input_layer(x).clamp(min=0)

		for _ in range(self.num_hidden):
			hidden = self.hidden_layer(hidden).clamp(min=0)
			
		return self.output_layer(hidden)


   class Train(object):
	def __init__(self, topology, training):
		self.topology   = topology
		self.training   = training
		self.network    = Feedforward(topology)


	def extract_grads(self):
		return [p.grad.data.numpy()*2 for p in list(self.network.parameters())]


	def update(self, module, grad_in, grad_out):
		grads = self.extract_grads()
		grad_out = grads


	def train(self):
		dh = DataHandler(self.training['data'])

		losses = []
		valid_acc = []
		loss_fn = torch.nn.MSELoss(size_average=False)
		optimizer = torch.optim.Adam(self.network.parameters(), lr=self.training['lr'])

		# training loop
		for x in range(self.training['iterations']):
			batch = dh.get_batch(self.training['batch_size'])
			x = Variable(torch.from_numpy(batch[0]), requires_grad=False)
			y = Variable(torch.from_numpy(batch[1]), requires_grad=False)

			optimizer.zero_grad()
			cost_fn = nn.MSELoss()
			cost = cost_fn(self.network(x), y)
			
			print 'before'
			print list(self.network.parameters())

			self.network.register_backward_hook(self.update)

			cost.backward()

			print 'after'
			print list(self.network.parameters())

			optimizer.step()
			break</code></pre>

albanD: <p>Hi,<br>
I am quite confuse about your code, and not sure to understand what it is trying to do.</p>
<p><code>register_backward_hook()</code> allows you to specify a function that will be called just after the backward pass of a <code>nn.Module</code> has been done. It will provide you with the gradients used as parameter for the backward (the derivative of your loss wrt the output of your <code>Module</code>) and the gradients returned by the backward (the derivative of your loss wrt the input of your <code>Module</code>).</p>
<p><code>network.parameters()</code> will return all the <code>nn.Parameters</code> contained in the <code>nn.Module</code> (and its children). so <code>[x.grad.data for x in network.parameters()]</code> will create a list of all the gradients for each parameter in your model, namely the derivative of the loss wrt the learnable parameters of your Module.</p>
<p>I hope this helps.</p>

chenjus: <p>I guess my confusion is more of a fundamental understanding of gradients. Are the gradients obtained from <code>[x.grad.data for x in network.parameters()]</code> the same gradients used to update the parameters in gradient descent e.g. <code>theta = theta - lr*gradient</code>, where gradient is the derivative of the cost function w.r.t. the parameters in the network? Sounds like they’re something different - this point seems subtle to me so please correct any misunderstands I have - and that the gradients obtained from <code>register_backwards_pass(hook)</code> are exactly the gradients from <code>theta = theta - lr*gradient</code>, right?</p>
<p>Also say for whatever reason, if I want to manually implement dropout, which one should I use to control gradient updates, <code>register_backwards_pass(hook)</code> or <code>[x.grad.data for x in network.parameters()]</code>?<br>
Like if I want to directly replace in_grad from <code>register_backward_hook(hook)</code>, can I simply do something like:</p>
<pre><code>def hook(module, in_grad, out_grad):
   in_grad = my_new_gradient

nn.Linear.register_backward_hook(hook)</code></pre>

albanD: <p>Consider a layer that has an input <code>i</code>, and output <code>o</code> and some weights <code>w</code>. And suppose you have a loss function <code>L</code>.<br>
If you want to minimize your loss by gradient descent, you want to obtain <code>dL/dw</code> and then update <code>w</code> as <code>w = w - lr * dL/dw</code>.<br>
This <code>dL/dw</code> is computed during the backward pass and is stored in the <code>Variable</code>'s <code>.grad</code> field.<br>
That means that <code>[x.grad.data for x in network.parameters()]</code> will give you the list of all the <code>dL/dw</code> associated to each <code>w</code> in your network. If you want to do gradient descent by hand you can do:</p>
<pre><code class="lang-auto">for w in network.parameters():
    w.data -= lr * w.grad.data
</code></pre>
<p>Which is more or less what is done by sgd <a href="https://github.com/pytorch/pytorch/blob/b3589b04fd371cbaba7673cec538d3f37576d413/torch/optim/sgd.py">here</a>.</p>
<p>If you use a hook, you will get <code>dL/di</code> and <code>dL/do</code>. These are just intermediary computations. They are used to compute the <code>dL/dw</code> during the backward pass using the chain rule as we write for our layer: <code>dL/dw = dL/do * do/dw</code> where <code>dL/do</code> is given by the chain rule applied to the next layer, and <code>do/dw</code> is written by hand for each layer.<br>
The gradients passed to these hook does not correspond to any weight that you are trying to learn.</p>
<p>Does that make it clearer?<br>
You can also take a look at the <a href="https://github.com/pytorch/examples/blob/1c6d9d276f3a0c484226996ab7f9df4f90ce52f4/mnist/main.py#L81-L86">examples</a> to see how the optimiser should used in practice.</p>

chenjus: <p>That explanation was perfect. Thanks <img src="//discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=5" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

albb762: <p>I think through backward hook you could get gradients for (this layer’s input tensor, weight, bias) and the output tensor.</p>

Roshan_Santhosh: <p>As mentioned in the above comment, register_backward_hook gives you the gradient with respect to the input and output. If you want the gradients for the specific parameters (weights and bias), then you need to use the register_hook method.</p>
<p>Again, register_forward_hook and register_backward_hook are only available for modules while register_hook is available only for parameters</p>