How to calculate the 2nd derivative of the diagonal of the hessian matrix from a function?

Wei_Deng: <p>For example, I used his blog to try to get the 2nd derivative [Second order derivatives and inplace gradient “zeroing” ], but it turns out that the grd.grad information is None. Can anyone give me some suggestions?</p>
<p>import torch<br>
from torch import Tensor<br>
from torch.autograd import Variable<br>
from torch.autograd import grad<br>
from torch import nn</p>
<p># some toy data<br>
x = Variable(Tensor([4., 2.]), requires_grad=False)<br>
y = Variable(Tensor([1.]), requires_grad=False)</p>
<p># linear model and squared difference loss<br>
model = nn.Linear(2, 1)<br>
loss = torch.sum((y - model(x))**2)</p>
<p>optimizer = torch.optim.Adam(model.parameters(), lr=1e-2)</p>
<p># instead of using loss.backward(), use torch.autograd.grad() to compute gradients<br>
loss_grads = grad(loss, model.parameters(), create_graph=True)</p>
<p>gn2 = sum([grd.norm()**2 for grd in loss_grads]) # 2nd derive<br>
print(‘loss %f grad norm %f’ % (loss.data, gn2.data))<br>
model.zero_grad()<br>
gn2.backward()<br>
optimizer.step()</p>
<p>for grd in loss_grads:<br>
print grd.grad</p>
<p>The output is None.</p>
<p>Can any one tell me how to get it?</p>

tom: <p>You can call <a href="http://pytorch.org/docs/0.3.1/autograd.html#torch.autograd.Variable.retain_grad" rel="nofollow noopener"><code>grd.retain_grad ()</code></a> before backward to keep the grad of a non-leaf variable.</p>
<p>Best regards</p>
<p>Thomas</p>

Wei_Deng: <p>Thanks Tom, I got the grad, but it is not correct. Like the following example, i want to get the second derivative of (2x)^2 at x0=0.5153, the final result could return the 1st order derivative correctly which is 8*x0=4.12221, but for the second derivative, it is not the expected 8, do you know why?</p>
<p>import torch<br>
from torch import Tensor<br>
from torch.autograd import Variable<br>
from torch.autograd import grad<br>
from torch import nn</p>
<p>torch.manual_seed(1)<br>
x = Variable(Tensor([2.]), requires_grad=False)</p>
<p>model = nn.Linear(1, 1, bias=False)</p>
<p>x0 = [par.data for par in model.parameters()][0]<br>
print(x0)</p>
<p>loss = torch.sum(model(x)**2)<br>
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)<br>
loss_grads = grad(loss, model.parameters(), create_graph=True)<br>
gn2 = sum([grd.norm()**2 for grd in loss_grads]) / 2 # 2nd derive<br>
print(‘loss %f grad norm %f’ % (loss.data, gn2.data))</p>
<p>for grd in loss_grads:<br>
grd = grd.retain_grad()</p>
<p>model.zero_grad()<br>
gn2.backward(retain_graph=True)</p>
<p>for grd in loss_grads:<br>
print 8 * x0, grd.data[0], grd.grad</p>

tom: <p>This calculated d gn2 / d grd = d (0.5 grd^2) / d grd = grd correctly, but maybe you want something else?</p>
<p>Best regards</p>
<p>Thomas</p>

Wei_Deng: <p>Gotcha, that’s why the answers are the same, Thank you so much. Do you know how to calculate the second derivative of (x1)^2 + (2*x2)^2 with respect to x1 and x2, which should be (2, 8)?</p>
<p>Really appreciate your suggestions. Thanks a lot.</p>

tom: <p>I must admit that I’m confused about how the linear layer fits into what you want to achieve.<br>
If you drop the nn.Linear and start with <code>x</code> as requires_grad = True, you get the 2nd derivative in x.grad…</p>

Wei_Deng: <p>Hi, Tom, sorry for not explaining explicitly on my question.</p>
<p>My ultimate question is that if I got a neural network loss function, can I get the 2nd derivative of the likelihood function with respect to every weight? It doesn’t have to be a hessian matrix, but just the diagonal of it.</p>
<p>Do you know if we can do it based on the current version?</p>

tom: <p>I don’t think that this is currently possible in geberal (unless iterating over the scalar parameters). The fundamental reason is that backpropagation in PyTorch isn’t prepared take deruvatives of vector-valued functions, so you are limited to taking the derivative of a scalar sum of derivatives, i.o.w. a Hessian-Vector product.<br>
For a small number of parameters using torch.autograd.grad will help, but I’m not sure it scales to all parameters of large nets.</p>
<p>Best regards</p>
<p>Thomas</p>

Wei_Deng: <p>Got it, thanks a lot.</p>

nima_rafiee: <p>is there a way to get the full Hessian matrix with w.r.s to the input. calling the backward() function two times only provides me with a diagonal of Hessian matrix but not the full one. I need something like<br>
tf.Hessian().</p>