How to include gradient of nn wrt. the input in the loss function?

XiaodongMa-MRI: <p>Hello all,</p>
<p>I want to include gradient of nn wrt. the input in the loss function, here is my code:</p>
<pre><code class="lang-auto">....
dydx  = torch.zeros(Nminibatch,Nin*2)
Y_net = torch.zeros(Nminibatch,Nout)
optimizer = optim.Adam(net.parameters(), lr=LR)

for epoch in range(Nepochs):
  print('starting epoch ' + str(epoch) + ', Learning rate = ' + str(LR))
  for batch_idx, (X, Y) in enumerate(loader):
    X.requires_grad = True
    Y_net = net(X)
    # loop over minibatch
    for idx in range(Y_net.size()[0]):
      dydx[idx,:] = torch.autograd.grad(Y_net[idx,0],X,create_graph=True)[0][idx,:]
    loss = loss_f(Y_net, Y) + (dydx[:,torch.arange(0,Nin)]).sum()
    optimizer.zero_grad()
    loss.backward(retain_graph=True)
    optimizer.step()
</code></pre>
<p>But I got this error message:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "&lt;stdin&gt;", line 14, in &lt;module&gt;
  File "/home/truhlard/ning0035/anaconda3/lib/python3.7/site-packages/torch/tensor.py", line 221, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/home/truhlard/ning0035/anaconda3/lib/python3.7/site-packages/torch/autograd/__init__.py", line 132, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [50, 50]], which is output 0 of TBackward, is at version 2; expected version 1 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!
</code></pre>
<p>Does anyone know how to fix this?</p>
<p>Thanks a lot!</p>
<p>Xiaodong</p>

ptrblck: <p>I guess the error is raised by using <code>retain_graph=True</code> and trying to update parameters multiple times, which would be wrong, since the gradient would be stale.<br>
Could you explain why you are using this argument?</p>

XiaodongMa-MRI: <p>Thanks for your reply! I think you are right. In my code, the parmaters inside the mini-batch loop could be updated multiple times:</p>
<pre><code class="lang-auto">for idx in range(Y_net.size()[0]):
      dydx[idx,:] = torch.autograd.grad(Y_net[idx,0],X,create_graph=True)[0][idx,:]
</code></pre>
<p>I have changed the code by defining another tensor <code>loss_g</code>, and it can run smoothly now:</p>
<aside class="quote no-group quote-modified" data-username="XiaodongMa-MRI" data-post="1" data-topic="103758">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/xiaodongma-mri/48/31372_2.png" class="avatar"> XiaodongMa-MRI:</div>
<blockquote>
<pre><code class="lang-auto">for epoch in range(Nepochs):
  print('starting epoch ' + str(epoch) + ', Learning rate = ' + str(LR))
  for batch_idx, (X, Y) in enumerate(loader):
    X.requires_grad = True
    Y_net = net(X)
    loss_g = torch.zeros([0.0])
    # loop over minibatch
    for idx in range(Y_net.size()[0]):
      dydx[idx,:] = torch.autograd.grad(Y_net[idx,0],X,create_graph=True)[0][idx,:]
      loss_g = loss_g + (dydx[:,torch.arange(0,Nin)]).sum()
    loss = loss_f(Y_net, Y) + loss_g
    optimizer.zero_grad()
    loss.backward(retain_graph=True)
    optimizer.step()
</code></pre>
</blockquote>
</aside>
<p>The reason for this implementation is that I wan to make sure the gradient of nn output wrt. the input is similar to some known values. I used <code>(dydx[:,torch.arange(0,Nin)]).sum()</code> here just for simplicity.</p>
<p>Xiaodong</p>