Very Strange Things (New Beginner)

username: TonoTakaki
cooked: <p>In one of my trials, I try to compute loss for each of my data then collect the “prob” in a list. Then, in each iteration(time step), I try to update my net. <strong>But with RunError:</strong> <code>RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [128, 2]], which is output 0 of TBackward, is at version 2; expected version 1 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!</code></p>
<p>But, I don’t know where is the wrong lies, since I don’t have any in-place operation. Could any one help me?</p>
<p>My code Like:</p>
<pre><code class="lang-auto">import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
import matplotlib.pyplot as plt
from torch.distributions import Categorical
from torch.nn.utils import clip_grad_norm_

torch.autograd.set_detect_anomaly(True)

class Policy(nn.Module):
    def __init__(self, n_input, n_output, hidden_dim = 128):
        super(Policy,self).__init__()
        self.n_input = n_input
        self.n_output = n_output
        self.lin1 = nn.Linear(self.n_input, hidden_dim)
        self.lin2 = nn.Linear(hidden_dim, self.n_output)
    
    def forward(self,state):
        x = F.relu(self.lin1(state))
        x = self.lin2(x)
        return x

p = Policy(4,2)
lists = []
optimizer = optim.Adam(p.parameters(), lr=0.01)

#psudeo data
data = [[1,2,3,4],[2,1,3,4],[3,1,3,4]]
data = torch.tensor(data).float()

for i in range(data.shape[0]):
    x = data[i,:].unsqueeze(0)
    prob = p(x).max()
    lists.append(prob)

#BIG PROBLEM HERE!!!
for t in range(len(lists)):
    print("t:",t)
    loss = -lists[t]
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
</code></pre>

username: TonoTakaki
cooked: <p>Oh, I think something wrong in self.lin2, since the weight of lin2 is (128,2)<br>
`<br>
[W python_anomaly_mode.cpp:60] Warning: Error detected in AddmmBackward. Traceback of forward call that caused the error:<br>
File “/Users/kakusou/Work/VScode/mimic/PG/test.py”, line 34, in <br>
prob = p(x).max()<br>
File “/Users/kakusou/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py”, line 722, in _call_impl<br>
result = self.forward(*input, **kwargs)<br>
File “/Users/kakusou/Work/VScode/mimic/PG/test.py”, line 22, in forward<br>
x = self.lin2(x)<br>
File “/Users/kakusou/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py”, line 722, in _call_impl<br>
result = self.forward(*input, **kwargs)<br>
File “/Users/kakusou/anaconda3/lib/python3.7/site-packages/torch/nn/modules/linear.py”, line 91, in forward<br>
return F.linear(input, self.weight, self.bias)<br>
File “/Users/kakusou/anaconda3/lib/python3.7/site-packages/torch/nn/functional.py”, line 1674, in linear<br>
ret = torch.addmm(bias, input, weight.t())<br>
(function print_stack)</p>
<p>`</p>

username: TonoTakaki
cooked: <p>And, If I run this code in torch1.4.0, no error at all.<br>
But, in torch1.6.0, it continues showing the above error. I just don’t understand it</p>

username: ptrblck
cooked: <p>The issue is that the stored <code>probs</code> in <code>lists</code> are invalid after the first <code>optimizer.step()</code> and parameter update.<br>
I.e. the <code>prob</code> tensors were creates with a model using the initial parameter set P0.<br>
After the first <code>loss.backward()</code> and <code>optimizer.step()</code> operation these parameters will be updated to P1.<br>
In the next <code>loss.backward()</code> and <code>optimizer.step()</code> iteration you are trying to calculate the gradients of the “old” loss w.r.t the new parameters, which is wrong since these already parameters didn’t calculate the loss.<br>
Older PyTorch versions didn’t implement this check and would thus yield wrong results.</p>
<p>The proper way would be to recalculate the <code>prob</code> after each update or to calculate all gradients before calling <code>optimizer.step()</code>.</p>