I don't find the error

username: Giuseppe
cooked: <pre><code class="lang-auto">    def update(self, log_prob1, log_prob2,value, reward):
       

        advantage = reward - value

        policy_loss = ((-log_prob1 * advantage) + (-log_prob2 * advantage)).mean()
        value_loss = F.smooth_l1_loss(value, reward)
        loss = policy_loss + value_loss

        self.optimizer.zero_grad()
        loss.backward()
        self.optimizer.step()
</code></pre>
<p>In loss.backward i have a runtime error:<br>
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [16, 1]], which is output 0 of SqrtBackward, is at version 1; expected version 0 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).</p>
<p>Theese variables are torch tensor of 16x1, where is the inplace operation?</p>

username: ptrblck
cooked: <p>I can’t see any inplace operations in your code snippet.<br>
Could you post dummy tensor shapes, so that we could reproduce this issue?</p>

username: Giuseppe
cooked: <pre><code class="lang-auto">                    rewards = []
                    for a1, a2 in zip(action1, action2):


                        rewards.append(loader[mode].dataset.get_reward(state.item(), a1[0], a1[1], a2[0], a2[1]))

                    rewards = torch.Tensor(rewards).view(-1,1).cuda()#BS x 1
                    log_prob1=log_prob1.view(-1,1)
                    log_prob2 = log_prob2.view(-1, 1)
                    state_value = state_value.view(-1,1)

                    if mode == 'train':
                        loss_to_plot, policy_to_plot, reward_to_plot, value_to_plot = estimator.update(log_prob1,
                                                                                                        log_prob2,
                                                                                                        state_value,
                                                                                                        rewards)



This is the code that call the method above.
All tensors are 16x1, the only thing different is in structure: miss "grad_fn=&lt;ViewBackward&gt; " in tensor reward. I can't see any inplace operation too.
I print these tensors:



</code></pre>
<p>tensor([[-2.2734],<br>
[-2.3429],<br>
[-2.3219],<br>
[-1.6636],<br>
[-3.4040],<br>
[-3.1213],<br>
[-4.9740],<br>
[-2.1452],<br>
[-2.5107],<br>
[-4.0225],<br>
[-3.3761],<br>
[-1.9465],<br>
[-3.5561],<br>
[-1.5919],<br>
[-1.8437],<br>
[-2.8795]], device=‘cuda:0’, grad_fn=ViewBackward)<br>
tensor([[-1.7228],<br>
[-2.8122],<br>
[-2.3623],<br>
[-2.4175],<br>
[-1.9202],<br>
[-1.6540],<br>
[-3.5124],<br>
[-1.7577],<br>
[-5.4544],<br>
[-1.6919],<br>
[-3.0612],<br>
[-3.8174],<br>
[-2.7434],<br>
[-4.5231],<br>
[-3.1026],<br>
[-2.8684]], device=‘cuda:0’, grad_fn=ViewBackward)<br>
tensor([[-0.0289],<br>
[ 0.0604],<br>
[ 0.0796],<br>
[-0.0133],<br>
[-0.1071],<br>
[-0.0423],<br>
[-0.0217],<br>
[ 0.1402],<br>
[-0.0927],<br>
[-0.0896],<br>
[ 0.0655],<br>
[ 0.0261],<br>
[ 0.0024],<br>
[ 0.0145],<br>
[-0.0375],<br>
[ 0.0536]], device=‘cuda:0’, grad_fn=ViewBackward)<br>
tensor([[1.0000],<br>
[0.5000],<br>
[0.0000],<br>
[0.5000],<br>
[1.0000],<br>
[0.0000],<br>
[0.0000],<br>
[0.0000],<br>
[1.0000],<br>
[1.0000],<br>
[0.0000],<br>
[0.5000],<br>
[0.5000],<br>
[0.5000],<br>
[0.5000],<br>
[0.0000]], device=‘cuda:0’)</p>
<pre><code class="lang-auto">

I print respectively in the same order when i pass them</code></pre>