In-place operation error while training MADDPG

ntuce002: <p>I’m trying to train the MADDPG model; however, it has occurred an in-place operation error.<br>
Here’s the traceback, and I’ve taken some excerpts that I think are critical:</p>
<blockquote>
<p>RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [16, 2]], which is output 0 of AsStridedBackward0, is at version 3; expected version 2 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!</p>
</blockquote>
<blockquote>
<p>[W …\torch\csrc\autograd\python_anomaly_mode.cpp:104] Warning: Error detected in AddmmBackward0. Traceback of forward call that caused the error:</p>
</blockquote>
<blockquote>
<p>c_loss, a_loss = all_agents.learn(memory)<br>
File “C:\Users\chhuang\new_dens_speed_offset\maddpg.py”, line 107, in learn<br>
pi = self.agents[idx - 1].actor(car_dens_state[‘%s’ % idx], scooter_dens_state[‘%s’ % idx],<br>
File “C:\Users\chhuang\anaconda3_1\lib\site-packages\torch\nn\modules\module.py”, line 1102, in _call_impl<br>
return forward_call(*input, **kwargs)<br>
File “C:\Users\chhuang\new_dens_speed_offset\networkv4.py”, line 73, in forward<br>
xyzw = self.mu(xyzw)</p>
</blockquote>
<p>It showed the error had occurred in networkv4.py, line 73.<br>
The code <code>xyzw = self.mu(xyzw)</code> is in the <code>forward()</code> method, under the <code>Actor()</code> class. The whole code of the <code>Actor()</code> class is down below:</p>
<pre><code class="lang-auto">class Actor(nn.Module):
    def __init__(self, alpha, name, chkpt_dir):
        super(Actor, self).__init__()
        
        self.chkpt_file = os.path.join(chkpt_dir, name)

        self.fc1 = nn.Linear(4, 16)
        self.fc2 = nn.Linear(64, 64)
        self.fc3 = nn.Linear(64, 32)
        self.fc4 = nn.Linear(32, 16)
        
        self.bn1 = nn.LayerNorm(16)
        self.bn2 = nn.LayerNorm(64)
        self.bn3 = nn.LayerNorm(32)
        self.bn4 = nn.LayerNorm(16)
        
        self.mu = nn.Linear(16, 2)
        
        self.optimizer = optim.Adam(self.parameters(), lr=alpha)
        self.device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
        self.to(self.device)

    def forward(self, car_D, scooter_D, car_S, scooter_S):
        torch.autograd.set_detect_anomaly(True)
        x = self.fc1(car_D)
        x = self.bn1(x)
        x = F.relu(x).flatten(1)
        
        y = self.fc1(scooter_D)
        y = self.bn1(y)
        y = F.relu(y).flatten(1)
        
        z = self.fc1(car_S)
        z = self.bn1(z)
        z = F.relu(z).flatten(1)
        
        w = self.fc1(scooter_S)
        w = self.bn1(w)
        w = F.relu(w).flatten(1)
        
        xyzw = torch.cat([x, y, z, w], dim=1)
        xyzw = self.fc2(xyzw)
        xyzw = self.bn2(xyzw)
        xyzw = F.relu(xyzw)
        
        xyzw = self.fc3(xyzw)
        xyzw = self.bn3(xyzw)
        xyzw = F.relu(xyzw)
        
        xyzw = self.fc4(xyzw)
        xyzw = self.bn4(xyzw)
        xyzw = F.relu(xyzw)
        xyzw = self.mu(xyzw)
        xyzw = torch.sigmoid(xyzw)
        return xyzw

</code></pre>
<p>And here’s the link to the whole code of <a href="https://github.com/chhuang17/MADDPG-trafflc_signal/blob/main/maddpg.py" rel="noopener nofollow ugc">maddpg.py</a>. They are a little bit ugly so I uploaded them to the github instead of posting them here.</p>
<p>I’ve stuck with this problem all day long, and still couldn’t find out where’s the bug. Hope someone can give me some directions to modify my code properly.</p>

ptrblck: <p>Double post from <a href="https://discuss.pytorch.org/t/cannot-find-wheres-the-in-place-operation-error/151610/2">here</a> with follow-up.</p>