Error related to in-place operation

username: scrungus
cooked: <p>I am getting a strange error related to inplace operations. I have a neural net:</p>
<pre><code class="lang-auto">class ActorNet(nn.Module):
    def __init__(self, obs_size, n_actions, depth, hidden_size = 512):
        super().__init__()

        self.actor_conv = nn.Sequential(nn.Conv1d(in_channels=1, out_channels=32,kernel_size=8,stride=4,padding=7),
        nn.ReLU(),
        nn.Conv1d(in_channels=32, out_channels=64,kernel_size=4,stride=2,padding=3),
        nn.ReLU(),
        nn.Conv1d(in_channels=64, out_channels=64,kernel_size=3,stride=1,padding=2),
        nn.ReLU(),
        )

        self.actor = nn.Sequential(
            nn.Linear(1088, hidden_size),
            nn.ReLU(),
            nn.Linear(hidden_size, n_actions),
            )

        self.critic = nn.Sequential(
            nn.Linear(1088, hidden_size),
            nn.ReLU(),
            nn.Linear(hidden_size, 1)
        )

    def forward(self, x):
        if x.dim() == 1:
            convs = self.actor_conv(x[None,...][None,...])
        else:
            convs = self.actor_conv(x)
        convs = convs.view(convs.shape[0],-1)
        logits = self.actor(convs)
        logits = torch.nan_to_num(logits)
        dist = Categorical(logits=logits)
        action = dist.sample()

        value = self.critic(convs)

        return dist, action, value
</code></pre>
<p>And a function which generates samples :</p>
<pre><code class="lang-auto">def make_batch(self):
        for i in range(self.hparams.epoch_steps):

            dist, action, val = self.actor(self.state)
            probs = dist.log_prob(action)
            next_state, reward, done, _ = self.env.step(action.item())
...
</code></pre>
<p>Which, when running <code>loss.backward()</code> throws the error:</p>
<p><code>RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [512, 19]], which is output 0 of AsStridedBackward0, is at version 2; expected version 1 instead.</code></p>
<p>I have isolated the cause to this line:</p>
<p><code> probs = dist.log_prob(action)</code></p>
<p>as the error does not occur when I take this out. Why would this throw an error like this?</p>
<p>Thanks for any help in advance</p>

username: scrungus
cooked: <p>Update:</p>
<p>I’ve found that adding <code>with torch.no_grad():</code> before getting the log probabilities solves the issue, but I don’t understand why.</p>

username: KFrank
cooked: <p>Hi Scrungus!</p>
<aside class="quote no-group" data-username="scrungus" data-post="1" data-topic="149253" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/s/34f0e0/48.png" class="avatar"> scrungus:</div>
<blockquote>
<p>Which, when running <code>loss.backward()</code> throws the error:</p>
<p><code>RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [512, 19]], which is output 0 of AsStridedBackward0, is at version 2; expected version 1 instead.</code></p>
</blockquote>
</aside>
<p>First, strip down your code as much as possible while still keeping the<br>
“inplace” issue.</p>
<p>Then post a simple-as-possible, fully-self-contained script that reproduces<br>
your issue, together with the output you get when you run it.</p>
<p>(Since the script you post will be fully self-contained, it will include where<br>
and how you calculate <code>loss</code> and call <code>loss.backward()</code>.)</p>
<p>Best.</p>
<p>K. Frank</p>