Guidelines for when and why one should set inplace = True?

John_Deterious: <p>I know what inplace does, but, can you please explain why should I opt to use it or not. Are there any caveats and what are the scenarios in which this option is important, sepecifically when building a NN. Take ReLU layer as an example in which this option is available.</p>

Nikronic: <p>Hello,</p>
<p>First, there is an important thing you have to consider; you only can use <code>inplace=True</code> when you are sure your model won’t cause any error. For example, if you trying to train a CNN, in the time of backpropagation, <code>autograd</code> needs all the values, but <code>inplace=True</code> operation can cause a change so your backprop is no longer valid. Actually, this kind of error has been handled by PyTorch, so you’ll be noticed about it.</p>
<p>Second, if you do not have any error, it is better to use <code>inplace=True</code> operation because it won’t allocate new memory for the output of your layer. So it can prevent from <code>Out of memory</code> error.</p>
<p>Finally, as far as I know developers usually use <code>inplace=True</code> unless they do not get any error.</p>
<aside class="quote quote-modified" data-post="3" data-topic="36289">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/c/ad7895/48.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/i-got-runtimeerror-one-of-the-variables-needed-for-gradient-computation-has-been-modified-by-an-inplace-operation-error/36289/3">I got 'RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation' error</a> 
  </div>
  <blockquote>
    Would you please show me how to fix this? I changed it like this: 
  def forward(self, logits, labels):
        N, C, H, W = logits.size()
        n_pixs = N * H * W
        logits = logits.permute(0, 2, 3, 1).contiguous().view(-1, C)
        with torch.no_grad():
            scores = F.softmax(logits, dim=1).cpu().detach()
            labels = labels.view(-1)
            labels_cpu = labels.cpu().detach()
            invalid_mask = labels_cpu==self.ignore_lb
            labels_cpu[invalid_mask]…
  </blockquote>
</aside>

<aside class="quote" data-post="1" data-topic="948">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/yunjey/48/1386_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/whats-the-difference-between-nn-relu-and-nn-relu-inplace-true/948">What's the difference between nn.ReLU() and nn.ReLU(inplace=True)?</a> 
  </div>
  <blockquote>
    I implemented generative adversarial network using both nn.ReLU() and nn.ReLU(inplace=True). It seems that nn.ReLU(inplace=True) saved very small amount of memory. 
What’s the purpose of the using inplace=True? 
Is the behavior different in backpropagation?
  </blockquote>
</aside>

<p>bests</p>

tom: <p>Indeed <a class="mention" href="/u/nikronic">@Nikronic</a> nails it with the rule of thumb <em>You can use inplace for memory efficiency unless you it breaks</em>.<br>
You might also be less eager to use inplace when planning to use the JIT, as it will fuse pointwise non-inplace operations like ReLU if there are several in a row.</p>
<p>The two things to avoid are:</p>
<ul>
<li>you move a leaf tensor into the graph (using inplace on something that you just defined with <code>requires_grad=True</code>),</li>
<li>when the operation before the inplace wants to have its result to compute the backward. Whether this is the case is not easy to tell with from the outside, unfortunately.</li>
</ul>
<p>As a corollary, you should avoid using inplace on the inputs of your re-usable module in lest the future use could be in one of the two situations.</p>
<p>Best regards</p>
<p>Thomas</p>