Ａ question about a=a+b?

username: Joker
cooked: <p>Hello , everyone ,  when  I   want    implement   this    statement  <code>a=a+b</code> ,I  know  it  could  be   <code>backward</code>.</p>
<p>But  if   I want  to    implement   this    statement  <code>a[5]=a[5]+b[3]</code> , I  know  this  action  trigger a<br>
<code>inplace        operation</code>    because  of    <code>a</code> is   a   array  ,and   it   can  not  be  <code>backward</code>in  pytorch 1.2.0  ,  it’s  memeroy  has  been  changed  I   think .</p>
<p>And  this  action   seems  can  be  done   in  pytorch 1.1.0  but  not   in  pytorch  1.2.0 .  But    if  I  want  to    implement  like  that  ,   what  should  I  write ?<br>
Thanks  for you  reply  and    help.</p>
<p>Below  is  my     wrong  log .</p>
<blockquote>
<p>RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [212, 256, 7, 7]], which is output 0 of IndexPutBackward, is at version 3; expected version 2 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).</p>
</blockquote>

username: Nikronic
cooked: <p>Hi,</p>
<p>Please see below posts:</p><aside class="quote quote-modified" data-post="1" data-topic="52044">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/yupjun/48/14585_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/setting-part-of-the-training-value-into-zero/52044">Setting part of the training value into zero</a> 
  </div>
  <blockquote>
    When I training my network, I want to set a specific part of latent space into the zero value based on the label. 
For example, after input values come out from convolution layer, there would be (B, C, H, W) 4-D tensor value would come out. 
I want to make half of the Channel part into zero based on the label. 
[image] 
Like this but I got an error like this. 
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [1,…
  </blockquote>
</aside>

<aside class="quote" data-post="2" data-topic="51574">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/nikronic/48/71873_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/why-this-code-will-throw-out-an-inplace-error/51574/2">Why this code will throw out an inplace error</a> <a class="badge-category__wrapper " href="/c/autograd/7"><span data-category-id="7" style="--category-badge-color: #AB9364; --category-badge-text-color: #FFFFFF;" data-drop-close="true" class="badge-category " title="A category of posts relating to the autograd engine itself."><span class="badge-category__name">autograd</span></span></a>
  </div>
  <blockquote>
    Hi, 
In this line, you are updating B variable but autograd needs all previous values to compute the backpropagation. 
You can do something like this to ensure you are not losing the value of B: 
import torch

A = torch.nn.Parameter(torch.randn(3, 3))
B = torch.randn(2, 4, 3)
z = B[0, :, :].mm(A)
loss = z.sum()
loss.backward()

When I assign updated B to z, I am not mutating B so autograd has access to original B. 
Best regards 
Nik
  </blockquote>
</aside>

<p>Bests<br>
Nik</p>

username: Joker
cooked: <p><a class="mention" href="/u/nikronic">@Nikronic</a>  Thanks  for your  help  ,  friend .I  solve  it .</p>
<p>After  reading   your   posts   and  pytorch  doc    about   <code>torch.clone()</code>  ,</p>
<pre><code class="lang-auto">clone() → Tensor 
Returns a copy of the self tensor. The copy has the same size and data type as self.

NOTE

Unlike copy_(), this function is recorded in the computation graph. Gradients propagating to the cloned tensor will propagate to the original tensor.
</code></pre>
<p>I  changed  my   code  to   this ,  and  it  could   been     backward  well.</p>
<pre><code class="lang-auto"> c=a.clone()
c[5] = a[5]+b[3]
loss=  m-c
loss.backward()
</code></pre>
<p>emmmmmmmmm      if  there  is   a   function  in  tensor  I  can  do  it  directly    it    will  be  very  convenient .</p>
<p>Like</p>
<pre><code class="lang-auto">a[5].operation(torch.add , a[5] , b[3])
a[5].operation(torch.mm , a[5] , b[3])

And  function  `operation`  will   create  a   new   memory  to  a[5]  but  not  change  its  original  memory . So  it  could  be   backward .
</code></pre>
<p>Anyway ,  thanks for your  help .</p>