Why are "x+=x" and "x=x+x" are handled differently?

Bibhabasu_Mohapatra: <p>I found that when “<code>requires_grad=True</code>” then x+=x does not work for backprop in loss.backward() it .<br>
I noticed this because in MyModel when I did x+=x in forward() then it gave autograd error for AddBackward() variable in-place operation.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/6/e65c1318239906b66b4374fc4d708bad3fe683fb.png" data-download-href="https://discuss.pytorch.org/uploads/default/e65c1318239906b66b4374fc4d708bad3fe683fb" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/e/6/e65c1318239906b66b4374fc4d708bad3fe683fb.png" alt="image" data-base62-sha1="wRR5oNunsrGfjkFY6nnUPjWMBJ1" width="690" height="88" data-dominant-color="151515"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1366×176 7.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
but when, x+=x<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/e/6e67bab68d7f27c0419944f7c6bd957a65bddee7.png" data-download-href="https://discuss.pytorch.org/uploads/default/6e67bab68d7f27c0419944f7c6bd957a65bddee7" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/6/e/6e67bab68d7f27c0419944f7c6bd957a65bddee7.png" alt="image" data-base62-sha1="fKGLaItGNMdr9HwKPJ6SWXDBzAX" width="690" height="101" data-dominant-color="171717"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1409×207 10.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I know x+=1 and x = x + 1 are not the same. but can’t it be handled in the same way in Torch, why the error?</p>

KFrank: <p>Hi Bibhabasu!</p>
<aside class="quote no-group" data-username="Bibhabasu_Mohapatra" data-post="1" data-topic="198045" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/bibhabasu_mohapatra/48/42965_2.png" class="avatar"> Bibhabasu_Mohapatra:</div>
<blockquote>
<p>I know x+=1 and x = x + 1 are not the same. but can’t it be handled in the same way in Torch, why the error?</p>
</blockquote>
</aside>
<p>This is (I presume) by design.  According to standard python syntax, when<br>
you execute <code>x = x + 1</code>, python evaluates the right-hand side, and, in the<br>
case of pytorch, causes a new tensor to be create.  Then the python “variable”<br>
<code>x</code> is set to refer to that new tensor.</p>
<p>On the other hand, the pytorch designers chose (logically, in my mind) to<br>
implement <code>x += 1</code> as in-place tensor addition.  So this version does not<br>
create a new tensor, but, instead, modifies the existing <code>x</code> tensor in place<br>
(hence the error you get).</p>
<p>Why do things this way?  First, I think having <code>x += 1</code> be implemented as<br>
in-place addition is what I would “expect,” based on normal python syntax.<br>
Also, this way you have a convenient choice between the in-place and<br>
out-of-place operation.</p>
<p>Best.</p>
<p>K. Frank</p>

J_Johnson: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/4/54e31c3f34e933fb001411dd7ed7a6abed3886c0.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/54e31c3f34e933fb001411dd7ed7a6abed3886c0" title="1000013691"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/5/4/54e31c3f34e933fb001411dd7ed7a6abed3886c0_2_689x442.jpeg" alt="1000013691" data-base62-sha1="c6WGJbUV2RCyzjup4yLMoX8YrBu" width="689" height="442" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/5/4/54e31c3f34e933fb001411dd7ed7a6abed3886c0_2_689x442.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/5/4/54e31c3f34e933fb001411dd7ed7a6abed3886c0_2_1033x663.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/5/4/54e31c3f34e933fb001411dd7ed7a6abed3886c0.jpeg 2x" data-dominant-color="EEEEEE"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">1000013691</span><span class="informations">1240×795 141 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Please see here:</p>
<p><a href="https://pytorch.org/tutorials/beginner/introyt/autogradyt_tutorial.html#autograd-and-in-place-operations" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/tutorials/beginner/introyt/autogradyt_tutorial.html#autograd-and-in-place-operations</a></p>