Is there any difference between calling "requires_grad_()" method and manually set "requires_grad" attribute?

frostime: <p>I found that there are two ways to change <code>Tensor.requires_grad</code>。</p>
<p>I can manually set <code>x.requires_grad = flag</code> or I can call the method <code>x.requires_grad_(flag)</code>. I did a simple experiment, and found nothing difference.</p>
<pre><code class="lang-auto">In [14]: a = torch.tensor([2.0])

In [15]: a.requires_grad
Out[15]: False

In [16]: a.requires_grad_()
Out[16]: tensor([2.], requires_grad=True)

In [17]: a.requires_grad
Out[17]: True

In [18]: a.requires_grad = False

In [19]: a
Out[19]: tensor([2.])
</code></pre>
<p>I’m wondering that is there any detail difference between the aforementioned two ways? Dose the <code>requires_grad_(flag)</code> do any other things except <code>self.requires_grad = flag</code>?</p>
<p>I tried to read the source code of <code>requires_grad_</code>, while found nothing, it seems to be written in C code~~ <img src="https://discuss.pytorch.org/images/emoji/apple/expressionless.png?v=9" title=":expressionless:" class="emoji" alt=":expressionless:"></p>

soulitzer: <p>There are some subtle differences between the two:<br>
For example if you have a non-leaf tensor, setting it to True using <code>self.requires_grad=True</code> will produce an error, but not when you do <code>requires_grad_(True)</code>.<br>
Both perform some error checking, such as verifying that the tensor is a leaf, before calling into the same <code>set_requires_grad</code> function (implemented in cpp). But the exact checks done, and error messages produced are not identical:</p>
<pre><code class="lang-auto">&gt;&gt;&gt; a = torch.tensor(1., requires_grad=True)
&gt;&gt;&gt; b = a + 1
&gt;&gt;&gt; b.requires_grad_(True)
tensor(2., grad_fn=&lt;AddBackward0&gt;)
&gt;&gt;&gt; b.requires_grad_(False)
Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in &lt;module&gt;
RuntimeError: you can only change requires_grad flags of leaf variables. If you want to use a computed variable in a subgraph that doesn't require differentiation use var_no_grad = var.detach().
&gt;&gt;&gt; b.requires_grad = True
Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in &lt;module&gt;
RuntimeError: you can only change requires_grad flags of leaf variables.
&gt;&gt;&gt; b.requires_grad = False
Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in &lt;module&gt;
RuntimeError: you can only change requires_grad flags of leaf variables. If you want to use a computed variable in a subgraph that doesn't require differentiation use var_no_grad = var.detach().
&gt;&gt;&gt; 
</code></pre>
<p>The difference arises because the two APIs go through slightly different code paths:<br>
but if you’d like to investigate the code yourself, the Python bindings for setting the requires_grad attribute directly on the tensor <code>self.requires_grad</code> lives in <code>torch/csrc/autograd/python_variable.cpp</code>.<br>
<code>requires_grad_</code> on the other hand is a “native function”, i.e., it has a schema defined in <code>native_functions.yaml</code>. This also means that all the python bindings are codegened, so you’d have to clone the repo and actually build to see the code.</p>

slf188: <p><code>requires_grad</code> is a method to check if our tensor tracks gradients.<br>
whereas<br>
<code>requires_grad_</code> is a method that sets your tensors <code>requires_grad</code> attribute to <code>True</code></p>