Why the requires_grad of autograd.Function is Ture

ronda: <p>Hi, I have a question,</p>
<pre><code class="lang-auto">class MyFunc(torch.autograd.Function):
    @staticmethod
    def forward(ctx, input):
        # return input.detach()
        return torch.tensor([1.0], requires_grad=False)

    @staticmethod
    def backward(ctx, grad_input):
        return grad_input

input = torch.tensor([2.], requires_grad=True)
output = MyFunc.apply(input)

print(output.requires_grad)  # True
</code></pre>
<p>I don’t understand why output.requires_grad is True while the result of  forward  is False</p>
<p>I am new to pytorch and hope I don’t bother you.</p>
<p>Thanks</p>

soulitzer: <p>This is pretty tricky indeed, but this behavior is noted in the docs:</p>
<blockquote>
<p>By default, all the output Tensors that are of differentiable type will be set to require gradient and have all autograd metadata set for them. If you don’t want them to require gradients, you can use the mark_non_differentiable method mentioned above. For output Tensors that are not of differentiable type (integer types for example), they won’t be marked as requiring gradients.</p>
</blockquote>
<p>See: <a href="https://pytorch.org/docs/stable/notes/extending.html" class="inline-onebox" rel="noopener nofollow ugc">Extending PyTorch — PyTorch 2.1 documentation</a></p>

ronda: <p>Thank you very much for your answer, may I ask where to view its source code, I would like to know some details of it.</p>

soulitzer: <p>Check here for the entry point of Python custom functions once you call <code>apply</code>:</p><aside class="onebox githubblob">
  <header class="source">

      <a href="https://github.com/pytorch/pytorch/blob/f914ab193ebe5d432985af42a274dd21cb8c4fb8/torch/csrc/autograd/python_function.cpp#L581" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/pytorch/pytorch/blob/f914ab193ebe5d432985af42a274dd21cb8c4fb8/torch/csrc/autograd/python_function.cpp#L581" target="_blank" rel="noopener nofollow ugc">pytorch/pytorch/blob/f914ab193ebe5d432985af42a274dd21cb8c4fb8/torch/csrc/autograd/python_function.cpp#L581</a></h4>



  <pre class="onebox">    <code class="lang-cpp">
      <ol class="start lines" start="571" style="counter-reset: li-counter 570 ;">
          <li>  TORCH_CHECK(cdata,</li>
          <li>    "Attribute 'name' is invalid for this instance of _C._FunctionBase. "</li>
          <li>    "Accessing this attribute directly on an instance of autograd.Function is a legacy "</li>
          <li>    "access pattern that is no longer supported. For examples on how to use new-style "</li>
          <li>    "autograd functions, see "</li>
          <li>    "https://pytorch.org/docs/stable/autograd.html#torch.autograd.Function ");</li>
          <li>  return THPUtils_packString(cdata-&gt;name());</li>
          <li>  END_HANDLE_TH_ERRORS</li>
          <li>}</li>
          <li>
          </li>
<li class="selected">PyObject *THPFunction_apply(PyObject *cls, PyObject *inputs)</li>
          <li>{</li>
          <li>  HANDLE_TH_ERRORS</li>
          <li>  RECORD_FUNCTION(</li>
          <li>    ((PyTypeObject*)cls)-&gt;tp_name,</li>
          <li>    std::vector&lt;c10::IValue&gt;(),</li>
          <li>    at::sequence_number::peek());</li>
          <li>
          </li>
<li>  THPObjectPtr backward_cls(PyObject_GetAttrString(cls, "_backward_cls"));</li>
          <li>  if (!backward_cls) return nullptr;</li>
          <li>  THPObjectPtr ctx_obj(PyObject_CallFunctionObjArgs(backward_cls, nullptr));</li>
      </ol>
    </code>
  </pre>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>This function is where the grad_fn is set for the output. You can see that <code>set_gradient_edge</code> is called as long as the output is differentiable (i.e., its scalar type is floating or complex).</p><aside class="onebox githubblob">
  <header class="source">

      <a href="https://github.com/pytorch/pytorch/blob/5e72571df3f81a1ac9d0a2c96eff4acdac17579f/torch/csrc/autograd/custom_function.cpp#L28" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/pytorch/pytorch/blob/5e72571df3f81a1ac9d0a2c96eff4acdac17579f/torch/csrc/autograd/custom_function.cpp#L28" target="_blank" rel="noopener nofollow ugc">pytorch/pytorch/blob/5e72571df3f81a1ac9d0a2c96eff4acdac17579f/torch/csrc/autograd/custom_function.cpp#L28</a></h4>



  <pre class="onebox">    <code class="lang-cpp">
      <ol class="start lines" start="18" style="counter-reset: li-counter 17 ;">
          <li>Variable VariableInfo::zeros(at::OptionalDeviceGuard&amp; device_guard) const {</li>
          <li>  if (is_empty) {</li>
          <li>    // Return undefined tensor.</li>
          <li>    return at::Tensor();</li>
          <li>  } else {</li>
          <li>    return at::zeros(</li>
          <li>        size, at::TensorOptions(scalar_type).device(device).layout(layout));</li>
          <li>  }</li>
          <li>}</li>
          <li>
          </li>
<li class="selected">std::vector&lt;c10::optional&lt;Variable&gt;&gt; _wrap_outputs(const variable_list &amp;input_vars,</li>
          <li>  const std::unordered_set&lt;at::TensorImpl*&gt; &amp;non_differentiable,</li>
          <li>  const std::unordered_set&lt;at::TensorImpl*&gt; &amp;dirty_inputs,</li>
          <li>  const at::ArrayRef&lt;c10::optional&lt;Variable&gt;&gt; raw_outputs,</li>
          <li>  const std::shared_ptr&lt;Node&gt; &amp;cdata) {</li>
          <li>
          </li>
<li>  std::unordered_set&lt;at::TensorImpl*&gt; inputs;</li>
          <li>  inputs.reserve(input_vars.size());</li>
          <li>  for (auto&amp; var : input_vars) {</li>
          <li>    inputs.emplace(var.unsafeGetTensorImpl());</li>
          <li>  }</li>
      </ol>
    </code>
  </pre>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>