.requires_grad_(True) doesnt work

Juuso_Korhonen: <p>Hi all,</p>
<p>What could be the reason for the following:</p>
<pre><code class="lang-auto">with torch.no_grad():
    outputs = self.module(input)
  
assert isinstance(outputs, (torch.Tensor, tuple)), \
    f"Output must be a tensor or a tuple of tensors. Got instead: {outputs.type}"

print(outputs.type()) # prints torch.cuda.FloatTensor
outputs.requires_grad_(True)
print(outputs.requires_grad) # prints false
</code></pre>
<p>So I try to modify a forward pass so that it completes the network forward in no_grad mode and calculates the loss gradient only w.r. to the output of the network. But calling .requires_grad_(True) seems to have no effect, as I still get false for the print.</p>

KFrank: <p>Hi Juuso!</p>
<aside class="quote no-group quote-modified" data-username="Juuso_Korhonen" data-post="1" data-topic="199601" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/juuso_korhonen/48/63920_2.png" class="avatar"> Juuso_Korhonen:</div>
<blockquote>
<pre><code class="lang-auto">outputs.requires_grad_(True)
print(outputs.requires_grad) # prints false
</code></pre>
<p>But calling .requires_grad_(True) seems to have no effect, as I still get false for the print.</p>
</blockquote>
</aside>
<p>This should work – <code>outputs.requires_grad</code> should print out as <code>True</code>.</p>
<p>Could you post a fully-self-contained, runnable script that reproduces your<br>
issue, together with the output you get when you run it?  Please also let us<br>
know what version of pytorch you are using.</p>
<p>Best.</p>
<p>K. Frank</p>

Juuso_Korhonen: <p>Hi <a class="mention" href="/u/kfrank">@KFrank</a>, this snippet is part of a larger program. I actually noticed that once I disabled the compiling of the training_step() function, changing of the requires_grad (during the forward()) works now. I’m not sure why it works this way however.</p>
<p>Here’s a code that reproduces the problem:</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn

class SimpleNN(nn.Module):
    def __init__(self, input_size, hidden_size, output_size):
        super(SimpleNN, self).__init__()
        self.fc1 = nn.Linear(input_size, hidden_size)
        self.activation = nn.ReLU()
        self.fc2 = nn.Linear(hidden_size, output_size)

    def forward(self, x):
        x = self.fc1(x)
        x = self.activation(x)
        x = self.fc2(x)
        return x
    

def task(model_1, model_2, input, target_output):
    # Forward pass
    with torch.no_grad():
        intermediate_output = model_1(input)
    intermediate_output.requires_grad_()
    print(intermediate_output.requires_grad) # prints false
    output = model_2(intermediate_output)
    loss_fn = nn.MSELoss()
    loss = loss_fn(output, target_data)
    return loss


# Subsequent modules test
input_size = 1 # xy location
hidden_size = 32
output_size = 1 # rgb 

model_1 = SimpleNN(input_size, hidden_size, output_size)
model_2 = SimpleNN(input_size, hidden_size, output_size)

# Creating random input and target tensors
torch.manual_seed(42)  # For reproducibility
N = 100  # Batch size
input_data = torch.randn(N, input_size)
target_data = torch.randn(N, output_size)

task = torch.compile(task)

for i in range(100):
    loss = task(model_1, model_2, input_data, target_data)
    loss.backward()
</code></pre>
<p>If I just comment out the task = torch.compile(task), changing of the gradient requirement works fine.</p>

KFrank: <p>Hi Juuso!</p>
<aside class="quote no-group" data-username="Juuso_Korhonen" data-post="3" data-topic="199601" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/juuso_korhonen/48/63920_2.png" class="avatar"> Juuso_Korhonen:</div>
<blockquote>
<p>If I just comment out the task = torch.compile(task), changing of the gradient requirement works fine.</p>
</blockquote>
</aside>
<p>I can reproduce* your issue with the code you posted. I’m not that<br>
knowledgeable about <code>torch.compile</code>, but this looks like a bug to me.</p>
<p>If you think that this github issue:</p>
<aside class="onebox githubissue" data-onebox-src="https://github.com/pytorch/pytorch/issues/117184">
  <header class="source">

      <a href="https://github.com/pytorch/pytorch/issues/117184" target="_blank" rel="noopener nofollow ugc">github.com/pytorch/pytorch</a>
  </header>

  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewBox="0 0 14 16" aria-hidden="true"><path fill-rule="evenodd" d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/pytorch/issues/117184" target="_blank" rel="noopener nofollow ugc">Weird interaction with p.grad in dynamo</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2024-01-10" data-time="22:22:17" data-timezone="UTC">10:22PM - 10 Jan 24 UTC</span>
      </div>


      <div class="user">
        <a href="https://github.com/janeyx99" target="_blank" rel="noopener nofollow ugc">
          <img alt="janeyx99" src="https://avatars.githubusercontent.com/u/31798555?v=4" class="onebox-avatar-inline" width="20" height="20">
          janeyx99
        </a>
      </div>
    </div>

    <div class="labels">
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          triaged
        </span>
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          oncall: pt2
        </span>
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          module: dynamo
        </span>
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          dynamo-must-fix
        </span>
    </div>
  </div>
</div>

  <div class="github-row">
    <p class="github-body-container">### 🐛 Describe the bug

I encountered this while trying to enable dynamo on unit<span class="show-more-container"><a href="" rel="noopener" class="show-more">…</a></span><span class="excerpt hidden"> tests. Not mentioning requires_grad and then setting the grad of a parameter within a compiled region when the parameter lives outside of that region causes internal dynamo to think that the grad is None.

In my repro below, to_sparse() could be replaced with any function (like .add will produce the same error). 

### Error logs

```
InternalTorchDynamoError: 'NoneType' object has no attribute 'to_sparse'

from user code:
   File "&lt;ipython-input-4-48c0ef0ff017&gt;", line 6, in f
    p.grad = p.grad.to_sparse()

Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information


You can suppress this exception and fall back to eager by setting:
    import torch._dynamo
    torch._dynamo.config.suppress_errors = True
```

### Minified repro

```
import torch
param = torch.rand(2, 3, dtype=torch.float32, device='cuda')  # moving this inside f would work! 

def f(p):
    p.grad = torch.rand_like(p)
    p.grad = p.grad.to_sparse()  # why does dynamo think p.grad at this moment is empty?

compiled_f = torch._dynamo.optimize("eager")(f)
compiled_f(param)
```

### Versions

on main

cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305 @zou3519 @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng</span></p>
  </div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>is the same as yours, you might add a comment to it, or if your issue looks<br>
different, you might want to file a new github issue.</p>
<p>*)  Apparently <code>torch.compile</code> doesn’t work on windows or with python 3.12+.<br>
So I ended up reproducing your issue on linux with pytorch version 2.1.2, the<br>
latest version I happened to have installed with python 3.11.</p>
<p>Best.</p>
<p>K. Frank</p>

Juuso_Korhonen: <p>Late thank you <a class="mention" href="/u/kfrank">@KFrank</a>. I ended up issuing a bug: <a href="https://github.com/pytorch/pytorch/issues/123713" class="inline-onebox" rel="noopener nofollow ugc">.requires_grad_(True) doesn't work with torch.compile · Issue #123713 · pytorch/pytorch · GitHub</a></p>