nn.parameter(requires_grad=False) displayed in summary as Trainable parameter?

Sebastien_Maraux: <p>Hello,</p>
<p>I am building and training a model with a nn parameter set at init with a tensor, which matches a PCA Lowrank output :<br>
self.PCA_matrix_V_Inv = torch.nn.Parameter(PCAMatrix.clone().transpose(0,1), requires_grad=False)</p>
<p>I do mention requires_grad=False, but torch info summary displays that this parameter count as trainable parameters ? I also tried to set required_grad=False on the initial PCAMatrix tensor, but as expected it did not change the behavior (as it should be False by default on Tensor)</p>
<p>As I set the requires_grad at init time of my nn.module model, I don’t get why it is listed as a trainable parameter ?</p>
<p>Any clue about that anyone ?<br>
Thank you</p>

ptrblck: <p>The <code>requires_grad</code> attribute can easily be changed by calling <code>.requires_grad_(True)</code> on the parent module which is why I guess it’s returned as a trainable parameter:</p>
<pre data-code-wrap="python"><code class="lang-python">class MyModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.param = nn.Parameter(torch.randn(1), requires_grad=False)
        
    def forward(self, x):
        x = x * self.param
        return x


model = MyModel()
print(dict(model.named_parameters()))
# {'param': Parameter containing:
# tensor([1.6849])}

print(model.param.requires_grad)
# False

model.requires_grad_(True)
print(model.param.requires_grad)
# True
</code></pre>
<p>If you never want to train it, register the tensor as a buffer.</p>
<p>However, it seems the behavior of <code>torchinfo.summary</code> might also have changed as I cannot reproduce the issue:</p>
<pre data-code-wrap="python"><code class="lang-python">model = MyModel()
print(dict(model.named_parameters()))
# {'param': Parameter containing:
# tensor([1.6849])}

print(model.param.requires_grad)
# False

summary(model)
# =================================================================
# Layer (type:depth-idx)                   Param #
# =================================================================
# MyModel                                  (1)
# =================================================================
# Total params: 1
# Trainable params: 0
# Non-trainable params: 1
# =================================================================

model.requires_grad_(True)
print(model.param.requires_grad)
# True

summary(model)
# =================================================================
# Layer (type:depth-idx)                   Param #
# =================================================================
# MyModel                                  1
# =================================================================
# Total params: 1
# Trainable params: 1
# Non-trainable params: 0
# =================================================================
</code></pre>

Sebastien_Maraux: <aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="213108">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>If you never want to train it, register the tensor as a buffer.</p>
<p>However, it seems the behavior of <code>torchinfo.summary</code> might also have changed as I cannot reproduce the issue:</p>
</blockquote>
</aside>
<p>Thank you for this answer, I will register the PCAMatrix tensor as a buffer, as I don’t expect it to ever be trained. Have a nice day !</p>