Loss and model parameters requires_grad= False, and grad=non

username: NGH
cooked: <p>This is the definition of my model:</p>
<pre><code class="lang-auto">class MyClassifier(nn.Module):
    def __init__(self,args):
        super(MyClassifier, self).__init__()
        self.avgpool = nn.AdaptiveAvgPool2d(output_size=(1, 1))
        self.fc1= nn.Linear(in_features= args.input_features, out_features=args.hidden_size, bias=True)
        if  args.classifier_act == 'Tanh':
            self.act = nn.Tanh()
        else:
            self.act = nn.Sigmoid()
        self.fc2 = nn.Linear(args.hidden_size,args.num_classes)
    def forward(self, input):
        x= self.avgpool(input)
        x = x.view(x.size(0), -1)
 x= self.act(self.fc1(x))
        x= self.fc2(x)
return x


</code></pre>
<pre><code>        for batch, (lr, hr,rcan,hran,srfbn,csnln,gmfn,drln,rnan,mardn,label,filename,idx_scale) in enumerate(self.loader_train):
            lr= self.prepare([lr])
            self.optimizer.zero_grad()
            self.scores = self.model(input, idx_scale)
            self.scores = torch.sigmoid(self.scores)
            loss = self.loss(self.scores , label)
           ` loss = Variable(loss, requires_grad=True)`
			loss.backward()
            self.optimizer.step()
            print(loss.grad)
			for name, param in self.model.named_parameters():
                    if param.grad is not None:
                        print(name, param.grad.sum())
                    else:
                        print(name, param.grad)
</code></pre>
<p>The loss and output of the model required_grad are false.<br>
scores required true: False<br>
loss required true: False</p>
<p>I put the loss in Variable as follow:<br>
loss = Variable(loss, requires_grad=True)<br>
But still, the grad is non and my model is not training.<br>
tensor(1., device=‘cuda:0’)<br>
model.fc1.weight None<br>
model.fc1.bias None<br>
model.fc2.weight None<br>
model.fc2.bias None<br>
I have no idea why this happens.</p>

username: KFrank
cooked: <p>Hi Ngh!</p>
<aside class="quote no-group quote-modified" data-username="NGH" data-post="1" data-topic="107735" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ngh/48/26804_2.png" class="avatar"> NGH:</div>
<blockquote>
<p>This is the definition of my model:<br>
…<br>
I have no idea why this happens.</p>
</blockquote>
</aside>
<p>It’s hard to tell what is going on here.</p>
<p>You haven’t posted all of your code, and the code you posted is<br>
garbled by the forum.</p>
<p>Please post a trimmed-down, complete, runnable script that illustrates<br>
your issue, and please enclose your code in a triple-backtick (```) code<br>
block so that the forum will format it nicely.</p>
<p>Having said that, here is a simplified version of your script that shows<br>
a non-trivial <code>weight</code> and <code>bias</code> for your <code>Linear</code> layers and an <code>output</code><br>
and <code>loss</code> for which <code>requires_grad = True</code>.  Maybe something in it<br>
will help you find your problem.</p>
<p>The script:</p>
<pre data-code-wrap="python"><code class="lang-python">import torch
import torch.nn as nn

print ('torch.__version__ =', torch.__version__)

class SomeArgs:
    pass

args = SomeArgs()
args.input_features = 3
args.hidden_size = 5
args.classifier_act = None
args.num_classes = 4

class MyClassifier(nn.Module):
    def __init__ (self, args):
        super (MyClassifier, self).__init__()
        self.fc1= nn.Linear(in_features= args.input_features, out_features=args.hidden_size, bias=True)
        if args.classifier_act == 'Tanh':
            self.act = nn.Tanh()
        else:
            self.act = nn.Sigmoid()
        self.fc2 = nn.Linear(args.hidden_size,args.num_classes)
    def forward(self, input):
        x = input
        x= self.act(self.fc1(x))
        x= self.fc2(x)
        return x

model = MyClassifier (args)

print ('model.fc1 = ...\n', model.fc1)
print ('model.fc1.weight = ...\n', model.fc1.weight)
print ('model.fc1.bias = ...\n', model.fc1.bias)
input = torch.randn ((1, 3, 3))
output = model (input)
print ('output.requires_grad =', output.requires_grad)
loss = output.sum()
print ('loss.requires_grad =', loss.requires_grad)
</code></pre>
<p>And its output:</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; exec (open ('./ngh.py').read())
torch.__version__ = 1.6.0
model.fc1 = ...
 Linear(in_features=3, out_features=5, bias=True)
model.fc1.weight = ...
 Parameter containing:
tensor([[ 0.4360, -0.3469,  0.2549],
        [-0.4405, -0.0466, -0.3481],
        [-0.3794,  0.1865, -0.3516],
        [-0.4399, -0.3565,  0.0524],
        [-0.4555, -0.1364, -0.1984]], requires_grad=True)
model.fc1.bias = ...
 Parameter containing:
tensor([ 0.4207,  0.4298,  0.0848, -0.0472,  0.2806], requires_grad=True)
output.requires_grad = True
loss.requires_grad = True
</code></pre>
<p>Good luck.</p>
<p>K. Frank</p>

username: NGH
cooked: <p>Hi Frank,</p>
<p>Thanks for your help.</p>