Loss.backward :Grad is None

username: xinbai
cooked: <p>I bulit a LSTM net work,and used nn.MSELoss().But it returned 0.I don’t know why made it return 0.I wish for help.</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn

import numpy as np
import matplotlib.pyplot as plt
import csv

from torch.autograd import Variable

</code></pre>
<pre><code class="lang-auto">closeSet=[];

with open("data.csv",'r') as csvfile:
    reader=csv.reader(csvfile)

    n=-1;
    for row in reader :
        
        n=n+1;
        
        if(n == 0):
            continue;
        
        closeSet.append(float(row[6]));
</code></pre>
<pre><code class="lang-auto">closeSet=torch.Tensor(closeSet);

average=closeSet.mean();
print(average);

still=closeSet.std();
print(still);

closeSet=(closeSet-average)/still;
print(closeSet)

</code></pre>
<pre><code class="lang-auto">train_data=closeSet[:150];
</code></pre>
<pre><code class="lang-auto">class LSTM(nn.Module):
    def __init__(self,input_size,hidden_size,num_layer,output_size):
        super(LSTM, self).__init__()
        
        self.input_size=input_size;
        self.hidden_size=hidden_size;
        self.num_layer=num_layer;
        self.output_size=output_size;
        
        self.Wfh,self.Wfx,self.bf=self.init_Weight_bias_gate();
        self.Wih,self.Wix,self.bi=self.init_Weight_bias_gate();
        self.Woh,self.Wox,self.bo=self.init_Weight_bias_gate();
        self.Wch,self.Wcx,self.bc=self.init_Weight_bias_gate();
        self.Wy=nn.Parameter(torch.randn(self.hidden_size,self.output_size,requires_grad=True));
        self.by=nn.Parameter(torch.randn(self.output_size,requires_grad=True));
        
        self.hList=[];
        self.cList=[];

        self.times=0;
        
        self.f_=torch.zeros(1,self.hidden_size,requires_grad=True);
        self.i_=torch.zeros(1,self.hidden_size,requires_grad=True);
        self.o_=torch.zeros(1,self.hidden_size,requires_grad=True);
        self.ct_=torch.zeros(1,self.hidden_size,requires_grad=True);
        self.h_=torch.zeros(1,self.hidden_size,requires_grad=True);
        self.c_=torch.zeros(1,self.hidden_size,requires_grad=True);
        
        self.y_=torch.zeros(1,self.hidden_size,requires_grad=True);

        self.hList.append(self.h_);
        self.cList.append(self.c_);

    def init_Weight_bias_gate(self):
        return (nn.Parameter(torch.randn(self.hidden_size,self.hidden_size),requires_grad=True),
                nn.Parameter(torch.randn(self.input_size,self.hidden_size),requires_grad=True),
                nn.Parameter(torch.randn(self.hidden_size),requires_grad=True))
                
    def forward(self,x):
        self.times+=1;

        self.f_=torch.sigmoid(self.hList[-1] @ self.Wfh + x @ self.Wfx + self.bf);
        
        self.i_=torch.sigmoid(self.hList[-1] @ self.Wih + x @ self.Wix + self.bi);
        
        self.ct_=torch.tanh(self.hList[-1] @ self.Wch + x @ self.Wcx + self.bc);

        self.o_=torch.sigmoid(self.hList[-1] @ self.Woh + x @ self.Wox + self.bo);
        
        self.c_=self.f_ * self.cList[-1] + self.i_ * self.ct_;
        
        self.h_=self.o_ * torch.tanh(self.c_);
        self.y_=self.hList[-1] @ self.Wy + self.by;
        
        self.f_.requires_grad_(True);
        self.i_.requires_grad_(True);
        self.ct_.requires_grad_(True);
        self.o_.requires_grad_(True);
        self.c_.requires_grad_(True);
        self.h_.requires_grad_(True);
        
        self.y_.requires_grad_(True);
        
        self.cList.append(self.c_);
        self.hList.append(self.h_);
        
        return self.y_;
    
    def reset(self):
        self.times=0;
        
        self.hList=[torch.zeros(1,self.hidden_size,requires_grad=True),];
        self.cList=[torch.zeros(1,self.hidden_size,requires_grad=True),];
</code></pre>
<pre><code class="lang-auto">l=LSTM(150,1,1,150);

criterion = nn.MSELoss();

optimizer = torch.optim.Adam(l.parameters(), lr=0.01)
</code></pre>
<pre><code class="lang-auto">for i in range(10):
    x=train_data;
    x=x.clone().detach().unsqueeze(0);
    x.requires_grad_(True)
    
    y_output=l.forward(x);
    
    y_true=train_data;
    y_true=y_true.clone().detach();
    y_true.requires_grad_(True)
    
    loss=criterion(l.forward(x), y_true);
    
    optimizer.zero_grad();
    
    loss.backward();
    optimizer.step();
    
    print("Wfh.grad:");
    print(l.Wfh.grad)
    
    l.reset();

    lossList.append(loss.item())
</code></pre>

username: ptrblck
cooked: <p>I cannot reproduce the issue and get a valid gradient for <code>l.Wfh.grad</code>:</p>
<pre><code class="lang-python">l=LSTM(150,1,1,150);
criterion = nn.MSELoss();
optimizer = torch.optim.Adam(l.parameters(), lr=0.01)
x = torch.randn(10, 10, 150)
y_output=l.forward(x);

y_true = torch.randn_like(y_output)
loss=criterion(l.forward(x), y_true);

optimizer.zero_grad();
loss.backward();
optimizer.step();

print("Wfh.grad:");
print(l.Wfh.grad)
# tensor([[0.0018]])
</code></pre>
<p>However, your code has a few issues:</p>
<ul>
<li>
<code>Variable</code>s are deprecated since PyTorch <code>0.4</code> so remove their usage and use tensors directly instead,</li>
<li>I don’t know why you would need to call <code>.requires_grad_(True)</code> on parameters inside the <code>forward</code> method,</li>
<li>your output seems to change in shape depending on the iteration, which I’m also unsure if this is desired,</li>
<li>don’t call the <code>model.forward</code> method as it will skip calling into potentially registered hooks, but call the model directly instead: <code>model(input)</code>.</li>
</ul>

username: xinbai
cooked: <p>Hello,thank you for your help.</p>
<p>I had deleted <code>.requires_grad_(True)</code> on parameters inside the <code>forward</code> method and used <code>model(input)</code>.But excpet for the gradient for Wix,the gradient for bi,the gradient for Wy and by,ohter parameter’s gradients are still 0.</p>
<p>I don’t know what lead to the result.And I can’t understand your third suggestion.I tried to output the shape of y_,it is torch.Size([1, 150]).Can you tell me in detail?</p>
<pre><code class="lang-auto">class LSTM(nn.Module):
    def __init__(self,input_size,hidden_size,num_layer,output_size):
        super(LSTM, self).__init__()

        self.input_size=input_size;
        self.hidden_size=hidden_size;
        self.num_layer=num_layer;
        self.output_size=output_size;
        
        self.Wfh,self.Wfx,self.bf=self.init_Weight_bias_gate();
        self.Wih,self.Wix,self.bi=self.init_Weight_bias_gate();
        self.Woh,self.Wox,self.bo=self.init_Weight_bias_gate();
        self.Wch,self.Wcx,self.bc=self.init_Weight_bias_gate();
        
        self.Wy=nn.Parameter(torch.randn(self.hidden_size,self.output_size));
        self.by=nn.Parameter(torch.randn(self.output_size));
        
        self.hList=[];
        self.cList=[];
        
        self.times=0;
        
        self.f_=torch.zeros(1,self.hidden_size,requires_grad=True);
        self.i_=torch.zeros(1,self.hidden_size,requires_grad=True);
        self.o_=torch.zeros(1,self.hidden_size,requires_grad=True);
        self.ct_=torch.zeros(1,self.hidden_size,requires_grad=True);
        self.h_=torch.zeros(1,self.hidden_size,requires_grad=True);
        self.c_=torch.zeros(1,self.hidden_size,requires_grad=True);
        
        self.y_=torch.zeros(1,self.hidden_size,requires_grad=True);
  
        self.hList.append(self.h_);
        self.cList.append(self.c_);
        
    def init_Weight_bias_gate(self):
        return (nn.Parameter(torch.randn(self.hidden_size,self.hidden_size)),
                nn.Parameter(torch.randn(self.input_size,self.hidden_size)),
                nn.Parameter(torch.randn(self.hidden_size)))
                
    def forward(self,x):
        self.times+=1;
        
        self.f_=torch.sigmoid(self.hList[-1] @ self.Wfh + x @ self.Wfx + self.bf);
        
        self.i_=torch.sigmoid(self.hList[-1] @ self.Wih + x @ self.Wix + self.bi);
        
        self.ct_=torch.tanh(self.hList[-1] @ self.Wch + x @ self.Wcx + self.bc);
        
        self.o_=torch.sigmoid(self.hList[-1] @ self.Woh + x @ self.Wox + self.bo);
        
        self.c_=self.f_ * self.cList[-1] + self.i_ * self.ct_;
        
        self.h_=self.o_ * torch.tanh(self.c_);
        
        self.y_=self.hList[-1] @ self.Wy + self.by;
        
        self.cList.append(self.c_);
        self.hList.append(self.h_);
        
        print("self.y_:");
        print(self.y_.shape);
        
        return self.y_;
    
    def reset(self):
        self.times=0;
        
        self.hList=[torch.zeros(1,self.hidden_size,requires_grad=True),];
        self.cList=[torch.zeros(1,self.hidden_size,requires_grad=True),];
</code></pre>
<pre><code class="lang-auto">l=LSTM(150,1,1,150);

criterion = nn.MSELoss();

optimizer = torch.optim.Adam(l.parameters(), lr=0.01)

</code></pre>
<pre><code class="lang-auto">for i in range(10):
    x=train_data;
    x=x.clone().detach().unsqueeze(0);
    
    x.requires_grad_(True)   
    
    y_output=l.forward(x);
    
    y_true=train_data;
    y_true=y_true.clone().detach();
    
    y_true.requires_grad_(True)
   
    loss=criterion(l(x), y_true);
    
    optimizer.zero_grad();
    
    loss.backward();
    optimizer.step();
    
    for name, parms in l.named_parameters():
            print('--&gt;name:', name, '--&gt;grad_requirs:', parms.requires_grad, \
                 ' --&gt;grad_value:', parms.grad, '--&gt;value:', parms)

    l.reset();

    lossList.append(loss.item())
</code></pre>

username: ptrblck
cooked: <aside class="quote no-group" data-username="xinbai" data-post="3" data-topic="176813">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/xinbai/48/59178_2.png" class="avatar"> xinbai:</div>
<blockquote>
<p>But excpet for the gradient for Wix,the gradient for bi,the gradient for Wy and by,ohter parameter’s gradients are still 0.</p>
</blockquote>
</aside>
<p>I still cannot reproduce the issue, so it would be great if you could post a minimal and executable code snippet which would show the issue.</p>
<aside class="quote no-group" data-username="xinbai" data-post="3" data-topic="176813">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/xinbai/48/59178_2.png" class="avatar"> xinbai:</div>
<blockquote>
<p>And I can’t understand your third suggestion.I tried to output the shape of y_,it is torch.Size([1, 150]).Can you tell me in detail?</p>
</blockquote>
</aside>
<p>In this code you can see that the output shape changes in the second iteration:</p>
<pre data-code-wrap="python"><code class="lang-python">l=LSTM(150,1,1,150);
criterion = nn.MSELoss();
optimizer = torch.optim.Adam(l.parameters(), lr=0.01)

x = torch.randn(10, 10, 150)
y_output = l(x);
print(y_output.shape)
# torch.Size([1, 150])

y_output = l(x);
print(y_output.shape)
# torch.Size([10, 10, 150])
</code></pre>
<p>which seems at least uncommon.</p>
<p>The rest of the code also shows valid gradients:</p>
<pre data-code-wrap="python"><code class="lang-python">y_true = torch.randn_like(y_output)
loss=criterion(l(x), y_true);

optimizer.zero_grad();
loss.backward();
optimizer.step();

print("Wfh.grad:");
print(l.Wfh.grad)
# tensor([[0.0018]])

for name, parms in l.named_parameters():
        print('--&gt;name:', name, '--&gt;grad_requirs:', parms.requires_grad, \
             ' --&gt;grad_value:', parms.grad.abs().sum(), '--&gt;value:', parms.abs().sum())
# --&gt;name: Wfh --&gt;grad_requirs: True  --&gt;grad_value: tensor(0.0001) --&gt;value: tensor(0.8921, grad_fn=&lt;SumBackward0&gt;)
# --&gt;name: Wfx --&gt;grad_requirs: True  --&gt;grad_value: tensor(0.0883) --&gt;value: tensor(121.5771, grad_fn=&lt;SumBackward0&gt;)
# --&gt;name: bf --&gt;grad_requirs: True  --&gt;grad_value: tensor(0.0015) --&gt;value: tensor(1.7418, grad_fn=&lt;SumBackward0&gt;)
# --&gt;name: Wih --&gt;grad_requirs: True  --&gt;grad_value: tensor(1.4800e-05) --&gt;value: tensor(0.3881, grad_fn=&lt;SumBackward0&gt;)
# --&gt;name: Wix --&gt;grad_requirs: True  --&gt;grad_value: tensor(0.5420) --&gt;value: tensor(120.5190, grad_fn=&lt;SumBackward0&gt;)
# --&gt;name: bi --&gt;grad_requirs: True  --&gt;grad_value: tensor(0.0131) --&gt;value: tensor(0.3184, grad_fn=&lt;SumBackward0&gt;)
# --&gt;name: Woh --&gt;grad_requirs: True  --&gt;grad_value: tensor(0.0020) --&gt;value: tensor(0.9126, grad_fn=&lt;SumBackward0&gt;)
# --&gt;name: Wox --&gt;grad_requirs: True  --&gt;grad_value: tensor(0.6173) --&gt;value: tensor(136.6227, grad_fn=&lt;SumBackward0&gt;)
# --&gt;name: bo --&gt;grad_requirs: True  --&gt;grad_value: tensor(0.0181) --&gt;value: tensor(1.3139, grad_fn=&lt;SumBackward0&gt;)
# --&gt;name: Wch --&gt;grad_requirs: True  --&gt;grad_value: tensor(0.0004) --&gt;value: tensor(1.6613, grad_fn=&lt;SumBackward0&gt;)
# --&gt;name: Wcx --&gt;grad_requirs: True  --&gt;grad_value: tensor(0.7321) --&gt;value: tensor(134.0075, grad_fn=&lt;SumBackward0&gt;)
# --&gt;name: bc --&gt;grad_requirs: True  --&gt;grad_value: tensor(0.0076) --&gt;value: tensor(1.8784, grad_fn=&lt;SumBackward0&gt;)
# --&gt;name: Wy --&gt;grad_requirs: True  --&gt;grad_value: tensor(0.2735) --&gt;value: tensor(103.7549, grad_fn=&lt;SumBackward0&gt;)
# --&gt;name: by --&gt;grad_requirs: True  --&gt;grad_value: tensor(1.5557) --&gt;value: tensor(113.4924, grad_fn=&lt;SumBackward0&gt;)
</code></pre>

username: xinbai
cooked: <p>Sorry for the late reply.I have found the error.<br>
When I replace the following code</p>
<pre><code class="lang-auto">def init_Weight_bias_gate(self):
        return (nn.Parameter(torch.randn(self.hidden_size,self.hidden_size),requires_grad=True),
                nn.Parameter(torch.randn(self.input_size,self.hidden_size),requires_grad=True),
                nn.Parameter(torch.randn(self.hidden_size),requires_grad=True))
</code></pre>
<p>with the following code</p>
<pre><code class="lang-auto">def init_Weight_bias_gate(self):
        return (nn.Parameter(torch.randn(self.hidden_size,self.hidden_size) * 0.01),
                nn.Parameter(torch.randn(self.input_size,self.hidden_size) * 0.01),
                nn.Parameter(torch.randn(self.hidden_size) * 0.01))
</code></pre>
<p>The problem is solved.<br>
Looks like a problem with random initialization parameters.</p>
<p>Thank you for your help.</p>