Model weights not being updated

miguelvr: <p>Hey again,</p>
<p>I’m currently developing a transversal machine learning tool that is able to support multiple ML frameworks and therefore I’m doing things a little differently when compared to the regular pytorch workflow.</p>
<p>My model inherits from <code>nn.Module</code> and has the regular <code>init</code> and <code>forward</code> methods. However, to fit the framework, I had to add an <code>update</code> method that calls the <code>forward</code>, computes the loss, calls <code>loss.backward()</code> and calls <code>optimizer.step</code>.</p>
<p>This is the current <code>update</code> bit from my code:</p>
<pre><code>def update(self, input=None, output=None):

    # input_source, input_target, tags, input_editor=None, input_client=None

    input_source = input['source']
    input_target = input['target']
    input_editor = input['editor']
    input_client = input['client']
    tags = output['tags']

    inputs = [input_source, input_target]
    if input_editor is not None:
        inputs += [input_editor]
    if input_client is not None:
        inputs += [input_client]

    if self.cuda:
        inputs = [Variable(inpt.cuda()) for inpt in inputs]
        tags = Variable(tags.cuda())
    else:
        inputs = [Variable(inpt) for inpt in inputs]
        tags = Variable(tags)

    self.optimizer.zero_grad()
    out = self.forward(*inputs)
    loss = self.loss_fn(out, tags)
    loss.backward()
    self.optimizer.step()

    return loss
</code></pre>
<p>Everything is working fine, EXCEPT the update bit of the weights. The <code>update</code> method is being called in a train_loop function that calls <code>model.update(**data)</code>. I’m printing the losses from every epoch and the values are exactly the same (with different types of data).</p>
<p>Anyone has a clue of what might be wrong?</p>

fmassa: <p>Try seeing if <code>model.parameters()</code> gives you all the parameters you are expecting. Maybe you have defined some of the parameters as <code>Variable</code> instead of <code>nn.Parameter</code>, so the logic behind <code>parameters()</code> can’t see your weights.</p>

miguelvr: <p>The custom modules my model uses are only for shape manipulation, everything else is native torch, so I guess there’s no need for nn.Parameter…</p>

fmassa: <p>Could you try seeing if <code>list(model.parameters())</code> prints all the parameters of your model?<br>
Also, verify that you are not setting <code>requires_grad=True</code> to all parameters of your network, as it would avoid backprop through the network</p>

miguelvr: <p>Every parameter seem to be in the parameter pool…</p>
<p>The problem seems to be the <code>loss.backward()</code> and <code>optimizer.step()</code> calls that are not updating the weights.</p>
<p>While debugging I did:</p>
<pre><code>a = list(self.parameters())[0]
loss.backward()
self.optimizer.step()
b = list(self.parameters())[0]
torch.equal(a.data, b.data)
</code></pre>
<p>And it returned <code>True</code> so the parameter is not being updated at all. I don’t have a clue of what may be wrong with my code anymore. Here’s a small reproducible example of my code:</p>
<p><a href="https://gist.github.com/miguelvr/251903423fed3b64d320b53a847aaa75" class="onebox" target="_blank" rel="nofollow noopener">https://gist.github.com/miguelvr/251903423fed3b64d320b53a847aaa75</a></p>

fmassa: <p>The example that you showed has a problem with it.<br>
Indeed, when you do <code>list(model.parameters())[0]</code>, you are taking a reference of the tensor.<br>
Try adding a <code>clone()</code> instead</p>
<pre><code class="lang-python">a = list(self.parameters())[0].clone()
loss.backward()
self.optimizer.step()
b = list(self.parameters())[0].clone()
torch.equal(a.data, b.data)
</code></pre>

miguelvr: <p>Thanks for the correction. However, it still returns <code>True</code>, so the weights are definitely not being updated…</p>

miguelvr: <p>So… I tried to remove the last linear layer with the softmax and changed the BCELoss to MSELoss and the weights are being updated!</p>
<p>I’m guessing this is a problem from BCELoss and not from my code… Could you check this out <a class="mention" href="/u/smth">@smth</a> <a class="mention" href="/u/apaszke">@apaszke</a> ?</p>

apaszke: <p>Did you remove your custom modules too? I guess you must have repacked the data into a new Variables somewhere, thus breaking the history graph. You might want to check if <code>list(model.parameters())[0].grad</code> is not <code>None</code>. If it is, you must be breaking the graph somewhere (you should never need to take the <code>.data</code> do some op, and wrap it in a Variable).</p>

miguelvr: <p>It is not <code>None</code> it is a full zero matrix!! And that explains why the weights are not being updated…</p>
<p>But I checked again and this only happens because I use BCELoss AND a softmax activation for the output! With a sigmoid activation, the weights are updated…</p>
<p>Am I doing something wrong? I’m pretty much recreating some other code I had in Keras where I used a categorical cross entropy loss (with one-hot encodings) and a final softmax activation. In pytorch I’m not doing one-hot encodings and I’m using BCELoss.</p>

AdityaAS: <p>Hey, I have the exact same error!</p>
<ol>
<li>My model doesn’t seem to be training.</li>
<li>Upon checking a = list(model.parameters())[0].clone() and b = a = list(model.parameters())[0].clone() before and after the call to loss.backward() and optimizer.step(). a==b returns false</li>
<li>Upon printing list(model.parameters())[0].grad it returns a matrix of all 0’s.</li>
</ol>
<p>I’ve been struggling with this problem for 3 days now. Please help me out. The original question can be found at <a href="https://discuss.pytorch.org/t/lstm-model-not-training/8093">LSTM Model not training</a></p>
<p><a class="mention" href="/u/smth">@smth</a> <a class="mention" href="/u/apaszke">@apaszke</a> <a class="mention" href="/u/fmassa">@fmassa</a> please take a look at this. I can share the complete code if I am missing out on providing some details now.</p>

jiacheng-xu: <p>Hi all. I get the similar problem. My bug is probably that I use the wrong combination of Softmax and Loss function, so value of grads are super small.</p>
<p>For me,<br>
My model doesn’t seem to be training.<br>
Upon checking a = list(model.parameters())[0].clone() and b = a = list(model.parameters())[0].clone() before and after the call to loss.backward() and optimizer.step(). a==b returns false<br>
Upon printing list(model.parameters())[0].grad it returns a matrix of all super small number like in order of 10^-8.</p>

TahaMagdy: <p>Hello There!</p>
<p>What do you mean by the following?</p>
<aside class="quote no-group" data-username="jiacheng-xu" data-post="12" data-topic="1842">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/j/e47774/48.png" class="avatar"> jiacheng-xu:</div>
<blockquote>
<p>I use the wrong combination of Softmax and Loss function, so value of grads are super small.</p>
</blockquote>
</aside>
<p>I don’t understand why the gradients are super small in your case.</p>

Gkv: <p>Hi, I am also facing the same problem. But in my case <strong>list(model.parameters())[0].grad is None</strong>. How can I find out the mistake? Any suggestions?</p>
<p>Thanks</p>

Yuanxing_Liu: <p>In my case, it was because the Softmax function calculate the zero value tensor. So the NLL loss was unchanging and the weights were not being updated.</p>

RGaonkar: <p>I am also dealing with the same problem. I get p.grad is None for all the parameters in the network. How do I check if the computation graph is broken somewhere?</p>

minhmanho: <p><a class="mention" href="/u/gkv">@Gkv</a> <a class="mention" href="/u/rgaonkar">@RGaonkar</a> I have the same problem here.<br>
My weights never be updated and  my model <strong>list(model.parameters())[0].grad is None</strong> . And my Network didn’t return any grad_fn in tensor.<br>
It turns out I have to return the function instead of variable. I don’t know it is correct or not but it worked for me.<br>
Bug source code</p>
<pre><code class="lang-auto">class Model(nn.Module):
    def __init__(self, size_in, size_out):
        super(Model, self).__init__()
        self.ReLU1 = nn.ReLU()
        self.conv1 = nn.Conv2d(in_channels=size_in, out_channels=size_out,
                               kernel_size=(3, 3),
                               stride=(1, 1),
                               padding=(1, 1),
                               dilation=1,
                               groups=1,
                               bias=False)

    def forward(self, x):
        x = self.ReLU1(x)
        x = self.conv1(x)
        return x
</code></pre>
<p>You guys could try this:</p>
<pre><code class="lang-auto">class Model(nn.Module):
    def __init__(self, size_in, size_out):
        super(Model, self).__init__()
        self.ReLU1 = nn.ReLU()
        self.conv1 = nn.Conv2d(in_channels=size_in, out_channels=size_out,
                               kernel_size=(3, 3),
                               stride=(1, 1),
                               padding=(1, 1),
                               dilation=1,
                               groups=1,
                               bias=False)

    def forward(self, x):
        x = self.ReLU1(x)
        return self.conv1(x)
</code></pre>

Shivam_Chandhok: <aside class="quote no-group" data-username="fmassa" data-post="4" data-topic="1842">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/fmassa/48/58_2.png" class="avatar"> fmassa:</div>
<blockquote>
<p>list(model.parameters())</p>
</blockquote>
</aside>
<p>what do you mean by<br>
verify that you are not setting  <code>requires_grad=True</code>  to all parameters of your network, as it would avoid backprop through the network</p>

Zafar: <p><strong>Ignore this message. The problem was that I had too shallow of a network! Sorry about the noise! It converged after I added a couple of conv layers.</strong></p>
<blockquote>
<p>I am having a very similar issue. Initially, I had a 3 conv+relu+maxpool followed by 2 linears + relu and sigmoid for binary classification. It would not learn – all the parameters would be set to something very close to 0, and the output would be “random”. I reduced the size of the network to Conv/relu + Linear/relu + Linear/sigmoid =&gt; same problem.</p>
</blockquote>

Shanya_Anand: <p>similar problem faced by me as I was using relu in the layer and then apply softmax on the layer and then cross entropy so make sure that last layer should not contain any activation function if you are using softmax</p>