Function 'MseLossBackward' returned nan values in its 0th output

digiperson: <p>I am fighting this error for few hours now.<br>
It happens during the first epoch after at about 5 minutes of running of train (about 100 batches). ipdb is not stopping. I checked the inputs, it has no Nans.</p>
<p>This is the loss. I tried to wrap it with ipdb to catch the error, but it is not triggered.</p>
<pre><code class="lang-auto">import ipdb

class RMSLELoss(nn.Module):
    def __init__(self):
        super().__init__()
        self.mse = nn.MSELoss()
        
    def forward(self, pred, actual):
        #print(pred,actual)
        res = torch.sqrt(self.mse(pred,actual))
        if torch.isnan(pred).sum().item()&gt;0:
            ipdb.set_trace()
        if torch.isnan(res).sum().item()&gt;0:
            ipdb.set_trace()
        return res

criterion = RMSLELoss()
</code></pre>
<p>I think that after a few iterations I am getting a float(-inf) somewhere, but I am outputting the weights and to weights&amp;biases and they seems normal.<br>
If I understand correctly, the error happens in the backward pass, so how can I debug that?</p>
<p>It seems that this issue doesn’t happen when running with less data. I run 10 epochs with 5000 samples, and 20 epochs with 500 samples. Both of them seems ok, and the loss is reducing for train and validation</p>
<p>The model is a combination of CNN and a categorical features. I normalized both the images and the features</p>
<p>See below the weights and biases from the layers.</p>
<pre><code class="lang-auto">{
                    'epoch': epoch+1,
                    'step': step ,
                    'data0' : data[0].sum(),
                    'data1' : data[1].sum(),
                    
                    'cc1w' : model.conv1.weight.sum().detach().cpu().item(),
                    'cc2w' : model.conv2.weight.sum().detach().cpu().item(),
                    'cc3w' : model.conv3.weight.sum().detach().cpu().item(),
                    'cc4w' : model.conv4.weight.sum().detach().cpu().item(),

                    'cc1b' : model.conv1.bias.sum().detach().cpu().item(),
                    'cc2b' : model.conv2.bias.sum().detach().cpu().item(),
                    'cc3b' : model.conv3.bias.sum().detach().cpu().item(),
                    'cc4b' : model.conv4.bias.sum().detach().cpu().item(),

                    
                    'fc1w' : model.fc1.weight.sum().detach().cpu().item(),
                    'fc1b' : model.fc1.bias.sum().detach().cpu().item(),
                    'fc2w' : model.fc2.weight.sum().detach().cpu().item(),
                    'fc2b' : model.fc2.bias.sum().detach().cpu().item(),
                    'fc3w' : model.fc3.weight.sum().detach().cpu().item(),
                    'fc3b' : model.fc3.bias.sum().detach().cpu().item(),
                    'fc4w' : model.fc4.weight.sum().detach().cpu().item(),
                    'fc4b' : model.fc4.bias.sum().detach().cpu().item(),
                  } 
</code></pre>
<p>How do you suggest me to continue from here?</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/a/8/a849745494203dcefdec925682a65cdf413770d8.png" data-download-href="https://discuss.pytorch.org/uploads/default/a849745494203dcefdec925682a65cdf413770d8" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/a/8/a849745494203dcefdec925682a65cdf413770d8_2_294x500.png" alt="image" data-base62-sha1="o0JABtEGT5hxvUXH7DdhpULx1K8" width="294" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/a/8/a849745494203dcefdec925682a65cdf413770d8_2_294x500.png, https://discuss.pytorch.org/uploads/default/optimized/3X/a/8/a849745494203dcefdec925682a65cdf413770d8_2_441x750.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/a/8/a849745494203dcefdec925682a65cdf413770d8_2_588x1000.png 2x" data-dominant-color="FBFCFD"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1059×1799 185 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>Your loss might get a zero value, which would create an Inf or NaN gradient:</p>
<pre><code class="lang-python">x = torch.randn(1, 1, requires_grad=True)
y = torch.zeros(1, 1)
criterion = nn.MSELoss()

loss = torch.sqrt(criterion(x * 0, y))
loss.backward()
print(x.grad)
&gt; tensor([[nan]])
</code></pre>
<p>You could add a small <code>eps</code> value to the <code>torch.sqrt</code> calculation to avoid it.</p>

kaddyp: <p>I am getting the same issue RuntimeError: Function ‘MseLossBackward0’ returned nan values in its 0th output.</p>
<p>I am following this code I found on github for SAC in RL on a custom environment I have. I checked the data inputs and everything seems to be working fine in terms of the inputs. The output from value_loss is coming out to be nan after some epochs and it is random everytime I train the environment</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/7/87a4909a600d9b8bad49c524ec581f36afed7534.png" data-download-href="https://discuss.pytorch.org/uploads/default/87a4909a600d9b8bad49c524ec581f36afed7534" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/8/7/87a4909a600d9b8bad49c524ec581f36afed7534_2_668x500.png" alt="image" data-base62-sha1="jlX2iz1czz4RW1nEzyrFaP9C2tC" width="668" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/8/7/87a4909a600d9b8bad49c524ec581f36afed7534_2_668x500.png, https://discuss.pytorch.org/uploads/default/original/3X/8/7/87a4909a600d9b8bad49c524ec581f36afed7534.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/8/7/87a4909a600d9b8bad49c524ec581f36afed7534.png 2x" data-dominant-color="27292A"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">846×633 94 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

qq-me: <aside class="quote no-group" data-username="kaddyp" data-post="3" data-topic="94875">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/5f8ce5/48.png" class="avatar"> kaddyp:</div>
<blockquote>
<p>I am getting the same issue RuntimeError: Function ‘MseLossBackward0’ returned nan values in its 0th output.</p>
<p>I am following this code I found on github for SAC in RL on a custom environment I have. I checked the data inputs and everything seems to be working fine in terms of the inputs. The output from value_loss is coming out to be nan after some epochs and it is random everytime I train the environment</p>
</blockquote>
</aside>
<p>I had the same issue for a different reason, if you have GTX 1650, it is bugged with float16 precision and will generate NaNs. If that happens to be the issue, use float32</p>

kaddyp: <p>I have a 4070, and I am using float32. The thing that is weird is it works completely fine with the environment the guy had in his code. I tried it with couple of different continuous environments from gym and it works fine. For some reason it only does that sometimes when I try to train it on my custom env.</p>