Why my model returns nan?

DXZ_999: <p>The model is here:</p>
<pre><code class="lang-auto">class Actor(nn.Module):
    def __init__(self, state_size, action_size, hidden_size=512):
        super(Actor, self).__init__()
        
        self.state_size = state_size
        self.hidden_size = hidden_size
        self.action_size = action_size
        
        self.block_state = nn.Sequential(
            nn.Linear(state_size, hidden_size),
            nn.LayerNorm(hidden_size),
            nn.ReLU(),
        )
        
        self.block_hidden = nn.Sequential(
            nn.Linear(hidden_size, hidden_size),
            nn.LayerNorm(hidden_size),
            nn.ReLU(),
            nn.Linear(hidden_size, hidden_size),
            nn.LayerNorm(hidden_size),
            nn.ReLU(),
        )
        
        self.block_mean = nn.Sequential(
            nn.Linear(hidden_size, action_size),
        )
        
        self.block_std = nn.Sequential(
            nn.Linear(hidden_size, action_size),
        )
        
    def forward(self, state):
        out = self.block_state(state)
        out = self.block_hidden(out)
        mean = self.block_mean(out)
        std = self.block_std(out)
        return mean,std
</code></pre>
<p>The output is:<br>
tensor([[nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,<br>
nan, nan, nan, nan, nan]], grad_fn=)<br>
tensor([[nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,<br>
nan, nan, nan, nan, nan]], grad_fn=)<br>
I’m sure the input doesn’t contain any nan value.</p>

rasbt: <p>There are many potential reasons. Most likely exploding gradients. The two things to try first:</p>
<ol>
<li>
<p>Normalize the inputs</p>
</li>
<li>
<p>Lower the learning rate</p>
</li>
</ol>

DXZ_999: <p>I will try to normalize the inputs. See whether it works.<br>
It can’t be learning rate problem, since it happened at the beginning.</p>

ptrblck: <p>Could you additionally check your input for <code>inf</code> of <code>nan</code> values?</p>

DXZ_999: <p>The input doesn’t contain any nan value. I guess something goes wrong in the block_hidden since both the block_mean and block_std contains nan value.</p>

wangwwno1: <p><a class="mention" href="/u/dxz_999">@DXZ_999</a> <a class="mention" href="/u/rasbt">@rasbt</a><br>
Hello, there is another possibility: If the output contain some large values (abs(value) &gt; 1e20), then nn.LayerNorm(output) might return a all nan vector.</p>
<p>Similiar problem happens in my attention model, I’m pretty sure that it can’t be exploding gradients in my model because:</p>
<ol>
<li>The model can converge (after some iteration, the model loss will be low and stable)</li>
<li>Debug result shows that only a limited number of samples has this problem.</li>
<li>Frequency is so rare that I have to use torch.any(torch.isnan(x)) to catch this bug, and even with this, it require multiple runs to catch one examples.<br>
4.Only intermediate result become nan, input normalization is implemented but problem still exist.</li>
</ol>
<p>My model handle time-series sequence, if there are one vector ‘infected’ with nan, it will propagate and ruin the whole output, so I would like to know whether it is a bug or any solution to address it.</p>

Flock1: <p>This might sound weird but restart your machine. I was facing some issue with the GPU and had to restart the system and to my surprise, it started training.</p>

Ahsan_Habib: <p>I was facing similar issue, and passing each sample through the function <code>torch.nan_to_num()</code> did the trick. I noticed very few samples were having nan.</p>

GoingMyWay: <p>Please check the weights. The weights could be nan!</p>

vincentmichael089: <p>I recommend doing torch.max(your_tensor) and torch.min(your_tensor) to check if any of your tensor is producing “inf”</p>

snknitin: <p>One of your features is probably a very high range value that even after standardized can have underflow or overflow issues , so during your batches it might see a very low value for that feature and adjust a really high weight to that feature and then suddenly some datapoint has a high value and it explodes. If there is one nan in your predictions, your loss turns to nan. it won’t train anymore or update. You can circumvent that in a loss function but that weight will remain high. Delete those unnecessary features that have a really high range for distribution. Scaling or normalizing them might not help</p>

Flock1: <p>Definitely check the weights. I once checked the weights and some of them were 0.0. I added 0.01 to all of them and then it started training.</p>

hitish_singla: <ol>
<li>check if you NAN values your dataset</li>
<li>dont forget to normalize your data</li>
</ol>

Ko_R: <p>if you use<br>
<code>torch.sqrt(x)</code><br>
or any zero-against function<br>
make sure there is no 0 value,<br>
so add a small number is a way to enhance numerical stability<br>
<code>torch.sqrt(x + 1e-8)</code></p>