Torch 1.6.0 RuntimeError: probability tensor contains either `inf`, `nan` or element < 0, But good with Torch 1.1.0

Shizhe_Cai: <p>I’m runing this piece of code:</p>
<pre><code class="lang-auto">    def forward(self, inputs, core_state=()):
        x = inputs["frame"]
        # time x batch x 64 x 64 x 3
        T, B, *_ = x.shape

        # merge time and batch
        # [T*B x 64 x 64 x 3]
        x = torch.flatten(x, 0, 1)

        x = x.float()

        # [T*B x 3 x 64 x 64]
        x = x.transpose(1, 3)

        # x = checkpoint_sequential(self.feat_extract, 2, x)
        x = self.feat_extract(x)
        x = x.view(T*B, -1)

        # core_input = checkpoint_sequential(self.fc, 2, x)
        core_input = self.fc(x)

        core_output = core_input
        core_state = tuple()

        policy_logits = self.policy(core_output)
        baseline = self.baseline(core_output)
        if self.training:
            action = torch.multinomial(F.softmax(policy_logits, dim=1), num_samples=1)
        else:
            action = torch.argmax(policy_logits, dim=1)

        policy_logits = policy_logits.view(T, B, self.num_actions)
        baseline = baseline.view(T, B)
        action = action.view(T, B)

        return dict(policy_logits=policy_logits, baseline=baseline,
                    action=action), core_state
</code></pre>
<p>And got this error:</p>
<blockquote>
<p>Traceback (most recent call last):<br>
File "/usr/local/easybuild-2019/easybuild/software/compiler/gcccore/8.3.0/python/3.7.4/lib/python3$<br>
self.run()<br>
File "/usr/local/easybuild-2019/easybuild/software/compiler/gcccore/8.3.0/python/3.7.4/lib/python3$<br>
self._target(*self._args, **self._kwargs)<br>
File “/data/gpfs/projects/punim1126/CL-RIDE-master-project/src/utils.py”, line 335, in act<br>
raise e<br>
File “/data/gpfs/projects/punim1126/CL-RIDE-master-project/src/utils.py”, line 287, in act<br>
agent_output, agent_state = model(env_output, agent_state)<br>
File “/home/shizhec/.local/lib/python3.7/site-packages/torch/nn/modules/module.py”, line 722, in _$<br>
result = self.forward(*input, **kwargs)<br>
File “/data/gpfs/projects/punim1126/CL-RIDE-master-project/src/models.py”, line 675, in forward<br>
action = torch.multinomial(F.softmax(policy_logits, dim=1), num_samples=1)<br>
RuntimeError: probability tensor contains either <code>inf</code>, <code>nan</code> or element &lt; 0</p>
</blockquote>
<p>I’m using torch 1.6.0, however, when I use torch 1.1.0, I don’t get any error anymore, and I could train the model correctly. Anyone know why this is happening?</p>

iffiX: <p>Output of <code>policy_logits</code>?</p>

Shizhe_Cai: <blockquote>
<p><span class="hashtag-raw">#one</span> output before error:<br>
tensor([[  4.0024, -22.5107,   8.6548, -26.3529, 199.5710, -23.9216, -40.0046,<br>
3.1537, -19.8343, -39.2633,  34.4721,  -7.2311, -56.9415,  -5.0400,<br>
15.7165]])<br>
<span class="hashtag-raw">#where</span> the error occur:<br>
tensor([[nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan]])</p>
</blockquote>

Shizhe_Cai: <p>for torch==1.1.0:<br>
I never got those nan tensors, all process works fine. This is strange.</p>

iffiX: <p>You code seems OK, something might have changed between these two versions, so you can either:</p>
<ol>
<li>Modify you framework, then track and find the origin of the first NaN value</li>
<li>Switch back to 1.1.0, for now.</li>
</ol>
<p>I would recommend you use the second solution, according to your complex situation presented in the previous question.</p>

iffiX: <p>NaN could really happen at anywhere, mainly:</p>
<ol>
<li>division by 0</li>
<li>something involving the annoying log/exp calculation, like log probability, I have just located a Nan problem myself this morning. Eg:</li>
</ol>
<pre><code class="lang-auto">   a=Normal(1, 1e-23)
   a.log_prob(a.sample()) -&gt; NaN because sigma is too small.
</code></pre>

Aayush_Shah: <h2><a name="have-you-tried-increasing-the-temperature-1" class="anchor" href="#have-you-tried-increasing-the-temperature-1"></a><img src="https://discuss.pytorch.org/images/emoji/apple/thermometer.png?v=12" title=":thermometer:" class="emoji" alt=":thermometer:" loading="lazy" width="20" height="20"> Have you tried increasing the temperature?</h2>
<p>Well try increasing the <code>temperature</code> value. I had very low temperature value along with other parameters such as <code>top_k</code> and <code>top_p</code> which made the next token distribution too steep and as the beam search’s logic, you will need to have multiple tokens available, and in the <strong>low temperature</strong> case I couldn’t have (because we know how temperature works, right?)</p>
<blockquote>
<p>So I increased the temperature and it worked.</p>
</blockquote>
<p>Try increasing the temp value and it should just work, if there are no other complexity involved.</p>