RuntimeError: invalid multinomial distribution (encountering probability entry < 0)

username: reddymap
cooked: <p>Hi</p>
<p>I am working on an environment in Reinforcement Learning based which is based on Graph network. I am trying to implement the Actor-Critic network. Here I have two agents which are following the policy. I encounter this error (code is shown below with error) most of the time when I run the model for a certain amount of episodes with specific learning rates. Very few times the program executed successfully. But lot of times it throws an error stating ‘invalid multinomial distribution’. I am using Categorical Distribution and I dont think I have any values with probabilities less than zero.<br>
I am pasting the line where I am getting the error.<br>
I have gone through the answer (<a href="https://discuss.pytorch.org/t/categorical-probs-sample-generates-runtimeerror-invalid-argument-2-invalid-multinomial-distribution-encountering-probability-entry-0/27386">https://discuss.pytorch.org/t/categorical-probs-sample-generates-runtimeerror-invalid-argument-2-invalid-multinomial-distribution-encountering-probability-entry-0/27386</a>)  which was given for the same kind of error, but I could not figure out any solution.</p>
<pre><code class="lang-auto">def getAction1(self, state):
        state = torch.FloatTensor(state)
        logits, _ = self.model1.forward(state)
        dist = F.softmax(logits, dim = -1)
        probs = Categorical(dist)
        return probs.sample()
</code></pre>
<p>The error</p>
<pre><code class="lang-auto">
  File "C:/Users/Prudhvinath.DESKTOP-09Q8801/sciebo/Thesis/JSSP/TwoAgents/20JobsTwoAgents.py", line 164, in getAction1
    return probs.sample()

  File "C:\Users\Prudhvinath.DESKTOP-09Q8801\Anaconda3\lib\site-packages\torch\distributions\categorical.py", line 107, in sample
    sample_2d = torch.multinomial(probs_2d, 1, True)

RuntimeError: invalid multinomial distribution (encountering probability entry &lt; 0)
</code></pre>
<p>The <code>Line164</code> redirects to this line in the main code <code>return probs.sample()</code></p>
<p>Can you please help me here?</p>
<p>thanks in advance.</p>

username: KFrank
cooked: <p>Hello Prudvinath!</p>
<aside class="quote no-group quote-modified" data-username="reddymap" data-post="1" data-topic="64542" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/reddymap/48/18931_2.png" class="avatar"> reddymap:</div>
<blockquote>
<pre><code class="lang-auto">def getAction2(self, state):
        state = torch.FloatTensor(state)
        logits, _ = self.model2.forward(state)
        dist = F.softmax(logits, dim = -1)
        probs = Categorical(dist)
        return probs.sample()
</code></pre>
<p>The error</p>
<pre><code class="lang-auto">  File "C:/Users/Prudhvinath.DESKTOP-09Q8801/sciebo/Thesis/JSSP/TwoAgents/20JobsTwoAgents.py", line 164, in getAction1
    return probs.sample()

RuntimeError: invalid multinomial distribution (encountering probability entry &lt; 0)
</code></pre>
</blockquote>
</aside>
<p>Your error message says that the error occurs “<code>in getAction1</code>,”<br>
but you’ve posted code for <code>def getAction2(self, state)</code>.</p>
<p>Could you double-check whether the code you’ve posted is<br>
relevant to your error?</p>
<p>(Also, when your error gives a line number, “<code>line 164</code>,” it would<br>
be helpful if you could flag the line in question when you post your<br>
code so we can be sure of what we’re looking at.)</p>
<p>Best.</p>
<p>K. Frank</p>

username: reddymap
cooked: <p>Hi Frank,</p>
<p>Thank you so much for the response.  I have just now checked my question. Since there are two <code> getAction</code> i.e <code>getAction1</code>and <code>getAction2</code>  in my code and both being similar, I have posted one.</p>
<p>Now I have updated the question.</p>
<p>Thanks and regards<br>
Prudhvi</p>