A Numerical Difference in log probability

sparseinference: <p>I see a small numerical difference when finding the log probability with a Categorical distribution, but I can’t see why.</p>
<p>First, find the logits and sample an action:</p>
<pre><code class="lang-auto">logits = policy(state).log_softmax(-1)
m = torch.distributions.Categorical(logits=logits)
action = m.sample()
</code></pre>
<p>Now get the log probability of that action in two ways:</p>
<pre><code class="lang-auto">logProb = logits.squeeze()[action]
</code></pre>
<pre><code class="lang-auto">logProb = m.log_prob(action)
</code></pre>
<p>The two methods produce slightly different values after the 6th decimal place.<br>
Shouldn’t these be exactly the same?</p>
<p>I’m not seeing the reason for the difference.</p>

KFrank: <p>Hi Peter!</p>
<aside class="quote no-group" data-username="sparseinference" data-post="1" data-topic="109345" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sparseinference/48/1999_2.png" class="avatar"> sparseinference:</div>
<blockquote>
<p>The two methods produce slightly different values after the 6th decimal place.<br>
Shouldn’t these be exactly the same?</p>
</blockquote>
</aside>
<p>No.  in general, the two results can differ by floating-point round-off error,<br>
so this is expected.</p>
<p>Try redoing your test using all <code>torch.float64</code> tensors (double precision).<br>
You should still see a discrepancy, but now in something like the 15th<br>
significant (decimal) digit.</p>
<p>Best.</p>
<p>K. Frank</p>

sparseinference: <p>Thanks, for your answer!</p>
<p>But, why am I wrong in assuming that if I pass in logits to <code>torch.distributions.Categorical</code> that no further computation needs to be done to extract the log probability?</p>
<p>I can see that if I passed in the output of a <code>softmax</code> then the distribution class needs to compute a <code>log()</code>, but in this case it’s already done.</p>

KFrank: <p>Hi Peter!</p>
<aside class="quote no-group" data-username="sparseinference" data-post="3" data-topic="109345" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sparseinference/48/1999_2.png" class="avatar"> sparseinference:</div>
<blockquote>
<p>But, why am I wrong in assuming that if I pass in logits to <code>torch.distributions.Categorical</code> that no further computation needs to be done to extract the log probability?</p>
</blockquote>
</aside>
<p>If I understand your question correctly, I think that you’re missing the<br>
fact that:</p>
<pre data-code-wrap="python"><code class="lang-python">torch.distributions.Categorical (logits = logits)
</code></pre>
<p>and:</p>
<pre><code class="lang-auto">torch.distributions.Categorical (probs = torch.softmax (logits, dim = -1))
</code></pre>
<p>return essentially the same (up to floating-point round-off)  distribution,<br>
as illustrated by this example:</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; torch.__version__
'1.7.1'
&gt;&gt;&gt; logits = torch.randn (5)
&gt;&gt;&gt; probs = torch.softmax (logits, dim = 0)
&gt;&gt;&gt; dist_logits = torch.distributions.Categorical (logits = logits)
&gt;&gt;&gt; dist_probs = torch.distributions.Categorical (probs = probs)
&gt;&gt;&gt; dist_logits
Categorical(logits: torch.Size([5]))
&gt;&gt;&gt; dist_logits.logits
tensor([-0.5839, -2.5019, -1.7824, -2.2692, -2.4226])
&gt;&gt;&gt; dist_logits.probs
tensor([0.5577, 0.0819, 0.1682, 0.1034, 0.0887])
&gt;&gt;&gt; dist_probs
Categorical(probs: torch.Size([5]))
&gt;&gt;&gt; dist_probs.logits
tensor([-0.5839, -2.5019, -1.7824, -2.2692, -2.4226])
&gt;&gt;&gt; dist_probs.probs
tensor([0.5577, 0.0819, 0.1682, 0.1034, 0.0887])
</code></pre>
<p>Best.</p>
<p>K. Frank</p>

sparseinference: <p>Ah yes, thanks so much for clarifying that.</p>
<p>Best Regards,<br>
– Peter</p>