Kl_divergence() returns negative values on Categorial distributions?

tor: <p>For Categorical distributions below, which are quite close, <code>distributions.kl.kl_divergence()</code> returns negative values.<br>
Interestingly, different ways of initialisation, ie via probs or logits, give different results.</p>
<pre><code class="lang-auto">#!/usr/bin/env python3
import torch
from torch.distributions.utils import probs_to_logits

print('torch.__version__', torch.__version__)

# p distrib
prob_p = torch.tensor([0.2739, 0.2857, 0.2089, 0.2316])
p_initprob = torch.distributions.Categorical(probs=prob_p)
p_initlogit = torch.distributions.Categorical(logits=probs_to_logits(prob_p))
assert torch.allclose(p_initprob.probs, p_initlogit.probs)
assert torch.allclose(p_initprob.logits, p_initlogit.logits)

# q distrib
prop_q = torch.tensor([0.2739, 0.2857, 0.2088, 0.2315])
q_initprob = torch.distributions.Categorical(probs=prop_q)
q_initlogit = torch.distributions.Categorical(logits=probs_to_logits(prop_q))
assert torch.allclose(q_initprob.probs, q_initlogit.probs)
assert torch.allclose(q_initprob.logits, q_initlogit.logits)

# KLDiv: p || q
print('KLDiv: p || q')
print('kldiv_p_q_initprob', torch.distributions.kl.kl_divergence(p_initprob, q_initprob))
print('kldiv_p_q_initlogit', torch.distributions.kl.kl_divergence(p_initlogit, q_initlogit))

# KLDiv: q || p
print('KLDiv: q || p')
print('kldiv_q_p_initprob', torch.distributions.kl.kl_divergence(q_initprob, p_initprob))
print('kldiv_q_p_initlogit', torch.distributions.kl.kl_divergence(q_initlogit, p_initlogit))
</code></pre>
<p>Output:</p>
<pre><code class="lang-auto">torch.__version__ 1.1.0
KLDiv: p || q
kldiv_p_q_initprob tensor(-1.6520e-08)
kldiv_p_q_initlogit tensor(1.0270e-07)
KLDiv: q || p
kldiv_q_p_initprob tensor(6.7579e-08)
kldiv_q_p_initlogit tensor(-5.1627e-08)
</code></pre>
<p>Is this a known issue due numerical instability?<br>
Any clue on how to address this?<br>
Thank you.</p>
<p>Related issues:<br>
<aside class="onebox githubissue">
  <header class="source">
      <a href="https://github.com/pytorch/pytorch/issues/18730" target="_blank" rel="nofollow noopener">github.com/pytorch/pytorch</a>
  </header>
  <article class="onebox-body">
    <a href="https://github.com/Diego999" rel="nofollow noopener">
<img src="https://avatars1.githubusercontent.com/u/1092464?v=2&amp;s=96" class="thumbnail onebox-avatar" width="96" height="96">
</a>

<h4><a href="https://github.com/pytorch/pytorch/issues/18730" target="_blank" rel="nofollow noopener">Issue: Numerical instability KL divergence RelaxedOneHotCategorical</a></h4>

<div class="date" style="margin-top:10px;">
	<div class="user" style="margin-top:10px;">
	opened by <a href="https://github.com/Diego999" target="_blank" rel="nofollow noopener">Diego999</a>
	on <a href="https://github.com/pytorch/pytorch/issues/18730" target="_blank" rel="nofollow noopener">2019-04-02</a>
	</div>
	<div class="user">
	</div>
</div>

<pre class="content" style="white-space: pre-wrap;">🐛 Bug
It seems that computing the KL divergence RelaxedOneHotCategorical leads to numerical instabilities.
To Reproduce
import torch
from torch.distributions import RelaxedOneHotCategorical

p_m = RelaxedOneHotCategorical(torch.tensor([2.2]), probs=torch.tensor([0.1,...</pre>

<div class="labels">
 	<span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">module: distributions</span>
 	<span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">topic: numerical-stability</span>
 	<span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">triaged</span>
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>