distributions.Categorical fails with constraint.Simplex but manual check passes

username: Haydnspass
cooked: <p>Hey,</p>
<p>I am trying to instantiate a Categorical, but for some reason, in the early stages of the training it sometimes fails with</p>
<pre><code class="lang-auto">ValueError: Expected parameter probs (Tensor of shape (64, 1600)) 
of distribution Categorical(probs: torch.Size([64, 1600])) to satisfy the 
constraint Simplex(), but found invalid values:
</code></pre>
<p>however, if I then check the values with</p>
<pre><code class="lang-auto">torch.distributions.constraints.simplex.check(p)
</code></pre>
<p>I get all True.<br>
Is it in fact another validation and the error message is misleading? An idea how to debug this?</p>
<p>Cheers,<br>
Lucas</p>

username: KFrank
cooked: <p>Hi Lucas!</p>
<aside class="quote no-group" data-username="Haydnspass" data-post="1" data-topic="163209" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/haydnspass/48/3780_2.png" class="avatar"> Haydnspass:</div>
<blockquote>
<p>however, if I then check the values with</p>
<pre><code class="lang-auto">torch.distributions.constraints.simplex.check(p)
</code></pre>
<p>I get all True.</p>
</blockquote>
</aside>
<p>I cannot reproduce your issue.  (I agree that this is unexpected behavior.)</p>
<p>Are you able to put together a short, runnable script that reproduces this?<br>
Perhaps you could hold on to a copy of your <code>probs</code> tensor, wrap your<br>
attempted instantiation of <code>Categorical</code> in a <code>try</code> block, and print out the<br>
fishy <code>probs</code> tensor (with adequate precision to reproduce the issue) if the<br>
<code>ValueError</code> is raised.</p>
<p>What is the <code>dtype</code> of your <code>probs</code> tensor (float, double, etc.)?</p>
<p>Also, could you tell us what version of pytorch you are using, and if you<br>
are seeing this on a gpu, what model it is?</p>
<p>(As an aside, it is perfectly possible for <code>simplex.check()</code> to fail but for<br>
the instantiation of <code>Categorical</code> to succeed – the opposite of your issue.<br>
This is because <code>Categorical</code> normalizes the sum of each row of <code>probs</code><br>
to one before applying <code>simplex.check()</code>.)</p>
<p>Best.</p>
<p>K. Frank</p>

username: Haydnspass
cooked: <p>Hey <a class="mention" href="/u/kfrank">@KFrank</a>,</p>
<p>thanks for the quick response. I dumped the tensor in a try except block. I uploaded it here:<br>
<a href="https://oc.embl.de/index.php/s/IxhtqcoNbUnpLVS" class="onebox" target="_blank" rel="noopener nofollow ugc">https://oc.embl.de/index.php/s/IxhtqcoNbUnpLVS</a></p>
<p>and the behaviour on my machine as<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/a/1/a1cb5289e1135e043438499f03fdc1d3c9b9374b.png" data-download-href="https://discuss.pytorch.org/uploads/default/a1cb5289e1135e043438499f03fdc1d3c9b9374b" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/a/1/a1cb5289e1135e043438499f03fdc1d3c9b9374b_2_690x354.png" alt="image" data-base62-sha1="n5iujKQVqAN1aV5DukUARdSnxaj" width="690" height="354" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/a/1/a1cb5289e1135e043438499f03fdc1d3c9b9374b_2_690x354.png, https://discuss.pytorch.org/uploads/default/optimized/3X/a/1/a1cb5289e1135e043438499f03fdc1d3c9b9374b_2_1035x531.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/a/1/a1cb5289e1135e043438499f03fdc1d3c9b9374b_2_1380x708.png 2x" data-dominant-color="F3F3F3"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1964×1010 135 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>The relevant parts of my environment are:</p>
<h1><a name="name-version-build-channel-1" class="anchor" href="#name-version-build-channel-1"></a>Name                    Version                   Build  Channel</h1>
<p>cudatoolkit               11.3.1              h9edb442_10    conda-forge<br>
python                    3.10.5          h582c2e5_0_cpython    conda-forge<br>
python_abi                3.10                    2_cp310    conda-forge<br>
pytorch                   1.12.0          py3.10_cuda11.3_cudnn8.3.2_0    pytorch<br>
pytorch-lightning         1.6.5                    pypi_0    pypi<br>
pytorch-mutex             1.0                        cuda    pytorch</p>
<p>complete env here: <a href="https://oc.embl.de/index.php/s/UwFFBFDJy6s71t0" rel="noopener nofollow ugc">https://oc.embl.de/index.php/s/UwFFBFDJy6s71t0</a></p>

username: KFrank
cooked: <p>Hi Lucas!</p>
<aside class="quote no-group" data-username="Haydnspass" data-post="3" data-topic="163209" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/haydnspass/48/3780_2.png" class="avatar"> Haydnspass:</div>
<blockquote>
<p>I dumped the tensor in a try except block.</p>
</blockquote>
</aside>
<p>I can reproduce your issue using your <code>fishy_tensor</code>.</p>
<p>I believe that it is caused by a somewhat unlikely conspiracy of round-off<br>
error, likely exacerbated by the fact that the rows of your <code>probs</code> tensor<br>
are rather long (length 1600).</p>
<p>In your case your <code>probs</code> (<code>fishy_tensor</code>) are proper probabilities, with rows<br>
normalized to sum to one (and it passes <code>simplex.check()</code>).  <code>Categorical</code><br>
does not require its row-sums to  be normalized to one, so it normalizes<br>
them (in your case, redundantly, but Categorical doesn’t know this) before<br>
it applies. <code>simplex.check()</code>.  The normalization that <code>Categorical</code> applies<br>
changes the round-off error a little bit and just happens to push one of the<br>
row-sums a little bit over <code>simplex.check()</code>'s tolerance of <code>1.e-6</code>, causing<br>
<code>Categorical</code> to raise the error due to its <em>internal</em> call to <code>simplex.check()</code>.</p>
<p>This script that uses your <code>fishy_tensor</code> illustrates these points:</p>
<pre data-code-wrap="python"><code class="lang-python">import torch
print (torch.__version__)
print (torch.version.cuda)
print (torch.cuda.get_device_name())

fishy = torch.load ('fishy_tensor.pt')

print ((fishy.sum (dim = 1) - 1.0).abs().max())                       # rows sum to one (up to round-off)
print (torch.distributions.constraints.simplex.check (fishy).all())   # passes test

fishy_norm = fishy / fishy.sum (dim = 1).unsqueeze (-1)               # "normalize" rows to sum to one
print ((fishy_norm.sum (dim = 1) - 1.0).abs().max())                  # rows sum not quite as well to one
print (torch.distributions.constraints.simplex.check (fishy_norm))    # fails
print ((fishy_norm.sum (dim = 1) - 1.0).abs() &gt; 1.e-6)                # simplex uses 1.e-6 as its tolerance

print (fishy_norm.sum (dim = 1)[45] - 1.0)                            # row 45 is the culprit
print (fishy_norm[45].sum() - 1.0)                                    # but details of cuda sum matter
</code></pre>
<p>And here is its output:</p>
<pre><code class="lang-plaintext">1.12.0
11.6
GeForce GTX 1050 Ti
tensor(7.1526e-07, device='cuda:0', grad_fn=&lt;MaxBackward1&gt;)
tensor(True, device='cuda:0')
tensor(1.0729e-06, device='cuda:0', grad_fn=&lt;MaxBackward1&gt;)
tensor([ True,  True,  True,  True,  True,  True,  True,  True,  True,  True,
         True,  True,  True,  True,  True,  True,  True,  True,  True,  True,
         True,  True,  True,  True,  True,  True,  True,  True,  True,  True,
         True,  True,  True,  True,  True,  True,  True,  True,  True,  True,
         True,  True,  True,  True,  True, False,  True,  True,  True,  True,
         True,  True,  True,  True,  True,  True,  True,  True,  True,  True,
         True,  True,  True,  True], device='cuda:0')
tensor([False, False, False, False, False, False, False, False, False, False,
        False, False, False, False, False, False, False, False, False, False,
        False, False, False, False, False, False, False, False, False, False,
        False, False, False, False, False, False, False, False, False, False,
        False, False, False, False, False,  True, False, False, False, False,
        False, False, False, False, False, False, False, False, False, False,
        False, False, False, False], device='cuda:0')
tensor(-1.0729e-06, device='cuda:0', grad_fn=&lt;SubBackward0&gt;)
tensor(-5.9605e-07, device='cuda:0', grad_fn=&lt;SubBackward0&gt;)
</code></pre>
<p>For what it’s worth, running your particular <code>fishy_tensor</code> on my cpu does not<br>
show this behavior.</p>
<p>I assume that your code calls <code>simplex.check()</code> to protect against <code>Categorical</code><br>
raising an error.  If you’re comfortable that your <code>probs</code> vector is legit, you could<br>
instantiate <code>Categorical</code> with <code>validate_args = False</code>.  (If you want to stick<br>
with the validation and call <code>simplex.check()</code> for protection, I imagine that<br>
(redundantly) normalizing your <code>probs</code> tensor (to mimic what <code>Categorical</code> does<br>
internally) before making your call to <code>simplex.check()</code> might be an adequate<br>
work-around.)</p>
<p>I also note – not that I think that you should read anything into this – summing<br>
row 45 of <code>fishy_tensor</code> (after normalizing it) in isolation does not cross<br>
<code>simplex.check()</code>'s <code>1.e-6</code> tolerance, while summing it as a row sum of the<br>
whole tensor does.  I assume that this causes a slight difference in innocent<br>
round-of error due to the gpu using differing orders of operations in the two<br>
cases.</p>
<p>I don’t really see this as a bug, but rather an “expected” consequence<br>
of round-of error.  One could argue, however, (and I suppose I do) that<br>
<code>Categorical</code> / <code>simplex.check()</code> should take into account the length<br>
of the probability vectors and widen out the tolerance as they become<br>
longer.</p>
<p>Best.</p>
<p>K. Frank</p>

username: Haydnspass
cooked: <p>Thanks for looking into that! Yes indeed the workaround is easy, I was already considering setting the validation to false, but I wanted to know what was going on.<br>
I can understand how this comes about; not sure whether this should be considered a bug.<br>
I guess if the class which itself applies the normalization induces the rounding error, it should be considered a bug, no?<br>
Maybe I should raise a GH issue, don’t know.</p>

username: KFrank
cooked: <p>Hi Lucas!</p>
<aside class="quote no-group" data-username="Haydnspass" data-post="5" data-topic="163209" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/haydnspass/48/3780_2.png" class="avatar"> Haydnspass:</div>
<blockquote>
<p>I can understand how this comes about;</p>
</blockquote>
</aside>
<p>A bit of an update:  So far, I have only been able to reproduce your issue<br>
using your <code>fishy</code> tensor on my gpu.  I’ve tried various schemes to reproduce<br>
it using random data (and lots of trials) without success.</p>
<aside class="quote no-group">
<blockquote>
<p>not sure whether this should be considered a bug.<br>
…<br>
Maybe I should raise a GH issue, don’t know.</p>
</blockquote>
</aside>
<p>I would say that this is legitimately a bug (but maybe a minor edge case).<br>
I think your logging a github issue would be appropriate.  If you do, be sure<br>
to attach your <code>fishy</code> tensor so that it can be reproduced.</p>
<p>Based on my (failed) experiments, I am less convinced by my original<br>
theory of unlikely, but straightforward round-off error (although that<br>
theory could still be true).</p>
<p>If it is round-off error, I would say that there is a (minor) bug in<br>
<code>simplex.check()</code>.  Perhaps <code>simplex.check()</code> should widen out<br>
its tolerance based on the length of the rows of the <code>probs</code> tensor.</p>
<p>But if straightforward round-off error doesn’t explain the issue, this would<br>
hint at a possible cuda bug – not that I have any idea what it could be.</p>
<p>Is there any chance that you could capture another distinct <code>fishy</code> tensor?<br>
If it’s not just round-off error, having more examples that reproduce it would<br>
likely be a big help in tracking down what is going on.</p>
<p><a class="mention" href="/u/ptrblck">@ptrblck</a>:  This seems to me to be a weird one – would you want to take<br>
a look?</p>
<p>Best.</p>
<p>K. Frank</p>

username: ptrblck
cooked: <p>Thanks for pinging me on this issue.<br>
Could you also post which PyTorch release you’ve used to reproduce the issue, please?</p>
<p><a class="mention" href="/u/haydnspass">@Haydnspass</a> did you already create a GitHub issue? Also, were you seeing the same issue using the latest nightly binary?</p>

username: Haydnspass
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> yep I opened the GH issue <a href="https://github.com/pytorch/pytorch/issues/87468" class="inline-onebox" rel="noopener nofollow ugc">Categorical fails simplex validation after its own normalisation on CUDA · Issue #87468 · pytorch/pytorch · GitHub</a> after <a class="mention" href="/u/kfrank">@KFrank</a> could reproduce the issue. Maybe the discussion should go move there?<br>
I posted the environment above, it should be pytorch 1.12.0 py3.10_cuda11.3_cudnn8.3.2_0 pytorch<br>
I will try the latest binary <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>

username: KFrank
cooked: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a> (and Lucas)!</p>
<aside class="quote no-group" data-username="ptrblck" data-post="7" data-topic="163209" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>Could you also post which PyTorch release you’ve used to reproduce the issue, please?</p>
</blockquote>
</aside>
<p>For completeness:</p>
<p>I reproduce the issue (solely with the <code>fishy_tensor</code>) on pytorch 1.12.0 /<br>
cuda 11.6 / gpu GeForce GTX 1050 Ti.</p>
<p>I also reproduced the issue on a version-1.14 nightly build from when I<br>
responded earlier, namely pytorch 1.14.0.dev20221014 / cuda 11.7 /<br>
gpu GeForce GTX 1050 Ti.</p>
<p>Best.</p>
<p>K. Frank</p>

username: Lingfei_Wang
cooked: <p>I got exactly the same issue with torch 1.13.0, torchaudio 0.12.1, torchvision 0.13.1, cuda 12.1. I tried different GPUs including A100, 4090, 3080ti, 3070. I can reproduce the issue every time.</p>

username: botinock
cooked: <p>Hello! I solved it in the next way.</p>
<pre><code class="lang-auto">from torch.distributions import Distribution
Distribution.set_default_validate_args(False)
</code></pre>
<p>if in your code one python instance you could use flag <code>python -O</code><br>
But if there are many python instances like in RLLib distributions, you need to turn off validation in custom class definition (or before usage torch distributions)</p>