Duplicate results due to lack of entropy using multinomial without replacement?

jeffc: <p>I’m trying to confirm that I’ve found a bug in multinomial without replacement. It seems to only use a 32 bit random number as a ‘seed’ for its own random number generator for choosing all the samples. This means at most there are only 2^32 permutations it will generate without replacement, even when there are far more possible. I wrote a short app showing that multinomial without replacement always generates at least one DUPLICATE PERMUTATION every few 10s of thousands of iterations. Am I missing something?</p>
<p>Here’s the app:</p>
<pre><code class="lang-auto">import torch
letters = [*"abcdefghijklmnopqrstuvwxyz"]
p = torch.tensor([1.0]*len(letters))
found = {}
for count in range(1, 1000000):
    t = torch.multinomial(p, num_samples=len(letters), replacement=False).tolist()
    result = ''.join(letters[n] for n in t)
    if result in found:
        print (f"We have generated {count:,} strings and the most recent one was '{result}'")
        print (f"This is the same as the {found[result]:,} generated string which was also '{result}'")
        break
    found[result] = count
</code></pre>
<p>The output looks like this:</p>
<pre><code class="lang-auto">We have generated 31,869 strings and the most recent one was 'kmfuqyesxvngithplacwbjdroz'
This is the same as the 1,375 generated string which was also 'kmfuqyesxvngithplacwbjdroz'
</code></pre>

KFrank: <p>Hi Jeff!</p>
<aside class="quote no-group" data-username="jeffc" data-post="1" data-topic="207120" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/j/ed8c4c/48.png" class="avatar"> jeffc:</div>
<blockquote>
<p>I’m trying to confirm that I’ve found a bug in multinomial without replacement.</p>
</blockquote>
</aside>
<p>I can confirm your result, and I do believe that it is a bug.</p>
<aside class="quote no-group">
<blockquote>
<p>It seems to only use a 32 bit random number as a ‘seed’ for its own random number generator for choosing all the samples.</p>
</blockquote>
</aside>
<p>This isn’t the cause (but I don’t know what the cause is).</p>
<p><code>torch.multinomial()</code> uses by default pytorch’s “global” random-number<br>
generator which, on the cpu, is (from memory) a mersenne twister and<br>
has a period that is vastly greater than <code>2**32</code>.</p>
<aside class="quote no-group">
<blockquote>
<p>This means at most there are only 2^32 permutations it will generate without replacement, even when there are far more possible.</p>
</blockquote>
</aside>
<p>I don’t this that this is true.  I speculate (but have not confirmed) that<br>
<code>multinomial()</code> will generate all permutations with non-zero probability,<br>
but does so sufficiently non-uniformly that duplicates are generated<br>
much too frequently.</p>
<aside class="quote no-group">
<blockquote>
<p>I wrote a short app showing that multinomial without replacement always generates at least one DUPLICATE PERMUTATION every few 10s of thousands of iterations. Am I missing something?</p>
</blockquote>
</aside>
<p>I don’t think that you are missing anything.  I see the same thing<br>
with a similar script.</p>
<p>I will start a new thread with the details to highlight this bug you’ve<br>
found.</p>
<p>[Edit:  <a href="https://discuss.pytorch.org/t/bug-in-torch-multinomial-generated-distribution-is-modestly-incorrect/207149">This</a> is the new thread.]</p>
<p>Best.</p>
<p>K. Frank</p>

jeffc: <p>Thanks!</p>
<p>Here’s why I’m suspicious that multinomial without replacement is only using 32 bits of entropy. I looked at the output of the random number generator using a predictable seed, and it appeared that multinomial was only consuming 32 bits from it without replacement, regardless of how many elements you ask it to sample.</p>
<p>Here’s an example that shows the output of rand with a predictable seed, then it tries again and shows how many values were consumed after a multinomial call which pulls 10 samples between 1 and 100 without replacement.</p>
<pre><code class="lang-auto">import torch

g = torch.Generator().manual_seed(2147483647)
print(torch.rand(16, generator=g))
print()

g = torch.Generator().manual_seed(2147483647)
print(torch.multinomial(torch.tensor([1.]*100), num_samples=10,replacement=False, generator=g))
print()
print(torch.rand(16, generator=g))
</code></pre>
<p>The output shows that only the first random value was consumed.</p>
<pre><code class="lang-auto">tensor([0.7081, 0.3542, 0.1054, 0.5996, 0.0904, 0.0899, 0.8822, 0.9887, 0.0080,
        0.2908, 0.7408, 0.4012, 0.8640, 0.7391, 0.3845, 0.2176])

tensor([32, 69, 47, 83, 10, 41, 37, 42, 72, 28])

tensor([0.3542, 0.1054, 0.5996, 0.0904, 0.0899, 0.8822, 0.9887, 0.0080, 0.2908,
        0.7408, 0.4012, 0.8640, 0.7391, 0.3845, 0.2176, 0.7054])
</code></pre>
<p>I tried this with a double precision probability tensor, and it did consume two entries (64 bits of randomness), but in my birthday problem test even with a double precision probability tensor it is still hitting collisions like it is generating the samples with only 32 bits of entropy. My suspicion is that it’s using 32 bits from the random number generator to seed its own generator that pulls the samples.</p>

KFrank: <p>Hi Jeff!</p>
<aside class="quote no-group" data-username="jeffc" data-post="3" data-topic="207120" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/j/ed8c4c/48.png" class="avatar"> jeffc:</div>
<blockquote>
<p>The output shows that only the first random value was consumed.</p>
</blockquote>
</aside>
<p>Yes, I agree.  I’ve replied in more detail in the “bug-report” thread.</p>
<p>By the way, this looks like a regression that didn’t occur in version<br>
1.11.0.</p>
<p>Best.</p>
<p>K. Frank</p>