Multinomial changing seed

username: Sam_Lerman
cooked: <p>I have a pretty standard model. I need it to be reproducible, so I use a random seed. But when I insert a multinomial operation anywhere in the training code, e.g.,</p>
<p><code>torch.ones(10).multinomial(num_samples=2, replacement=False)</code></p>
<p>It changes the output/performance of the model.</p>

username: KFrank
cooked: <p>Hi Sam!</p>
<aside class="quote no-group quote-modified" data-username="Sam_Lerman" data-post="1" data-topic="107663" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sam_lerman/48/37165_2.png" class="avatar"> Sam_Lerman:</div>
<blockquote>
<p>I use a random seed. But when I insert a multinomial operation<br>
…<br>
It changes the output/performance of the model.</p>
</blockquote>
</aside>
<p>I think I understand what you are asking …</p>
<p>Pytorch, roughly speaking, has a global random number generator that<br>
is used for all random operations.  So when you call <code>multinomial()</code>, it<br>
“consumes” some random numbers so that future random operations<br>
become different (but should be statistically equivalent).  The “random<br>
operations” in your training code could be things like random initialization<br>
of model weights, <code>Dropout</code> operations, or a <code>RandomSampler</code> in a<br>
<code>DataLoader</code>.</p>
<p>One approach would be to pre-calculate the multinomial values you will<br>
need before setting your random seed and starting your training.</p>
<p>This is illustrated by this script:</p>
<pre data-code-wrap="python"><code class="lang-python">import torch
torch.__version__

torch.random.manual_seed (2020)
torch.randn ((5,))  # desired reproducible result

torch.random.manual_seed (2020)
torch.ones(10).multinomial(num_samples=2, replacement=False)  # this uses up some random numbers
torch.randn ((5,))   # therefore changing this result

torch.random.manual_seed (1010)
cached_multinomial = torch.ones(10).multinomial(num_samples=2, replacement=False)   # cache some values for future use

torch.random.manual_seed (2020)
cached_multinomial   # use cached values for whatever purpose
torch.randn ((5,))   # doesn't mess up desired result
</code></pre>
<p>Here is its output:</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; import torch
&gt;&gt;&gt; torch.__version__
'1.6.0'
&gt;&gt;&gt;
&gt;&gt;&gt; torch.random.manual_seed (2020)
&lt;torch._C.Generator object at 0x7fe662956910&gt;
&gt;&gt;&gt; torch.randn ((5,))  # desired reproducible result
tensor([ 1.2372, -0.9604,  1.5415, -0.4079,  0.8806])
&gt;&gt;&gt;
&gt;&gt;&gt; torch.random.manual_seed (2020)
&lt;torch._C.Generator object at 0x7fe662956910&gt;
&gt;&gt;&gt; torch.ones(10).multinomial(num_samples=2, replacement=False)  # this uses up some random numbers
tensor([7, 9])
&gt;&gt;&gt; torch.randn ((5,))   # therefore changing this result
tensor([-0.3136,  0.6418,  1.1961,  0.9936,  1.0911])
&gt;&gt;&gt;
&gt;&gt;&gt; torch.random.manual_seed (1010)
&lt;torch._C.Generator object at 0x7fe662956910&gt;
&gt;&gt;&gt; cached_multinomial = torch.ones(10).multinomial(num_samples=2, replacement=False)   # cache some values for future use
&gt;&gt;&gt;
&gt;&gt;&gt; torch.random.manual_seed (2020)
&lt;torch._C.Generator object at 0x7fe662956910&gt;
&gt;&gt;&gt; cached_multinomial   # use cached values for whatever purpose
tensor([2, 0])
&gt;&gt;&gt; torch.randn ((5,))   # doesn't mess up desired result
tensor([ 1.2372, -0.9604,  1.5415, -0.4079,  0.8806])
</code></pre>
<p>You could also instantiate a non-global <a href="https://pytorch.org/docs/1.7.0/generated/torch.Generator.html" rel="noopener nofollow ugc">Generator</a> object that you then<br>
pass into your <code>multinomial()</code> calls so that global random numbers<br>
won’t be consumed.</p>
<p>Best.</p>
<p>K. Frank</p>

username: Sam_Lerman
cooked: <p>Thank you! Another option is to temporarily use numpy.choice in that call, but this would be better.</p>

username: Sam_Lerman
cooked: <p>Do you know if numpy also does what pytorch does in that it also “consumes” some random numbers thus changing outputs with each extra call?</p>

username: KFrank
cooked: <p>Hi Sam!</p>
<aside class="quote no-group quote-modified" data-username="Sam_Lerman" data-post="4" data-topic="107663" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sam_lerman/48/37165_2.png" class="avatar"> Sam_Lerman:</div>
<blockquote>
<p>Do you know if numpy … also “consumes” some random numbers thus changing outputs with each extra call?</p>
</blockquote>
</aside>
<p>I’m not a numpy expert, but, yes, almost certainly.  (You could easily run<br>
a quick test.)</p>
<p>If various numpy distributions – normal, multinomial, etc. – don’t require<br>
you to pass in a random-number-generator object, then they are drawing<br>
random numbers from some global generator, and doing so will have the<br>
effect you describe.</p>
<p>Best.</p>
<p>K. Frank</p>

username: Sam_Lerman
cooked: <p>Got it. Is there a way I could extract the seed prior to the extra random operation so that I can set it back to that seed?</p>
<p>I noticed you set the seed to <code>1010</code> in your example. Was that line necessary? It seems like just resetting the seed to <code>2020</code> would be sufficient.</p>

username: KFrank
cooked: <p>Hi Sam!</p>
<aside class="quote no-group" data-username="Sam_Lerman" data-post="6" data-topic="107663" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sam_lerman/48/37165_2.png" class="avatar"> Sam_Lerman:</div>
<blockquote>
<p>Is there a way I could extract the seed prior to the extra random operation so that I can set it back to that seed?</p>
</blockquote>
</aside>
<p>You don’t really want to do this, but if you decide that you do want to<br>
do this, there are a couple of approaches.  See the documentation for<br>
<a href="https://pytorch.org/docs/1.7.0/random.html" rel="noopener nofollow ugc">torch.random</a> for details.</p>
<p>As I understand it, you make calls to <code>multinomial()</code> during your training<br>
process.  If your calls to <code>multinomial()</code> are deterministic – called the<br>
same number of times at the same point in your training process – then<br>
your training process should be reproducible, notwithstanding the fact<br>
that <code>multinomial()</code> consumes some random numbers.  (If your calls<br>
to <code>multinomial()</code> are not deterministic, why not?)</p>
<p>The problem with having <code>multinomial()</code> consume some random<br>
numbers from the global generator and then resetting its state is that<br>
your <code>multinomial()</code> values will now be correlated with the next set<br>
of random values used by the next part of your training process.  In<br>
practice, this is very unlikely to have any problematic effect, but reusing<br>
(pseudo) random numbers like this makes them no longer properly<br>
(pseudo) random.</p>
<p>If you require that your training process be reproducible but you can’t,<br>
for whatever reason, make your <code>multinomial()</code> calls deterministic,<br>
then the standard approach would be to instantiate a separate<br>
<code>Generator</code> object (seeded with an uncorrelated seed) for your<br>
<code>multinomial()</code> calls (or you could pre-calculate your necessary<br>
<code>multinomial()</code> values before seeding the global generator).</p>
<aside class="quote no-group">
<blockquote>
<p>I noticed you set the seed to <code>1010</code> in your example. Was that line necessary? It seems like just resetting the seed to <code>2020</code> would be sufficient.</p>
</blockquote>
</aside>
<p>Calling <code>manual_seed()</code> prior to calling <code>multinomial()</code> is not required<br>
to make the call to <code>randn()</code> reproducible.  It is there to make the call<br>
to <code>multinomial()</code> reproducible – certainly not necessary, but I put it<br>
in for consistency.</p>
<p>Best.</p>
<p>K. Frank</p>

username: Sam_Lerman
cooked: <p>It’s just that I need to compare it to a model that is the same in every way but does not call multinomial, so the randomness changes. For modularity reasons it’s hard to insert a manual random seed in the same place in both models, i.e., in the place corresponding to right after the multinomial call.</p>
<aside class="quote no-group" data-username="KFrank" data-post="7" data-topic="107663">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/ecb155/48.png" class="avatar"> KFrank:</div>
<blockquote>
<p>to instantiate a separate<br>
<code>Generator</code> object (seeded with an uncorrelated seed) for your<br>
<code>multinomial()</code> calls (or you could pre-calculate your necessary<br>
<code>multinomial()</code> values before seeding the global generator).</p>
</blockquote>
</aside>
<p>How would I go about doing this?</p>

username: Sam_Lerman
cooked: <p>How can I pass a generator into multinomial? Can you provide an example? Multinomial doesn’t take a generator as input.</p>

username: Sam_Lerman
cooked: <p>Is this okay?</p>
<pre><code class="lang-auto">rand_state = torch.random.get_rng_state()
samp = probas.multinomial(num_samples, replacement)  
torch.random.set_rng_state(rand_state)
</code></pre>

username: Sam_Lerman
cooked: <pre><code class="lang-auto">rand_state = torch.random.get_rng_state()
torch.random.manual_seed(torch.randn(1).data)
probas.multinomial(num_samples)  # (temporarily) changes seed
torch.random.set_rng_state(rand_state)
</code></pre>