What is manual_seed?

John_Zhang: <p>hi, i saw this from a dcgan codes .  it seems it relates to cuda<br>
here are parts of the code</p>
<p>if opt.manualSeed is None:<br>
opt.manualSeed = random.randint(1, 10000)<br>
print("Random Seed: ", opt.manualSeed)<br>
random.seed(opt.manualSeed)<br>
torch.manual_seed(opt.manualSeed)<br>
if opt.cuda:<br>
torch.cuda.manual_seed_all(opt.manualSeed)</p>

fmassa: <p><code>manual_seed</code> sets the random seed from pytorch random number generators, as explained in <a href="http://pytorch.org/docs/master/torch.html?highlight=manual_seed#torch.manual_seed">the docs</a>.</p>

happy: <p>But I still cannot understand why there should exist such code. Where are these generated random numbers used? It seems that “opt.manualSeed” has never been called later on.</p>

fmassa: <p>You just need to call <code>torch.manual_seed(seed)</code>, and it will set the seed of the random number generator to a fixed value, so that when you call for example <code>torch.rand(2)</code>, the results will be reproducible.<br>
An example</p>
<pre><code class="lang-python">import torch

torch.manual_seed(2)
print(torch.rand(2))
</code></pre>
<p>gives you</p>
<pre><code class="lang-auto"> 0.4360
 0.1851
[torch.FloatTensor of size 2]
</code></pre>
<p>Try now without the <code>torch.manual_seed</code>, and you’ll see that it changes over time.</p>

Mylinda: <p>Hey <a class="mention" href="/u/fmassa">@fmassa</a>, in the above codes, what’s the differences between random.seed(opt.manualSeed) and torch.manual_seed(opt.manualSeed)? It seems the both of the two are produce seeds. Are they repetitive? Thanks!</p>

smth: <p><a class="mention" href="/u/mylinda">@Mylinda</a> first one is for python RNG, second is for pytorch RNG</p>

Mylinda: <p>Yes, I got this point. Can the codes use torch.manual_seed(opt.manualSeed) directly and merely? What is the significance of random.seed(opt.manualSeed)? Is it redundant?  <a class="mention" href="/u/smth">@smth</a></p>

smth: <p>depends on the code. if you are using python’s <code>random</code> package and any of it’s functions, you want to also do <code>random.seed(...)</code></p>

Mylinda: <p>Got it. Thanks <a class="mention" href="/u/smth">@smth</a></p>

Brando_Miranda: <p>where is <code>opt.manualSeed</code> coming from? how is it getting its number for seeding purposes?</p>

ptrblck: <p>It’s probably defined in the DCGAN example. In <a href="https://github.com/pytorch/examples/blob/master/dcgan/main.py#L36">this</a> line the arguments from the <code>argparser</code> are assigned to <code>opt</code>.</p>
<p>However, this is just an example code. You can use whatever number you like.</p>

Brando_Miranda: <p>is it possible to use <code>os.urandom(n)</code> to seed pytorch?</p>
<hr>
<p>Do:</p>
<pre><code>ord(os.urandom(1))</code></pre>

Brando_Miranda: <p>For the record, <code>manual_seed</code> seeds all of pytorch:</p>
<p><a href="http://pytorch.org/docs/master/torch.html?highlight=manual%20seed#torch.manual_seed" class="onebox" target="_blank" rel="nofollow noopener">http://pytorch.org/docs/master/torch.html?highlight=manual%20seed#torch.manual_seed</a></p>

Yicong_Li: <p>Hi<br>
Does that means once we set manual_seed fixed, parameter in RandomResizedCrop , RandomRotation will be fixed in every iteration?</p>

ptrblck: <p>No.<br>
The random values will be still “random” but in a defined order.<br>
I.e. if you restart your script, the same random numbers will be created.<br>
Have a look at <a href="https://en.wikipedia.org/wiki/Pseudorandom_number_generator">PRNG</a> for more information.</p>

Deeply: <p>Thus, to seed everything, on the assumption one is using PyTorch and Numpy:</p>
<pre><code class="lang-auto"># use_cuda = torch.cuda.is_available() 
# ...
def random_seeding(seed_value, use_cuda):
    numpy.random.seed(seed_value) # cpu vars
    torch.manual_seed(seed_value) # cpu  vars
    if use_cuda: torch.cuda.manual_seed_all(seed_value) # gpu vars
</code></pre>
<p>Anything else is missing?</p>

achaiah: <p>You should also seed the python random:</p>
<pre><code class="lang-auto">import random
random.seed(seed)
</code></pre>

111163: <p>Thank you<br>
I got it</p>
<p>and i have try it</p>

kroditakis: <p>Seeding is related to <strong>Reproducibility</strong> of results.<br>
Consider <a href="https://pytorch.org/docs/stable/notes/randomness.html" rel="nofollow noopener">https://pytorch.org/docs/stable/notes/randomness.html</a><br>
or any other newly-updated documentation to avoid (up-to-a-point) or ensure non-determinism.</p>

JeremySMorgan: <p>Question about this - I have about 10 files all of which have <code>import torch</code> included. Do I need to call <code>torch.manual_seed(0)</code> on all of them? Or only on the first.</p>
<p>Thanks,<br>
Jeremy</p>