Pytorch RNG seeds fix

Nick_8229: <p>Currently I am generated normal random number by the following,</p>
<pre><code class="lang-auto">zn = torch.FloatTensor(dim1, dim2)
zn.normal_()
</code></pre>
<p>I am wondering how could i set seed for each tensor?<br>
Thanks</p>

KFrank: <p>Hi Nick!</p>
<aside class="quote no-group quote-modified" data-username="Nick_8229" data-post="1" data-topic="54215" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/nick_8229/48/14026_2.png" class="avatar"> Nick_8229:</div>
<blockquote>
<p>Currently I am generated normal random number by the following,</p>
<p>‘’’<br>
zn = torch.FloatTensor(dim1, dim2)<br>
zn.normal_()<br>
‘’’</p>
<p>I am wondering how could i set seed for each tensor?</p>
</blockquote>
</aside>
<p><code>manual_seed()</code> should work, e.g.:</p>
<pre data-code-wrap="python"><code class="lang-python">zn1 = torch.FloatTensor(dim1, dim2)
zn2 = torch.FloatTensor(dim1, dim2)
zn3 = torch.FloatTensor(dim1, dim2)

torch.manual_seed (seed1)
zn1.normal_()
torch.manual_seed (seed2)
zn2.normal_()
torch.manual_seed (seed3)
zn3.normal_()
</code></pre>
<p>Is this what you were asking?</p>
<p>Bear in mind, though, that you don’t generally want to do this.<br>
You are probably better off letting pytorch’s (pseudo) random<br>
number generator do its thing, rather than potentially making<br>
it less random (at the margins – probably won’t matter in<br>
practice) by injecting new seeds all the time.</p>
<p>My typical use of setting the random-number seed is to make<br>
some overall run repeatable from run to run, so that, for example,<br>
my random network weights start out as the same random weights<br>
for each run.  But I wouldn’t use different seeds, for example, for<br>
the weights in different layers.</p>
<p>Good luck.</p>
<p>K. Frank</p>

Nick_8229: <p>Thanks Frank.</p>
<p>What you gave is fix seed for each RNG.</p>
<p>What I want is fix seed for each tensor, kind of like followings,</p>
<pre><code class="lang-auto">zn = torch.FloatTensor(dim)
for i:
   zn[i].manual_seed(get_seed(i))
   zn[i].normal_()
</code></pre>
<p>Thanks again</p>

KFrank: <p>Hello Nick!</p>
<aside class="quote no-group" data-username="Nick_8229" data-post="3" data-topic="54215" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/nick_8229/48/14026_2.png" class="avatar"> Nick_8229:</div>
<blockquote>
<p>What you gave is fix seed for each RNG.</p>
</blockquote>
</aside>
<p>Not really.  Pytorch has (roughly speaking) a single global RNG.<br>
This global RNG is used (drawing random number from its<br>
current state) by things like <code>torch.FloatTensor.normal_()</code>.<br>
I simply reseeded the global RNG three times.</p>
<aside class="quote no-group">
<blockquote>
<p>What I want is fix seed for each tensor, kind of like followings,</p>
</blockquote>
</aside>
<p>I’m not aware of any way to attach a per-tensor RNG to a specific<br>
tensor.  (But you shouldn’t want to do this.)</p>
<aside class="quote no-group">
<blockquote>
<pre><code class="lang-auto">zn = torch.FloatTensor(dim)
for i:
   zn[i].manual_seed(get_seed(i))
   zn[i].normal_()
</code></pre>
</blockquote>
</aside>
<p>This code suggests that you want to be able to attach a separate<br>
(and separately seeded) RNG to each “row” of your tensor.   Is<br>
that what you mean?</p>
<p>But, in any event, other than the (nominally irrelevant) actual values<br>
of the randomly set elements of the tensor, how are the results of<br>
your pseudocode (or what you are trying to achieve) intended to<br>
differ from the results of the code I posted?  In both cases I see<br>
the elements of your tensors being set to (pseudo) random values drawn from a normal distribution.</p>
<p>What – in terms of results – is it that you want that differs from<br>
what my code snippet does?</p>
<p>Best.</p>
<p>K. Frank</p>