How to initialize tensors from different distributions

AladdinPerzon: <p>There seems to be multiple ways of initializing a tensor from a distribution, for example if we want to sample from a uniform distribution I found this can be done by:</p>
<h4>Method 1</h4>
<pre><code class="lang-auto">from torch.distributions.uniform import Uniform
U = Uniform(low=torch.tensor([0.0]), high=torch.tensor([1.0]))
samples = U.sample(sample_shape=(5,5)) # This shape turns out 5x5x1 (?)
</code></pre>
<h4>Method 2</h4>
<pre><code class="lang-auto">w = torch.empty(3,  5)
nn.init.uniform_(w, a=0, b=1)
</code></pre>
<h4>Method 3</h4>
<pre><code class="lang-auto"># The arguments seem to be uniform_(from, to) but writing from gives
# me an error because it is a keyword
w = torch.empty(3, 5).uniform_(0, to=2)
</code></pre>
<p>Perhaps there are more ways to do this that I’ve missed. I feel using the nn.init seems to make sense, but from <a href="https://pytorch.org/docs/stable/nn.init.html" rel="nofollow noopener">https://pytorch.org/docs/stable/nn.init.html</a> it seems to be quite lacking in comparison to <a href="https://pytorch.org/docs/stable/distributions.html" rel="nofollow noopener">https://pytorch.org/docs/stable/distributions.html</a>.</p>
<h3>Question:</h3>
<p>Which method is recommended to use and is there a reason why we have multiple ways of doing it? All of these methods are also a little bit different than the way numpy does it which seems to be most intuitive to me: <code>s = np.random.uniform(low=0, high=1, size=(1, 5))</code></p>

ptrblck: <p>My personal point of view is:</p>
<ul>
<li>
<code>torch.distributions</code> have a lot of sampling methods and other utility functions (which can be used in e.g. a reinforcement learning setup), so that I wouldn’t use them to just initialize a tensor.</li>
<li>I’m using <code>nn.init</code> only to initialize <code>nn.Modules</code> via <code>model.apply(init_fn)</code>.</li>
<li>Method 3 is the way I would prefer to initialize a tensor.</li>
</ul>
<p>Of course your preference might be different and I can also imagine that Method 2 could be preferred. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

AladdinPerzon: <p>Thank you, I really appreciate you taking the time. Your personal viewpoint is good enough for me, method 3 is now my go to <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"> Only issue I have is if I for example want to sample from a poisson distribution is there a way to do that? I guess I’m just having trouble finding the documentation for the distributions we can sample from</p>

ptrblck: <p>To create a tensor from a poisson distribution you could directly call <a href="https://pytorch.org/docs/stable/torch.html#torch.poisson"><code>torch.poisson</code></a> on the rates of your Poisson distribution via:</p>
<pre><code class="lang-python">rates = torch.rand(4, 4) * 5  # rate parameter between 0 and 5
torch.poisson(rates)
&gt; tensor([[9., 1., 3., 5.],
          [8., 6., 6., 0.],
          [0., 4., 5., 3.],
          [2., 1., 4., 2.]])
</code></pre>
<p>Alternatively, <code>distributions.poisson.Poisson(rates).sample()</code> would also work, and it really depends on your personal preference here. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>