Torch.rand_like only generating positive numbers

alromz: <p>I have tested the function on different devices and it always comes to the same conclusion: torch.rand_like is generating only positive values, and I don’t understand why, because this function should be generating numbers from a normal distribution. I get even more confused when consulting the Pytorch docs, quoting:</p>
<blockquote>
<p>torch.randn_like(input) is equivalent to torch.randn(input.size(), dtype=input.dtype, layout=input.layout, device=input.device)</p>
</blockquote>
<p>The funny thing is that torch.randn does generate negative numbers, which is correct with the fact that it is sampling from a normal distribution too.</p>
<p>Has someone else noticed this before? Or, does anybody know if this is an error? Perhaps I am the one who is confused, and if this is the case, I would appreciate if someone explained to me why.</p>
<p>Additionally, I’ve seen a lot of implementations of Diffusion(ddpm) and VAE with this function; doesn’t this affect the correct functioning of the model? Because both networks should be sampling from a normal distribution, not only the positive part.</p>

Neilstid: <p>Could you provide more details like the input you used for the randn_like ?</p>
<p>I have not been able to reproduce the bug you have described:</p>
<pre><code class="lang-auto">import torch
a = torch.randn(10)
torch.randn_like(a)
</code></pre>
<p>The snippet above gave me:<br>
tensor([ 0.8246,  0.6652,  0.4302,  0.4329,  1.3249,  0.7312, -0.2145,  0.9759, -1.8222,  0.3011])<br>
which holds negative numbers (-0.2145, and -1.8222)</p>
<p>My pytorch version: ‘1.13.1+cu117’ and Python: 3.10.12</p>
<p>May be your input is an uint and therefore give you only uint output (so only positive ?)</p>

alromz: <p>That was the first thing that I tested. I am doing a similar test as the one you posted:</p>
<pre><code class="lang-auto">import torch
t1 = torch.randn( ( int(1e6), ) )
t2 = torch.rand_like(t1)
print( any(t1&lt;0), any(t2&lt;0) )
</code></pre>
<p>And this gave me the output: (True, False). rand_like is receiving a torch.float32 as t1 has been produced by randn. When making the histogram of the numbers generated is even stranger:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/d/7/d773c259456c9345621710c6e46b275074bb3316.png" data-download-href="https://discuss.pytorch.org/uploads/default/d773c259456c9345621710c6e46b275074bb3316" title="hist"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/d/7/d773c259456c9345621710c6e46b275074bb3316_2_690x338.png" alt="hist" data-base62-sha1="uJYFtLDTccLzcy0EW59wKTvork2" width="690" height="338" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/d/7/d773c259456c9345621710c6e46b275074bb3316_2_690x338.png, https://discuss.pytorch.org/uploads/default/optimized/3X/d/7/d773c259456c9345621710c6e46b275074bb3316_2_1035x507.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/d/7/d773c259456c9345621710c6e46b275074bb3316_2_1380x676.png 2x" data-dominant-color="B1CFE4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">hist</span><span class="informations">1536×754 10.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
Clearly, the rand_like function is sampling from a uniform (0,1), which doesn’t make sense based on its definition in docs.</p>
<p>I ran these tests on 2 different versions:</p>
<ul>
<li>Python 3.10.11 Pytorch: 2.1.0+cpu</li>
<li>Python: 3.10.9 Pytorch: 1.13.1+cu117</li>
</ul>

alromz: <aside class="quote no-group" data-username="alromz" data-post="3" data-topic="200041">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/41988e/48.png" class="avatar"> alromz:</div>
<blockquote>
<p><code>rand_like</code></p>
</blockquote>
</aside>
<p>My bad, I was using torch.rand_like, no torch.randn_like. <img src="https://discuss.pytorch.org/images/emoji/apple/face_with_peeking_eye.png?v=12" title=":face_with_peeking_eye:" class="emoji" alt=":face_with_peeking_eye:" loading="lazy" width="20" height="20"></p>