Generating random tensors according to the uniform distribution pytorch?

pinocchio: <p>I saw:</p>
<aside class="onebox stackexchange">
  <header class="source">
      <a href="https://stackoverflow.com/questions/44328530/how-to-get-a-uniform-distribution-in-a-range-r1-r2-in-pytorch" target="_blank" rel="nofollow noopener">stackoverflow.com</a>
  </header>
  <article class="onebox-body">
      <a href="https://stackoverflow.com/users/4933403/bishwajit-purkaystha" target="_blank" rel="nofollow noopener">
    <img alt="Bishwajit Purkaystha" src="https://lh3.googleusercontent.com/-gLU3P3k3wpo/AAAAAAAAAAI/AAAAAAAAAbA/zpa4QluGQg0/photo.jpg?sz=128" class="thumbnail onebox-avatar" width="60" height="60">
  </a>
<h4>
  <a href="https://stackoverflow.com/questions/44328530/how-to-get-a-uniform-distribution-in-a-range-r1-r2-in-pytorch" target="_blank" rel="nofollow noopener">How to get a uniform distribution in a range [r1,r2] in PyTorch?</a>
</h4>

<div class="tags">
  <strong>pytorch, uniform-distribution</strong>
</div>

<div class="date">
  asked by
  
  <a href="https://stackoverflow.com/users/4933403/bishwajit-purkaystha" target="_blank" rel="nofollow noopener">
    Bishwajit Purkaystha
  </a>
  on <a href="https://stackoverflow.com/questions/44328530/how-to-get-a-uniform-distribution-in-a-range-r1-r2-in-pytorch" target="_blank" rel="nofollow noopener">12:05PM - 02 Jun 17 UTC</a>
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p>and thought that was a strange way to do it.</p>
<p>So there isn’t something default like <code>torch.uniform</code>?</p>
<hr>
<p>their code:</p>
<pre><code class="lang-auto">If U is a random variable uniformly distributed on [0, 1], then (r1 - r2) * U + r2 is uniformly distributed on [r1, r2].

Thus, you just need:

(r1 - r2) * torch.rand(a, b) + r2
Alternatively, you can simply use:

torch.FloatTensor(a, b).uniform_(r1, r2)
</code></pre>

ptrblck: <p><code>toch.rand</code> returns a tensor samples uniformly in <code>[0, 1)</code>. Scaling it as shown in your example should work.<br>
Also, the second approach is fine.</p>
<p>If you would like to have a standalone <code>torch.uniform</code> method, I would recommend to create an issue in github and start the discussion there. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

pinocchio: <aside class="quote no-group" data-username="pinocchio" data-post="1" data-topic="53030">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/pinocchio/48/22093_2.png" class="avatar"> pinocchio:</div>
<blockquote>
<p>If U is a random variable uniformly distributed on [0, 1], then (r1 - r2) * U + r2 is uniformly distributed on [r1, r2]. Thus, you just need: (r1 - r2) * torch.rand(a, b) + r2 Alternatively, you can simply use: torch.FloatTensor(a, b).uniform_(r1, r2)</p>
</blockquote>
</aside>
<p>ok! My first time doing this, hopefully this is the right way to do it:</p>
<aside class="onebox githubissue" data-onebox-src="https://github.com/pytorch/pytorch/issues/24162">
  <header class="source">

      <a href="https://github.com/pytorch/pytorch/issues/24162" target="_blank" rel="noopener nofollow ugc">github.com/pytorch/pytorch</a>
  </header>

  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewBox="0 0 14 16" aria-hidden="true"><path fill-rule="evenodd" d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/pytorch/issues/24162" target="_blank" rel="noopener nofollow ugc">Implement torch.uniform</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2019-08-11" data-time="18:06:12" data-timezone="UTC">06:06PM - 11 Aug 19 UTC</span>
      </div>

        <div class="date">
          closed <span class="discourse-local-date" data-format="ll" data-date="2019-08-12" data-time="20:05:06" data-timezone="UTC">08:05PM - 12 Aug 19 UTC</span>
        </div>

      <div class="user">
        <a href="https://github.com/renesax14" target="_blank" rel="noopener nofollow ugc">
          <img alt="renesax14" src="https://avatars.githubusercontent.com/u/20900122?v=4" class="onebox-avatar-inline" width="20" height="20">
          renesax14
        </a>
      </div>
    </div>

    <div class="labels">
    </div>
  </div>
</div>

  <div class="github-row">
    <p class="github-body-container">## 🚀 Feature
sample uniform vectors

## Motivation

Have a out of the box u<span class="show-more-container"><a href="" rel="noopener" class="show-more">…</a></span><span class="excerpt hidden">niform samples

## Pitch

`x = torch.uniform(a,b)`

## code

```

def uniform(a,b):
'''
If U is a random variable uniformly distributed on [0, 1], then (r1 - r2) * U + r2 is uniformly distributed on [r1, r2].
'''

u = (r1 - r2) * torch.rand(a, b) + r2
u = torch.FloatTensor(a, b).uniform_(r1, r2)
return u

```</span></p>
  </div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>


dugr: <p>Hello,</p>
<p>yes this is true. According to the PyTorch documentation:<br>
torch.rand returns a tensor filled with random numbers from a uniform distribution on the interval [0,1)<br>
<a href="https://pytorch.org/docs/stable/torch.html#torch.rand" class="onebox" target="_blank" rel="nofollow noopener">https://pytorch.org/docs/stable/torch.html#torch.rand</a><br>
I would appreciate if you can explain the following. I am trying to figure out how the authors of the papers generate normal and uniform distribution:<br>
“<strong>Gaussian Noise.</strong> The synthetic Gaussian noise dataset consists of 10,000 random 2D Gaussian<br>
noise images, where each RGB value of every pixel is sampled from an i.i.d Gaussian distribution<br>
with mean 0.5 and unit variance. We further clip each pixel value into the range [0, 1].<br>
<strong>Uniform Noise.</strong> The synthetic uniform noise dataset consists of 10,000 images where each RGB<br>
value of every pixel is independently and identically sampled from a uniform distribution on [0, 1].”<br>
</p><aside class="onebox whitelistedgeneric">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="32" height="32">
      <a href="https://github.com/facebookresearch/odin" target="_blank" rel="nofollow noopener">GitHub</a>
  </header>
  <article class="onebox-body">
    <img src="https://avatars1.githubusercontent.com/u/16943930?s=400&amp;v=4" class="thumbnail onebox-avatar" width="400" height="400">

<h3><a href="https://github.com/facebookresearch/odin" target="_blank" rel="nofollow noopener">facebookresearch/odin</a></h3>

<p>A simple and effective method for detecting out-of-distribution images in neural networks. - facebookresearch/odin</p>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p>In the code they seemed to use this for generating Gaussian dataset:</p>
<pre><code class="lang-auto">        images = torch.randn(1, 3, 32, 32) + 0.5
        images = torch.clamp(images, 0, 1)
</code></pre>
<p>and this for generating Uniform dataset:</p>
<pre><code class="lang-auto">       images = torch.rand(1, 3, 32, 32)
</code></pre>
<p>According to your explanation the generation of the Uniform dataset is being correct.<br>
What I would like to know is how they used the <em>rand</em> for the Gaussian dataset, although this discussion aligns with what they did:</p>
<aside class="onebox stackexchange">
  <header class="source">
      <a href="https://stackoverflow.com/questions/51136581/how-to-create-a-normal-distribution-in-pytorch" target="_blank" rel="nofollow noopener">stackoverflow.com</a>
  </header>
  <article class="onebox-body">
      <a href="https://stackoverflow.com/users/9242759/dq-shen" target="_blank" rel="nofollow noopener">
    <img alt="dq.shen" src="https://i.stack.imgur.com/VbKJE.jpg?s=128&amp;g=1" class="thumbnail onebox-avatar" width="128" height="128">
  </a>
<h4>
  <a href="https://stackoverflow.com/questions/51136581/how-to-create-a-normal-distribution-in-pytorch" target="_blank" rel="nofollow noopener">How to create a normal distribution in pytorch</a>
</h4>

<div class="tags">
  <strong>python, statistics, pytorch, linear-algebra</strong>
</div>

<div class="date">
  asked by
  
  <a href="https://stackoverflow.com/users/9242759/dq-shen" target="_blank" rel="nofollow noopener">
    dq.shen
  </a>
  on <a href="https://stackoverflow.com/questions/51136581/how-to-create-a-normal-distribution-in-pytorch" target="_blank" rel="nofollow noopener">12:46PM - 02 Jul 18 UTC</a>
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
<br>
<em>A simple option is to use the  <code>randn</code>  function from the base module. It creates a random sample from the standard Gaussian distribution. To change the mean and the standard deviation you just use addition and multiplication. Below I create sample of size 5 from your requested distribution.</em>
<pre><code class="lang-auto">import torch
torch.randn(5) * 0.5 + 4 # tensor([4.1029, 4.5351, 2.8797, 3.1883, 4.3868])
</code></pre>
<p>I am still little confused about how they could use “rand” function for generating Gaussian images?<br>
Thank you</p>

ptrblck: <aside class="quote no-group" data-username="dugr" data-post="4" data-topic="53030">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/dugr/48/22382_2.png" class="avatar"> dugr:</div>
<blockquote>
<p>I am still little confused about how they could use “rand” function for generating Gaussian images?</p>
</blockquote>
</aside>
<p>They are not using <code>rand</code> but <code>randn</code>, which is the normal (gaussian) distribution.</p>

Brando_Miranda: <p>I find this very confusing naming. Why have <code>rand</code> and <code>randn</code>? It’s not very descriptive names of what they are.</p>
<p>For example, I assumed the default way to sample was normal for a long time…</p>

Brando_Miranda: <p>Anyway, do you mind providing a good example of how to use the <code>torch.distributions.Uniform</code> interface?</p>
<p>I saw the docs and the example in the linked issue but wasn’t very helpful. In particular what passing tensors of size larger than 1 do?</p>
<p>my specific question can be found here: <a href="https://github.com/pytorch/pytorch/issues/24162#issuecomment-658861894" class="inline-onebox" rel="noopener nofollow ugc">Implement torch.uniform · Issue #24162 · pytorch/pytorch · GitHub</a></p>
<p>copy pasted:</p>
<blockquote>
<blockquote>
<p>I think this already exists?</p>
<pre data-code-wrap="python"><code class="lang-python">a = torch.tensor([0., 10.])
b = torch.tensor([1., 11.])
torch.distributions.Uniform(a, b).sample()
&gt;&gt;&gt; tensor([ 0.8583, 10.0226])
</code></pre>
</blockquote>
<pre><code class="lang-auto">
This example is confusing me. What does uniform do when receiving tensors of size larger than 1x1 mean? Whats the behavior when they are greater than 1 dimension?

I saw the docs but they weren't helpful nor any example or discussion of the case you provides is given: https://pytorch.org/docs/stable/distributions.html#torch.distributions.uniform.Uniform

What I want is to replicate this numpy code:

</code></pre>
<p>xn = np.random.uniform(low=-1, high=1, size=(num_samples,Din))</p>
<pre><code class="lang-auto"></code></pre>
</blockquote>
<hr>
<p>Also what is the use of creating a sampler object first?</p>

Brando_Miranda: <p>This is the way I found works:</p>
<pre><code class="lang-auto"># generating uniform variables

import numpy as np

num_samples = 3
Din = 1
lb, ub = -1, 1

xn = np.random.uniform(low=lb, high=ub, size=(num_samples,Din))
print(xn)

import torch

sampler = torch.distributions.Uniform(low=lb, high=ub)
r = sampler.sample((num_samples,Din))

print(r)

r2 = torch.torch.distributions.Uniform(low=lb, high=ub).sample((num_samples,Din))

print(r2)

# process input
f = nn.Sequential(OrderedDict([
    ('f1', nn.Linear(Din,Dout)),
    ('out', nn.SELU())
]))
Y = f(r2)
print(Y)
</code></pre>
<p>but I have to admit I don’t know what the point of generating sampler is and why not just call it directly as I do in the one liner (last line of code).</p>
<p>Comments:</p>
<ul>
<li>sampler are good for it’s so you can transform/compose/cache/etc distributions. see <a href="https://arxiv.org/abs/1711.10604" rel="nofollow noopener">https://arxiv.org/abs/1711.10604</a>, and the top of the docs of <a href="https://pytorch.org/docs/stable/distributions.html#" rel="nofollow noopener">https://pytorch.org/docs/stable/distributions.html#</a> and <a href="https://arxiv.org/abs/1506.05254" rel="nofollow noopener">https://arxiv.org/abs/1506.05254</a>
</li>
<li>you can feed in tensors to uniform to let it know the high dimensional interval (hypercube) to generate the uniform samples (that’s why it receives tensors as input rather than simply numbers)</li>
</ul>
<hr>
<p>Reference:</p>
<ul>
<li><a href="https://stackoverflow.com/questions/44328530/how-to-get-a-uniform-distribution-in-a-range-r1-r2-in-pytorch/62919760#62919760" rel="nofollow noopener">https://stackoverflow.com/questions/44328530/how-to-get-a-uniform-distribution-in-a-range-r1-r2-in-pytorch/62919760#62919760</a></li>
<li><a href="https://discuss.pytorch.org/t/generating-random-tensors-according-to-the-uniform-distribution-pytorch/53030/8" class="inline-onebox">Generating random tensors according to the uniform distribution pytorch?</a></li>
<li><a href="https://github.com/pytorch/pytorch/issues/24162" rel="nofollow noopener">https://github.com/pytorch/pytorch/issues/24162</a></li>
</ul>

ptrblck: <aside class="quote no-group" data-username="Brando_Miranda" data-post="6" data-topic="53030">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/brando_miranda/48/14355_2.png" class="avatar"> Brando_Miranda:</div>
<blockquote>
<p>Why have <code>rand</code> and <code>randn</code> ?</p>
</blockquote>
</aside>
<p>These function names are taken from numpy, which adapted them from MATLAB, if I’m not mistaken, so they are commonly used in scientific software.</p>
<p>Good to hear your <code>Uniform</code> distribution works. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>

Brando_Miranda: <p>Thanks for the reply!</p>
<p>I think what puts me off is that the interface isn’t that similar to numpy. e.g. even if those names are the same there is a <code>.random</code> to call them in numpy but there is not in pytorch which is where my confusion and me searching for uniform came about.</p>
<p>For numpy it’s <code>numpy.random.rand or randn</code> while in pytorch you can just do <code>torch.randn or rand</code> and for uniform one has to do <code>torch.distribution.uniform.Uniform or distribution.Distribution</code>.</p>
<p>Anyway, thanks for clarifying. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
<p>Just curious, why doesn’t it match numpy more closely?</p>

ptrblck: <aside class="quote no-group" data-username="Brando_Miranda" data-post="10" data-topic="53030">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/brando_miranda/48/14355_2.png" class="avatar"> Brando_Miranda:</div>
<blockquote>
<p>Just curious, why doesn’t it match numpy more closely?</p>
</blockquote>
</aside>
<p>I think these were early design decisions, which were already introduced in Torch7 (Lua).<br>
Once these functions are commonly used, it’s hard to change them due to backwards compatibility. Especially once you reach a non-beta version.</p>

Brando_Miranda: <p>Thanks for taking the time to explain. <img src="https://discuss.pytorch.org/images/emoji/apple/brain.png?v=9" title=":brain:" class="emoji" alt=":brain:"></p>
<p>Cheers!</p>