How to sample from normal distribution with torch::Tensor parameters in C++?

username: cruzas
cooked: <p>Hi everyone,</p>
<p>I am wondering how to sample from a Gaussian distribution with mean mu, and standard deviation std in C++. Ideally, I would like to do this given that mu and std are the outputs of a NN, hence, torch::Tensor objects, so that autograd can keep track of their operations.</p>
<p>Is there a method that satisfies these requirements?</p>
<p>In Python, I believe this is solved using torch.distributions.Normal.</p>
<p>Thank you.</p>

username: cj0741
cooked: <p>I am facing the same prollem, using torch.distributions.Normal in C++.<br>
help~</p>

username: cruzas
cooked: <p>Hi <a class="mention" href="/u/cj0741">@cj0741</a>!</p>
<p>Hopefully this will help. Based on <a href="https://ch.mathworks.com/help/matlab/math/random-numbers-with-specific-mean-and-variance.html" rel="nofollow noopener">this link</a>, we can sample entries from a Normal distribution in Pytorch for C++ as follows:</p>
<pre><code class="lang-auto">auto sample = torch::randn({1})*sigma + mu;
</code></pre>
<p>which is similar to the implementation found <a href="https://github.com/pytorch/pytorch/blob/34523b70c1369024f805a0480bc39a5f21e64a03/aten/src/ATen/native/cuda/DistributionTemplates.h#L415" rel="nofollow noopener">here</a> (I thank <a class="mention" href="/u/glaringlee">@glaringlee</a> for having referred me to it a while ago).</p>
<p>Similarly, if you require to calculate the p.d.f., you could perhaps do it as follows, and also calculate the logprob with it after (see the implementation <a href="https://github.com/pytorch/pytorch/blob/8850fd1952c3983793dcac4022fdc8e3913dad96/torch/distributions/normal.py#L70" rel="nofollow noopener">here</a>):</p>
<pre><code class="lang-auto">auto pdf = (1.0 / (sigma * std::sqrt(2*M_PI))) * torch::exp(-0.5 * torch::pow((sample - mu) / sigma, 2));
</code></pre>
<p>Then, the logprob is:</p>
<pre><code class="lang-auto">auto log_prob = torch::log(pdf);
</code></pre>
<p>Feel free to ask any more questions!</p>

username: cj0741
cooked: <p>Super! that is exactly what I need.<br>
I am new to pytorch C++, if there are more questions, I will come back to you.<br>
Tanks~</p>

username: cj0741
cooked: <p>Hi <a class="mention" href="/u/cruzas">@cruzas</a>,<br>
I also have problem with convert between Tensor and C++ values such as vector, is there method in C++ API can do the same things like “x_inp.data.numpy()” and “torch.from_numpy” in Python?<br>
Thanks for your help.<br>
Jin</p>

username: cruzas
cooked: <p>Hi <a class="mention" href="/u/cj0741">@cj0741</a>,</p>
<p>I must admit I am no expert in Pytorch for C++. However, I do not know of a function in the API that does this conversion automatically.</p>
<p>The only way I am aware of  to access the data from a Tensor object, which could be used to build a C++ vector manually, is as follows:</p>
<pre><code class="lang-auto">auto sample_data = *(sample.data_ptr&lt;double&gt;())
</code></pre>
<p>Where <code>sample</code> is a Tensor object, assuming that it is a tensor of type double.</p>
<p>Be careful though. If you want to, for example, use backpropagation on a neural network, the values that you create must be linked to Tensor objects. If you directly take the data from a Tensor object, and somehow manipulate it, the autograd graph will not keep track of those operations. Again, I admit I am no expert, but that is how I have understood it so far. The documentation here is probably clearer in explaining this: <a href="https://pytorch.org/tutorials/beginner/blitz/autograd_tutorial.html" rel="nofollow noopener">https://pytorch.org/tutorials/beginner/blitz/autograd_tutorial.html</a>.</p>
<p>Feel free to post this as its own question on the forum, perhaps someone else knows. I would also be interested in knowing if such a function exists. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

username: cj0741
cooked: <p>Hi <a class="mention" href="/u/cruzas">@cruzas</a>  ,</p>
<p>Thank you for your help!</p>