TruncatedNormal loc argument

alicemabille: <p>I just discovered the truncated normal distribution and I’m trying to use torchrl TruncatedNormal but I can’t find anythin about “location” in the paper that introduced this idea and I don’t understand torchrl’s documentation :</p>
<blockquote>
<ul>
<li><strong>loc</strong> ([<em>torch.Tensor</em>]) – normal distribution location parameter</li>
</ul>
</blockquote>
<p>Can someone explain more in detail what that parameter is ? Is it like the <em>mean</em> parameter of torch.normal ?</p>

ptrblck: <p>Yes, that’s my understanding as the <code>location</code> parameter should define where the distribution is positioned and in the case of a normal distribution the <code>location</code> should correspond to the <code>mean</code>. CC <a class="mention" href="/u/kfrank">@KFrank</a> to correct me as I’m sure you can properly define and explain it.</p>

KFrank: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a> (and Alice)!</p>
<aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="220898" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>in the case of a normal distribution the <code>location</code> should correspond to the <code>mean</code>.</p>
</blockquote>
</aside>
<p>Yes, this is correct.  <code>loc</code> is the same as <code>mean</code> (and <code>scale</code> is <code>std</code>).  For the <code>Normal</code><br>
distribution (but not other distributions), it’s just a different name for the same thing.</p>
<p>It make sense to use a different name because, for example, the actual mean of an<br>
(asymmetrically-truncated) <code>TruncatedNormal</code> distribution will not be <code>loc</code> – that is<br>
the <code>mean</code> of the underlying <code>Normal</code> distribution – and the actual standard deviation<br>
will not be <code>scale</code> – the <code>std</code> of the underlying <code>Normal</code> distribution.</p>
<p><code>torchrl.modules.TruncatedNormal</code> follows <code>torch.distributions.normal.Normal</code><br>
for which:</p>
<blockquote>
<p><strong>loc</strong> (<em>float</em> <em>or</em> <em>Tensor</em>) – mean of the distribution (often referred to as mu)</p>
</blockquote>
<p>(I believe that<code>torch.distributions.Normal</code> uses <code>loc</code> and <code>scale</code>, rather than <code>mean</code><br>
and <code>std</code>, because there are other distributions in <code>torch.distributions</code> for which the<br>
conventional <code>loc</code> and <code>scale</code> parameters aren’t equal to the mean and standard deviation.<br>
On the other hand, <code>torch.normal()</code> calls the two parameters <code>mean</code> and <code>std</code>.)</p>
<p>Best.</p>
<p>K. Frank</p>

vmoens: <p>I can confirm, we use <code>loc</code> because <code>mean</code> is the “first momentum” of a distribution, but not a parameter of that distribution per se. It just happens that for the normal distribution, <code>E[X] = mu</code> but it’s not true in general. Calling it <code>mean</code> in Truncated / Tanh versions of the Normal distribution wouldn’t be correct as the mean may not even have a closed form formula.</p>