How to create a custom parameter?

username: ADONAI_TZEVAOT
cooked: <p>I want to create a parameter v which is a vector and make sure that it is initialized to values that are non-zero. How do I do that?</p>

username: ptrblck
cooked: <p>You could create a tensor with values drawn from a Gaussian distribution and wrap it in an <code>nn.Parameter</code> as seen here:</p>
<pre><code class="lang-python">param = nn.Parameter(torch.randn(size))
</code></pre>

username: ADONAI_TZEVAOT
cooked: <p>So when the parameter updates will it only take values from randn? Also since 0 is one of the values in randn , how to make sure that none of the values in the tensor is 0?</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="ADONAI_TZEVAOT" data-post="3" data-topic="117190">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/adonai_tzevaot/48/31089_2.png" class="avatar"> ADONAI_TZEVAOT:</div>
<blockquote>
<p>So when the parameter updates will it only take values from randn?</p>
</blockquote>
</aside>
<p>No, since the gradient determines the updates, not the initial values.</p>
<aside class="quote no-group" data-username="ADONAI_TZEVAOT" data-post="3" data-topic="117190">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/adonai_tzevaot/48/31089_2.png" class="avatar"> ADONAI_TZEVAOT:</div>
<blockquote>
<p>Also since 0 is one of the values in randn , how to make sure that none of the values in the tensor is 0?</p>
</blockquote>
</aside>
<p>The probability to sample a “perfect” <code>0.0</code> value is not very high, but in case you want to make sure no zeros are there, you could replace them before creating the parameter:</p>
<pre data-code-wrap="python"><code class="lang-python">x = torch.randn(10)
x[0] = 0.0
zero_idx = x==0.0
while zero_idx.any():
    x[zero_idx] = torch.randn_like(x[zero_idx])
    zero_idx = x==0.0

param = nn.Parameter(x)
</code></pre>

username: ADONAI_TZEVAOT
cooked: <p>thanks. please could I also know how to make sure that the parameter values lie between -2000 and 1500 during gradient updates?</p>

username: ptrblck
cooked: <p>You could <code>.clamp_</code> the parameter after each gradient update to make sure it’s in the desired value range.</p>

username: ADONAI_TZEVAOT
cooked: <p>I have a custom parameter <code>W = nn.Parameter(torch.randn(1))</code> When i try to do <code>W += 2</code> or when i try to do <code>W.data += 2</code> , I get the following error:<br>
<code>RuntimeError: a leaf Variable that requires grad is being used in an in-place operation.</code><br>
Please help<br>
please tell me how to do inplace operations on custom parameters</p>

username: ptrblck
cooked: <p>If the parameter value is needed for the gradient computation, an inplace operation is disallowed, as explained in the error message.<br>
In such a case, remove the inplace operation and use the out-of-place one:</p>
<pre><code class="lang-python">x = x + 2
</code></pre>
<p>Also, don’t use the <code>.data</code> attribute, as it might yield unwanted side effects.</p>

username: ADONAI_TZEVAOT
cooked: <p>Its not working for a 2d parameter W<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/3/4/348ff521ae07fe083c6986870b2c13656e1e04fc.png" alt="image" data-base62-sha1="7uZfSnmXbfND2T5H9BBML3hpocc" width="524" height="228"></p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/6/d/6d040dcfbdcf00c5a65994acc1c5b4c982be6261.png" alt="image" data-base62-sha1="fyoJfsJm49WEIxirwqvdv71Ud69" width="575" height="231"><br>
<code>RuntimeError: a view of a leaf Variable that requires grad is being used in an in-place operation.</code><br>
Here,<br>
<code>z = torch.zeros(1)</code><br>
Please help</p>