Nn.parameter re-define question

username: Ardeal
cooked: <p>Hi,<br>
Please check my sampler code as follows.<br>
my questions are:</p>
<ol>
<li>
<code>self.mask = self.mask_weight*temp</code>, by this line, could I keep self.mask_weght and self.mask are in the same memory? will they be upated in sync?</li>
<li>why could I not be able to <code> self.mask_weight = self.mask_weight * temp </code> to re-assign the parameter?</li>
<li>does it mean that nn.parameter could only be defined once, and could not be re-defined?</li>
</ol>
<pre><code class="lang-python">class SMConv2d(nn.Module):
    def __init__(self, in_channels, out_channels, kernel_size, padding=1, stride=1):
        super(SMConv2d, self).__init__()
    
        self.mask_weight = nn.Parameter(torch.Tensor(out_channels, in_channels, kernel_size, kernel_size))
        nn.init.constant_(self.mask_weight, 1)
    
    def compute_mask(self,temp):
        self.mask = self.mask_weight*temp
        # self.mask_weight = self.mask_weight * temp
       
        return self.mask 

    def forward(self, x, temp=1):
        masked_weight = self.compute_mask(temp)
        out = F.conv2d(x, masked_weight, stride=self.stride, padding=self.padding)        
        return out
</code></pre>

username: gphilip
cooked: <aside class="quote no-group" data-username="Ardeal" data-post="1" data-topic="129523">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ardeal/48/33257_2.png" class="avatar"> Ardeal:</div>
<blockquote>
<p>why could I not be able to <code>self.mask_weight = self.mask_weight * temp</code> to re-assign the parameter?</p>
</blockquote>
</aside>
<p>What is the error you got when you tried to do this?</p>

username: Ardeal
cooked: <p>Hi,<br>
<a class="mention" href="/u/gphilip">@gphilip</a></p>
<aside class="quote no-group" data-username="gphilip" data-post="2" data-topic="129523">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/g/858c86/48.png" class="avatar"> gphilip:</div>
<blockquote>
<p>What is the error you got when you tried to do this?</p>
</blockquote>
</aside>
<p>I got the following error:</p>
<pre data-code-wrap="python"><code class="lang-python">cannot assign 'torch.cuda.FloatTensor' as parameter 'mask_weight' (torch.nn.Parameter or None expected)

</code></pre>
<p><strong>One more question/obervation:</strong><br>
if I use the code <code>self.mask_weight.data = self.mask_weight.data*temp</code>, there is no error,<br>
however, <code>self.mask_weight</code> will not be optimized during the training.<br>
is there any good syntax to do this?</p>

username: gphilip
cooked: <p>An <code>nn.Parameter</code> is something that the network learns (by doing SGD, for instance). This is probably why an error is reported when you try to set its value yourself.</p>
<p>Do you want the network to learn <code>SMConv2d.mask_weight</code> on its own, or do you want to be able to set it yourself? I don’t think you can have it both ways (but I may be wrong).</p>
<aside class="quote no-group" data-username="Ardeal" data-post="3" data-topic="129523">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ardeal/48/33257_2.png" class="avatar"> Ardeal:</div>
<blockquote>
<p>if I use the code <code>self.mask_weight.data = self.mask_weight.data*temp</code>, there is no error,<br>
however, <code>self.mask_weight</code> will not be optimized during the training.<br>
is there any good syntax to do this?</p>
</blockquote>
</aside>
<p>I don’t know for you sure, but I <em>think</em> this would not be possible. I think (again, I am not sure) that if you want some parameter to be optimized by the training, then you cannot also change it in arbitrary ways outside of the training.</p>

username: Ardeal
cooked: <p><a class="mention" href="/u/gphilip">@gphilip</a><br>
Thank you!</p>
<p>I agree with what you said. however, you said “but I may be wrong”, I would like to further confirm/make the thing clear.</p>
<p>accoring to what you said, nn.parameter could be defined, initialized, and referred, and could not be re-asigined.</p>
<p>Is there any official explaination/doc about my question?<br>
are you pytorch official developer?</p>

username: gphilip
cooked: <aside class="quote no-group" data-username="Ardeal" data-post="5" data-topic="129523">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ardeal/48/33257_2.png" class="avatar"> Ardeal:</div>
<blockquote>
<p>are you pytorch official developer?</p>
</blockquote>
</aside>
<p>I am not.</p>
<aside class="quote no-group" data-username="Ardeal" data-post="5" data-topic="129523">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ardeal/48/33257_2.png" class="avatar"> Ardeal:</div>
<blockquote>
<p>Is there any official explaination/doc about my question?</p>
</blockquote>
</aside>
<p>Did you try asking Google?</p>

username: Ardeal
cooked: <aside class="quote no-group" data-username="gphilip" data-post="6" data-topic="129523">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/g/858c86/48.png" class="avatar"> gphilip:</div>
<blockquote>
<p>Did you try asking Google?</p>
</blockquote>
</aside>
<p>I have roughly asked Google, however, Google didn’t answer such a specific quesiton.</p>