nn.Parameter not getting updated not sure about the usage

username: mahmood
cooked: <p>Hello,<br>
I have declared two <code>nn.Parameter()</code> variables with <code>requires_grad=True</code> and I am using those in a different function that’s being called inside the init method of the class where variables are declared. <code>lparam</code> and <code>rparam</code> are not getting updated<br>
My question is am I doing it the right way?<br>
if not how it should be done?</p>
<p>here is the code example:</p>
<pre><code class="lang-auto">class LG(BaseNetwork):
    def __init__(self, opt):
        super().__init__()
        self.opt = opt
        self.lparam = nn.Parameter(torch.zeros(1), requires_grad=True).cuda(device=opt.gpu_ids[0])
        self.rparam = nn.Parameter(torch.zeros(1), requires_grad=True).cuda(device=opt.gpu_ids[0])
        
    def foo(self, a, b, k=1.0, lparam=0, rparam=0):
      t = bar(a, b, k=k, lparam=lparam, rparam=rparam)
      return t

    def forward(self, a, b):
       x = self.foo(a, b, k=self.opt.k, lparam=self.lparam, rparam=self.rparam)
       return x
</code></pre>
<p>BaseNetwork is just initializing functions  and uses <code>nn.Module</code></p>
<pre><code class="lang-auto">def bar(a, b, k=1.0, lparam=0, rparam=0):
    return n(a) * (b.std() * (k * lparam)) + (b.mean() * (k * rparam))
</code></pre>
<p>When I print the named params I can not get <code>lparam</code> and <code>rparam</code></p>

username: ptrblck
cooked: <p>The <code>.cuda()</code> operation on the <code>nn.Parameter</code> is differentiable and will create a non-leaf tensor.<br>
Remove the <code>.cuda()</code> operation and call it on the <code>nn.Module</code> instead or alternatively call it on the tensor before wrapping it into the <code>nn.Parameter</code>.</p>

username: mahmood
cooked: <p>Thank You so much <a class="mention" href="/u/ptrblck">@ptrblck</a> you’re mind saver <img src="https://discuss.pytorch.org/images/emoji/apple/smile.png?v=12" title=":smile:" class="emoji" alt=":smile:" loading="lazy" width="20" height="20"></p>

username: hzhou
cooked: <p>Hi, thanks for this! my problem is also solved by not using <code>cuda()</code>. But I am confused by this. Does this mean that, if I use <code>cuda()</code>, the <code>nn.Parameter</code> won’t be updated?</p>

username: ptrblck
cooked: <p>Yes, since the <code>cuda()</code> and <code>to()</code> operations are differentiable as explained before with recommendations of a proper usage (call these ops on the module or on the tensor before wrapping it into an <code>nn.Parameter</code>):</p><aside class="quote" data-post="2" data-topic="157226">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/nn-parameter-not-getting-updated-not-sure-about-the-usage/157226/2">nn.Parameter not getting updated not sure about the usage</a> 
  </div>
  <blockquote>
    The .cuda() operation on the nn.Parameter is differentiable and will create a non-leaf tensor. 
Remove the .cuda() operation and call it on the nn.Module instead or alternatively call it on the tensor before wrapping it into the nn.Parameter.
  </blockquote>
</aside>