AutoGrad not updating parameters

Jamesswiz: <p>Hi,<br>
I have the following code structure where I am updating the weights via manually defined parameter vector ‘u’</p>
<pre><code>class Net(torch.nn.Module):
    def __init__(self, W):
        super(Net, self).__init__()
    
        dim=100
        self.hidden1 = nn.Linear(dim, dim, bias=False)
        self.output = nn.Linear(dim, 10)
    
        self.hidden1.weight.requires_grad = False
         
        self.u = torch.nn.Parameter(torch.randn(dim,requires_grad=True))
        self.U = torch.ger(self.u,self.u)
       
        self.hidden1.weight.data=(self.U*W)
    

    
   def forward(self, x,):
      x = self.hidden1(x)
      x = self.output(x)
   return F.log_softmax(x,dim=1)

model = Net(W)
output = model(data, target)
loss = F.nll_loss(output, target)
loss.backward()
</code></pre>
<p>However, the parameter ‘u’ is not getting updated by loss.backward(). Can somebody help what is the mistake I am doing here?</p>

marcmuc: <p>Maybe move the requieres_grad to the Parameter(), it‘s currently in the inner bracket of torch.rand() and check the is_leaf() before and after. Maybe this note in the docs helps (and the docs to .is_leaf):</p>
<p><a href="https://pytorch.org/docs/stable/autograd.html#torch.Tensor.requires_grad" class="onebox" target="_blank" rel="nofollow noopener">https://pytorch.org/docs/stable/autograd.html#torch.Tensor.requires_grad</a></p>

Jamesswiz: <p>Hi <a class="mention" href="/u/marcmuc">@marcmuc</a>,<br>
Tried this but still it doesn’t work.</p>
<p>Yes the leaf is True. So how to fix this?</p>

albanD: <p>Hi,</p>
<p>The problem is with the fact that you use <code>.data</code>. You should not use <code>.data</code> anymore.<br>
There were 2 possible use case for <code>.data</code> and their new versions are:<br>
If you want to do ops that are not registered by the autograd engine, use <code>with torch.no_grad():</code>. If you want to break the computational graph to prevent gradients from flowing back, use <code>.detach()</code>.</p>
<p>You will see that what you try to do is not a valid use of <code>.data</code>.  And in particular, you hide the fact that you changed the weights from the autograd engine.<br>
You will need to do the computation of <code>U</code> and changing the weights inside your forward pass.<br>
Also <code>Linear</code> layers are not built to be used this way and so will prevent you from making their <code>.weight</code> something else than an <code>nn.Parameter</code>. I guess the simplest here is to do <code>self.hidden1.weight.copy_(torch.ger(self.u, self.u))</code> in your forward pass and keep the <code>requires_grad=False</code> in the <code>__init__</code> function.</p>

Jamesswiz: <p><a class="mention" href="/u/alband">@albanD</a><br>
thanks for solution, but I am new to pytorch and I get the following error with your solution</p>
<pre><code>Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.</code></pre>

albanD: <p>This error means that you computed something once and used it in multiple forward passes and thus multiple backward pass.<br>
Make sure that  your forward pass only uses the original tensor that you created with requires_grad=True (a leaf Tensor) and not intermediary results.<br>
In your case, as I said, U should be recomputed at every forward pass !</p>

Jamesswiz: <p><a class="mention" href="/u/alband">@albanD</a><br>
I didn’t get it…<br>
Should I compute U before forward function</p>
<p>U=torch.ger(self.u, self.u)<br>
def forward()<br>
self.hidden1.weight.copy_(U)</p>
<p>or inside</p>
<p>def forward()<br>
U=torch.ger(self.u, self.u)<br>
self.hidden1.weight.copy_(U)</p>

albanD: <p>You should compute it inside.</p>
<p>If it still give you an error please post a small code sample that reproduces the issue, that will be simpler to help.</p>

Jamesswiz: <p><a class="mention" href="/u/alband">@albanD</a> Here is the code after your suggestion</p>
<pre><code>class Net(torch.nn.Module):
    def __init__(self):
        super(Net, self).__init__()

        dim=100
        
        self.hidden1 = nn.Linear(dim, dim, bias=False)
        self.output = nn.Linear(dim, 10)

        self.hidden1.weight.requires_grad = False
        self.u = torch.nn.Parameter(torch.randn(dim), requires_grad=True)
        
    def forward(self, x, W):
        U = torch.ger(self.u,self.u)
        self.hidden1.weight.copy_(U*W)

        x = self.hidden1(x)
        x = self.output(x)
    return F.log_softmax(x,dim=1)

model = Net()
model = model.cuda()
output = model.forward(data, W)
loss = F.nll_loss(output, target)
loss.backward() 
</code></pre>
<p>Now the error is</p>
<pre><code>Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.</code></pre>

albanD: <p>Now you need to make sure that all your datas are on the same device.<br>
Be careful though that doing <code>y = x.cuda()</code> is an autograd operation and so will cause the same issue of doing backward twice if you use y in your forward pass while x requires gradients.</p>

Jamesswiz: <p><a class="mention" href="/u/alband">@albanD</a><br>
I have made the changes to code (see above) and passed the variable W to forward function directly instead of <em>init</em>(). Now both x and W are on cuda device, but still I am getting the same error.</p>

albanD: <p>Is your model on the gpu as well? You need to do <code>model.cuda()</code> as well.</p>

Jamesswiz: <p><a class="mention" href="/u/alband">@albanD</a><br>
Yes it is on gpu.</p>

albanD: <p>If you have</p>
<pre><code class="lang-auto">model.cuda()
data = data.cuda()
tareget = target.cuda()
W = W.cuda()
</code></pre>
<p>In your code then I’m not sure where the error can come form…</p>

Jamesswiz: <p><a class="mention" href="/u/alband">@albanD</a></p>
<p>Yes I have everything on cuda device as you mentioned, but still the same error. But thanks anyway for prompt responses. May be I need to do something else to make this work (not sure right now).</p>

Horse: <p>I found a way that works in general with autograd when replacing parameters with some tensor for backpropagation through that tensor: instead of</p>
<blockquote>
<p><code>self.hidden1.weight.copy_(U*W)
</code></p>
</blockquote>
<p>use</p>
<blockquote>
<p><code>self.hidden1._parameters['weight'] = U*W
</code></p>
</blockquote>
<p><code>module._parameters[&lt;name&gt;] = &lt;tensor&gt;
</code> sidesteps nn.Parameter and, as far as I can tell, just treats the tensor normally with autograd.</p>