None gradients with nn.Parameter

username: _zero
cooked: <p>Hi,<br>
I have the following component that would need to do some operations:</p>
<ul>
<li>Store some tensors (var1)</li>
<li>Store some tensors that can be updated with autograd (var2)</li>
<li>Store something that keeps track of which tensor have been added (var3)</li>
<li>Count how many times every var2 was used (var4)</li>
</ul>
<p>The forward pass then computes similarities (according to some metric) between the input and var1, and returns the corresponding top k var2. I then do some operations on this result.</p>
<p>When I check with the code below I have two problems:</p>
<ol>
<li>I get the warning from checkpoint:</li>
</ol>
<blockquote>
<p>UserWarning: None of the inputs have requires_grad=True. Gradients will be None<br>
warnings.warn(“None of the inputs have requires_grad=True. Gradients will be None”)</p>
</blockquote>
<ol start="2">
<li><code>myThing.var2.grad</code> is None (before loss.backward())</li>
</ol>
<pre><code class="lang-auto">class MyThing(nn.Module):
    def __init__(self):
        super(MyThing, self).__init__()

        self.var1 = torch.zeros(0, 10, requires_grad=False)
        self.var2 = nn.Parameter(torch.rand(0, 5, requires_grad=True))
        self.var3 = defaultdict(bool)
        self.var4 = torch.zeros(0, 1, requires_grad=False)

    def add(self, elements, sorter):
        with torch.no_grad(): # I don't want to store gradients for adding to variables
            c = 0
            highest_sorted = sorter.argsort(dim=0, descending=True)
            elements = elements[highest_losses]

            for element in elements:
                if not self.var3[element]:
                    self.var1= torch.cat((self.var1, element.unsqueeze(0)), dim=0)
                    
                    to_add = torch.rand(1, 5, requires_grad=True)
                    self.memory_values = nn.Parameter(torch.cat((self.var2, to_add), dim=0)).requires_grad_()
                    
                    self.var4 = torch.cat((self.var4, torch.zeros(1,1)), dim=0)
                    c += 1
                self.var3[element] = True
                
                if c &gt;= SOME_MAXIMUM_VALUE:
                    break

    def forward(self, x):
        a = FC_LAYER_1(x)
        b = FC_LAYER_2(self.var1)
        
        sims = torch.matmul(a, b.t())

        idxs = sims.sort(dim=1, descending=True).indices
        k_highest_sims = smart_sort(sims, idxs)[:,:K]
        c = self.var2[idxs[:,:K]]

        self.var4[idxs[:,:K]] += 1
        return k_highest_sims, c
</code></pre>
<p>Code used for the forward pass outside the component:</p>
<pre><code class="lang-auto">x = SOME_TENSORS  # these have requires_grad set to True above
y = SOME_OTHER_TENSOR  # these have requires_grad set to True above
myThing = MyThing()

sims, outs = checkpoint(myThing, x) # need it for memory reasons. Warning here

z = FC_LAYER_3(sims)

result1 = FC_LAYER_4(y)
softmaxed_sims = F.softmax(sims, dim=1)
result2 = FC_LAYER_5(outs)

final_result = (result1 * (1-z) + (result2 * softmaxed_sims).sum(dim=1) * z)
</code></pre>
<p>The main problem is that the values in var2 always stay the same (confirming the None gradients).<br>
Am I doing something wrong?</p>

username: albanD
cooked: <p>The warning seem to indicate the issue: if nothing requires gradients, then nothing will be computed.<br>
You need to find where the Tensors stopped requirering gradients. This will point to the op that is non-differentiable,.</p>

username: _zero
cooked: <p>I would agree, but the problem is that I do <code>x.requires_grad = True</code> shortly before with no operations in between</p>

username: albanD
cooked: <p>Can you share that code? I don’t see any call to backward/grad here.</p>
<p>Also you should only set that flag for Tensors for which you want gradients to be computed. If you have to set it on an intermediary results, the backprop will stop there and not go any further.</p>

username: _zero
cooked: <p>Sure, the only thing that happens before the given code is</p>
<pre><code class="lang-auto">if flag:
    with torch.no_grad():
        outputs = main_model(tensor)
else:
    outputs = main_model(tensor)

SOME_TENSORS = outputs
if flag:
    SOME_TENSORS.requires_grad_() # also tried SOME_TENSORS.requires_grad = True
</code></pre>
<p>and after the given code:</p>
<pre><code class="lang-auto"># checking here gives None gradients
loss = metric(final_result, labels)
loss.backward()
# checking here gives None gradients
optimiser.step()
</code></pre>
<p>Even if x was set without <code>requires_grad</code>, shouldn’t var2 get updated? If not, could you explain why not?</p>

username: albanD
cooked: <p>If <code>flag</code> is set, you run the model in no_grad mode. This means that you won’t be able to compute any gradient for it.<br>
As I mentioned above, here you set the requires_grad flag on an intermediary Tensor. But that won’t make you able to backprop before it.<br>
If you set flag to False, it should work fine no?</p>

username: _zero
cooked: <p>At some point in the training I want to stop training the main model but only train the other FC layers. Is what I’m doing not the right way to do it? Shouldn’t var2 be updated even when the flag is set to True?</p>

username: albanD
cooked: <p>Ho the model above is not the same as the one defined in the nn.Module MyThing above. Ok.</p>
<p>Do the outputs of the checkpoint require gradients properly?</p>

username: _zero
cooked: <p><code>main_model</code> and <code>myThing</code> are two separate things, that’s why I’m confused.<br>
I will check the require_gradients of the output of the checkpoint too.</p>

username: _zero
cooked: <p>Hi Alban, sorry to bother you again. I have tried checking the <code>requires_grad</code> of all the variables involved and they are all fine except <code>c</code>. Although, all the consequent variables that make use of <code>c</code> have <code>requires_grad</code> set to <code>True</code></p>
<aside class="quote no-group quote-modified" data-username="_zero" data-post="1" data-topic="78097">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/_zero/48/31983_2.png" class="avatar"> _zero:</div>
<blockquote>
<pre><code class="lang-auto">c = self.var2[idxs[:,:K]]
# here c.requires_grad == False
</code></pre>
</blockquote>
</aside>
<p>Do you know why?</p>

username: albanD
cooked: <p>If self.var2.requires_grad=True, self.var2[idxs[:,:K]].requires_grad should also be True.<br>
The only reason why it would be false is if you run in <code>no_grad()</code> mode.</p>