Check Intermediate gradients when performing several .backward() passes

username: LeoSerena
cooked: <p>Hello.</p>
<p>In the context of a central server and several nodes that all have a similar model with different weights, I compute and back-propagate the loss of a norm on the weights on the central model given all the updated nodes models. I want to compute the gradient norm of the difference between the central model and the nodes for every node separetly. The thing is that I load the weigths of the nodes one by one, compute the backward pass on the general model, and finally perform the update step:</p>
<pre><code class="lang-auto">    optimizer.zero_grad()
    for i, node in enumerate(nodes):
        user_modelname = f'user_models/user_{i}.pth'
        user_model.load_model(name = user_modelname)
        reg = models_diff(simple_model, user_model)
        reg.backward()
    optimizer.step()
</code></pre>
<p>My question is the following: when I check the <code>p.grad</code> of the general model parameters, I get the sum of all the gradients, but I would like the gradients individually of every node and I can’t perform one update at a time since the model difference would change at every iteration, is there a way to get the intermediate gradient values of the backward pass?</p>

username: ptrblck
cooked: <p>You could probably use <code>grads = torch.autograd.grads</code> which would return the gradients directly instead of using <code>reg.backwar()</code>.</p>

username: LeoSerena
cooked: <p>But what if I want to still apply the backpropagation? My temporary solution for now is to store the gradient difference between two nodes but this seems inefficient and hacky.</p>

username: ptrblck
cooked: <p>I’m not sure I understand the question correctly. <code>autograd.grad</code> will calculate the gradients as well, but won’t accumulate them in the <code>.grad</code> attributes of the leaf tensors, but will return them instead.</p>

username: LeoSerena
cooked: <p>Ok I got it (I didn’t quite understood how <code>torch.autograd.grad</code> worked).<br>
Basically, for every layer that has its <code>requires_grad = True</code>, I can perform:</p>
<pre><code class="lang-auto">torch.autograd.grad(reg, layer)
</code></pre>
<p>ant it returns the gradient of the regularization on the layer. Thanks a lot.</p>
<p>Just an auxiliary question though: do we agree that this does not update the <code>.grad</code> value of the layer and that I can still call <code>.backward()</code> like normal afterwards?</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="LeoSerena" data-post="5" data-topic="138938">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/leoserena/48/44755_2.png" class="avatar"> LeoSerena:</div>
<blockquote>
<p>Just an auxiliary question though: do we agree that this does not update the <code>.grad</code> value of the layer and that I can still call <code>.backward()</code> like normal afterwards?</p>
</blockquote>
</aside>
<p>Yes, as seen here:</p>
<pre data-code-wrap="python"><code class="lang-python">lin = nn.Linear(1, 1)
x = torch.randn(1, 1)

out = lin(x)

grads = torch.autograd.grad(out, lin.parameters())
print(grads)
&gt; (tensor([[1.3361]]), tensor([1.]))
print([p.grad for p in lin.parameters()])
&gt; [None, None]
</code></pre>

username: LeoSerena
cooked: <p>Perfect, thank you again.</p>