Parameters with zero .grad change in value - how to exclude some parameters from backpropagation update

venkatasg: <p>I have a custom Module with certain parameters. One of them is a ModuleDict() which has 2 parameters(Linear modules) for key values say ‘a’ and ‘b’ . Depending on the training example(one minibatch) provided, only one of these parameters should be used per minibatch in the forward pass - they key value for the moduledict is decided by the minibatch. I might be seriously misunderstanding pyTorch and NN’s in general here, but is it not the case that if a certain parameter is not used at all in the forward pass, its gradient(param.grad.data.sum()) should be zero AND it should not change in value at all for that minibatch? The gradient is zero, however the value of both the parameters(in this case, the weights and biases of the Linear modules) change every minibatch - only one of them should change.</p>
<p>A side question - Is there a good way to visualize the history a variable while debugging in python? Preferably not inside a jupyter notebook.</p>

tom: <aside class="quote no-group" data-username="venkatasg" data-post="1" data-topic="28467">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/v/a587f6/48.png" class="avatar"> venkatasg:</div>
<blockquote>
<p>The gradient is zero, however the value of both the parameters(in this case, the weights and biases of the Linear modules) change every minibatch - only one of them should change.</p>
</blockquote>
</aside>
<p>You’re not using an optimizer with momentum by chance?<br>
For those, the momentum will cause updates even when the gradients are zero.<br>
(It’s also doing funny things to the statistics, probably, but with dropout we rarely think about it too much.)</p>
<p>As a trick you can (at least you could last time I checked) set the gradients to “None” instead of just zeroing them and then the parameters won’t be updated.</p>
<aside class="quote no-group">
<blockquote>
<p>A side question - Is there a good way to visualize the history a variable while debugging in python? Preferably not inside a jupyter notebook.</p>
</blockquote>
</aside>
<p>Some people like TensorboardX.</p>
<p>Best regards</p>
<p>Thomas</p>

venkatasg: <p>You’re exactly right, I’m using Adam. I checked the update rule and its the momentum vectors that are updating the gradients. I should give this more thought. Thank you!</p>

SofiaCP: <aside class="quote no-group" data-username="tom" data-post="2" data-topic="28467">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/tom/48/3162_2.png" class="avatar"> tom:</div>
<blockquote>
<p>“None”</p>
</blockquote>
</aside>
<p>Hi <a class="mention" href="/u/tom">@tom</a>, this solution doesn’t seem to work for me. I only want to make certain parts of the gradient None. Is that possible?</p>
<p>I don’t want to update parameters where the grad is zero, so I try to make them <code>None</code> like this</p>
<pre><code class="lang-auto">model.fc.weight.grad[model.fc.weight==0]=None
</code></pre>
<p>but I’m getting the error <code>can't assign a NoneType to a torch.cuda.FloatTensor</code></p>

tom: <p>You can zero part of the gradients if that’s what you need. Note, however, that some optimizers might react to this (anything weight decay, things taking tensor norms).</p>
<p>Best regards</p>
<p>Thomas</p>

SofiaCP: <p>Hi <a class="mention" href="/u/tom">@tom</a>, thanks for replying. I am in fact zeroing out part of the gradient and that is not a problem. The problem is that even though that part of the gradient is zero, the corresponding parameters still get updated because of Adam’s momentum. For that reason, I was trying to set part of the gradient to <code>None</code> instead of <code>0</code> but it throws the error I’ve mentioned.</p>
<p>Do you know how to solve this?<br>
Thanks</p>