No gradient in layers text classification tutorial

username: anna_l
cooked: <p>Hi!</p>
<p>I was running the tutorial on text classification, exactly as in <a href="https://github.com/pytorch/opacus/blob/master/tutorials/building_text_classifier.ipynb" class="inline-onebox" rel="noopener nofollow ugc">opacus/building_text_classifier.ipynb at master · pytorch/opacus · GitHub</a>, but I get the following error when I try to train:</p>
<p>AttributeError: The following layers do not have gradients: [‘module.bert.encoder.layer.11.attention.self.query.weight’, ‘module.bert.encoder.layer.11.attention.self.query.bias’, ‘module.bert.encoder.layer.11.attention.self.key.weight’, ‘module.bert.encoder.layer.11.attention.self.key.bias’, ‘module.bert.encoder.layer.11.attention.self.value.weight’, ‘module.bert.encoder.layer.11.attention.self.value.bias’, ‘module.bert.encoder.layer.11.attention.output.dense.weight’, ‘module.bert.encoder.layer.11.attention.output.dense.bias’, ‘module.bert.encoder.layer.11.attention.output.LayerNorm.weight’, ‘module.bert.encoder.layer.11.attention.output.LayerNorm.bias’, ‘module.bert.encoder.layer.11.intermediate.dense.weight’, ‘module.bert.encoder.layer.11.intermediate.dense.bias’, ‘module.bert.encoder.layer.11.output.dense.weight’, ‘module.bert.encoder.layer.11.output.dense.bias’, ‘module.bert.encoder.layer.11.output.LayerNorm.weight’, ‘module.bert.encoder.layer.11.output.LayerNorm.bias’, ‘module.bert.pooler.dense.weight’, ‘module.bert.pooler.dense.bias’, ‘module.classifier.weight’, ‘module.classifier.bias’]. Are you sure they were included in the backward pass?</p>
<p>Could someone help me understand why this is happening?<br>
I’m on ubuntu and am using python 3.8.5</p>
<p>cheers!</p>

username: ptrblck
cooked: <p>Based on cell 8 it seems you are freezing some layers and train only others:</p>
<pre><code class="lang-python">trainable_layers = [model.bert.encoder.layer[-1], model.bert.pooler, model.classifier]
total_params = 0
trainable_params = 0

for p in model.parameters():
        p.requires_grad = False
        total_params += p.numel()

for layer in trainable_layers:
    for p in layer.parameters():
        p.requires_grad = True
        trainable_params += p.numel()

print(f"Total parameters count: {total_params}") # ~108M
print(f"Trainable parameters count: {trainable_params}") # ~7M
</code></pre>
<p>so I would assume that the frozen parameters do not have valid gradients.<br>
I’m however unsure where this message is raised from and if it’s an error etc. so could you explain the issue a bit more?</p>

username: anna_l
cooked: <p>Hmm makes sense. The issue arises when virtual_step() is called:<br>
… in <br>
optimizer.virtual_step()<br>
… line 282, in virtual_step<br>
self.privacy_engine.virtual_step()<br>
… line 435, in virtual_step<br>
self.clipper.clip_and_accumulate()<br>
…  line 179, in clip_and_accumulate<br>
named_params=self._named_grad_samples(),<br>
… line 263, in _named_grad_samples<br>
where the error is thrown</p>

username: ptrblck
cooked: <p>I’m unsure what <code>virtual_step()</code> does and assume it’s coming from a 3rd party library?<br>
Do you know, if this method expects all <code>.grad</code> attributes to be set and if so, could you filter the frozen parameters out while passing them to the optimizer?</p>

username: ffuuugor
cooked: <p>Hi <a class="mention" href="/u/anna_l">@anna_l</a> !<br>
Thanks for your question and for taking interest in opacus.</p>
<p>I’d need some more info to be able to help, as I wasn’t able to reproduce the issue in my setup.</p>
<ul>
<li>Can you please share which versions of <code>transformers</code> and <code>opacus</code> are you using?</li>
<li>Does the error happen on the first training iteration or later?</li>
</ul>
<p>To comment on some of the discussion points above:</p>
<ul>
<li>
<code>virtual_step()</code> is a method defined in <code>PrivacyEngine</code> in <code>opacus</code>. It a way to simulate large batches without heavy memory footprint.</li>
<li>In our tutorial we indeed freeze some layers, as correctly pointed out. However, the error above lists trainable layers as not having gradients, which is not what should happen. (e.g. <code>bert.encoder.layer.11</code> is <code>bert.encoder.layer[-1]</code>)</li>
</ul>

username: anna_l
cooked: <p>Hi <a class="mention" href="/u/ffuuugor">@ffuuugor</a>, pardon slow reply. It happens on the first training iteration, my transformers version is 4.6.1</p>

username: ffuuugor
cooked: <p>Hey<br>
Sorry, but I’m still having trouble reproducing the issue.<br>
I’ve tried multiple package versions (opacus 0.13, 0.14, master), but none produce the error you’ve described.</p>
<p>Can you maybe share a Colab notebook with the error to help find the reason?</p>
<p>PS: While investigating this we’ve <a href="https://github.com/pytorch/opacus/pull/192" rel="noopener nofollow ugc">found and fixed</a> quite bad memory inefficiency, so thanks for pointing that way <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>