Not updating the BERT embeddings during the fine tuning process
Author: avisil
Pull request: no

Is there any way of not updating the BERT embeddings during the fine tuning process? For example while running on SQUAD, I want to see the effect of not updating the parameters associated with the BERT embeddings. I saw that `required_grad` is set to True for cpu and fp16. Which makes me think that it's assuming `do_grad` for all the parameters. 

I'm asking if there's any quick way to disable the update to those embeddings but let the model update other parameters.

Comments:
You can do it by setting the `requires_grad` attribute of the embedding layer in `BertModel`. That will look something like this: 

```
    model = BertForQuestionAnswering.from_pretrained(args.bert_model,
                cache_dir=PYTORCH_PRETRAINED_BERT_CACHE / 'distributed_{}'.format(args.local_rank))
    model.bert.embeddings.requires_grad = False
```

I haven't tested this code but it should do what you are asking.

More explanation are available on the [PyTorch forums](https://discuss.pytorch.org/t/how-the-pytorch-freeze-network-in-some-layers-only-the-rest-of-the-training/7088)

Thanks let me try this..I was thinking of going through the BERTAdam optimizer.


With `requires_grad=false`: {"exact_match": 80.65279091769158, "f1": 88.04683744174879}
Without `requires_grad=false`: {"exact_match": 81.33396404919584, "f1": 88.43774959214048}

Thanks for your feedback. 

Have you checked that the values of the embedding matrix are indeed unchanged by the finetuning?

Nope I haven't.. I plan to do that today. @thomwolf do you think we're on the right track? Just need your 2 cents :) 

Yes you can do
```python
for p in model.bert.embeddings.parameters():
    p.requires_grad = False
```
You can also just not send these parameters to the optimizer (when you create the optimizer) as detailed on the PyTorch forums. Both methods will work. Combining the two will gives the lowest overhead (no un-necessary computation of gradient and no un-necessary check of update during the optimizer step). The PyTorch forum is the best reference for this kind of general question.

Correct me if I'm wrong, but setting `model.bert.embeddings.requires_grad = False` does not seem to propagate.

```python
bert = BertModel.from_pretrained('bert-base-uncased')
bert.embeddings.requires_grad = False
for name, param in bert.named_parameters():                
    if param.requires_grad:
        print(name)
```

Output:

> embeddings.word_embeddings.weight
> embeddings.position_embeddings.weight
> embeddings.token_type_embeddings.weight
> embeddings.LayerNorm.weight
> embeddings.LayerNorm.bias
> encoder.layer.0.attention.self.query.weight
> encoder.layer.0.attention.self.query.bias
> encoder.layer.0.attention.self.key.weight
> encoder.layer.0.attention.self.key.bias
> encoder.layer.0.attention.self.value.weight
> encoder.layer.0.attention.self.value.bias
> encoder.layer.0.attention.output.dense.weight
> encoder.layer.0.attention.output.dense.bias
> encoder.layer.0.attention.output.LayerNorm.weight
> encoder.layer.0.attention.output.LayerNorm.bias
> encoder.layer.0.intermediate.dense.weight
> encoder.layer.0.intermediate.dense.bias
> encoder.layer.0.output.dense.weight
> encoder.layer.0.output.dense.bias
> encoder.layer.0.output.LayerNorm.weight
> encoder.layer.0.output.LayerNorm.bias
> encoder.layer.1.attention.self.query.weight
> encoder.layer.1.attention.self.query.bias
> encoder.layer.1.attention.self.key.weight
> encoder.layer.1.attention.self.key.bias
> encoder.layer.1.attention.self.value.weight
> encoder.layer.1.attention.self.value.bias
> encoder.layer.1.attention.output.dense.weight
> encoder.layer.1.attention.output.dense.bias
> encoder.layer.1.attention.output.LayerNorm.weight
> encoder.layer.1.attention.output.LayerNorm.bias
> encoder.layer.1.intermediate.dense.weight
> encoder.layer.1.intermediate.dense.bias
> encoder.layer.1.output.dense.weight
> encoder.layer.1.output.dense.bias
> encoder.layer.1.output.LayerNorm.weight
> encoder.layer.1.output.LayerNorm.bias
> encoder.layer.2.attention.self.query.weight
> encoder.layer.2.attention.self.query.bias
> encoder.layer.2.attention.self.key.weight
> encoder.layer.2.attention.self.key.bias
> encoder.layer.2.attention.self.value.weight
> encoder.layer.2.attention.self.value.bias
> encoder.layer.2.attention.output.dense.weight
> encoder.layer.2.attention.output.dense.bias
> encoder.layer.2.attention.output.LayerNorm.weight
> encoder.layer.2.attention.output.LayerNorm.bias
> encoder.layer.2.intermediate.dense.weight
> encoder.layer.2.intermediate.dense.bias
> encoder.layer.2.output.dense.weight
> encoder.layer.2.output.dense.bias
> encoder.layer.2.output.LayerNorm.weight
> encoder.layer.2.output.LayerNorm.bias
> encoder.layer.3.attention.self.query.weight
> encoder.layer.3.attention.self.query.bias
> encoder.layer.3.attention.self.key.weight
> encoder.layer.3.attention.self.key.bias
> encoder.layer.3.attention.self.value.weight
> encoder.layer.3.attention.self.value.bias
> encoder.layer.3.attention.output.dense.weight
> encoder.layer.3.attention.output.dense.bias
> encoder.layer.3.attention.output.LayerNorm.weight
> encoder.layer.3.attention.output.LayerNorm.bias
> encoder.layer.3.intermediate.dense.weight
> encoder.layer.3.intermediate.dense.bias
> encoder.layer.3.output.dense.weight
> encoder.layer.3.output.dense.bias
> encoder.layer.3.output.LayerNorm.weight
> encoder.layer.3.output.LayerNorm.bias
> encoder.layer.4.attention.self.query.weight
> encoder.layer.4.attention.self.query.bias
> encoder.layer.4.attention.self.key.weight
> encoder.layer.4.attention.self.key.bias
> encoder.layer.4.attention.self.value.weight
> encoder.layer.4.attention.self.value.bias
> encoder.layer.4.attention.output.dense.weight
> encoder.layer.4.attention.output.dense.bias
> encoder.layer.4.attention.output.LayerNorm.weight
> encoder.layer.4.attention.output.LayerNorm.bias
> encoder.layer.4.intermediate.dense.weight
> encoder.layer.4.intermediate.dense.bias
> encoder.layer.4.output.dense.weight
> encoder.layer.4.output.dense.bias
> encoder.layer.4.output.LayerNorm.weight
> encoder.layer.4.output.LayerNorm.bias
> encoder.layer.5.attention.self.query.weight
> encoder.layer.5.attention.self.query.bias
> encoder.layer.5.attention.self.key.weight
> encoder.layer.5.attention.self.key.bias
> encoder.layer.5.attention.self.value.weight
> encoder.layer.5.attention.self.value.bias
> encoder.layer.5.attention.output.dense.weight
> encoder.layer.5.attention.output.dense.bias
> encoder.layer.5.attention.output.LayerNorm.weight
> encoder.layer.5.attention.output.LayerNorm.bias
> encoder.layer.5.intermediate.dense.weight
> encoder.layer.5.intermediate.dense.bias
> encoder.layer.5.output.dense.weight
> encoder.layer.5.output.dense.bias
> encoder.layer.5.output.LayerNorm.weight
> encoder.layer.5.output.LayerNorm.bias
> encoder.layer.6.attention.self.query.weight
> encoder.layer.6.attention.self.query.bias
> encoder.layer.6.attention.self.key.weight
> encoder.layer.6.attention.self.key.bias
> encoder.layer.6.attention.self.value.weight
> encoder.layer.6.attention.self.value.bias
> encoder.layer.6.attention.output.dense.weight
> encoder.layer.6.attention.output.dense.bias
> encoder.layer.6.attention.output.LayerNorm.weight
> encoder.layer.6.attention.output.LayerNorm.bias
> encoder.layer.6.intermediate.dense.weight
> encoder.layer.6.intermediate.dense.bias
> encoder.layer.6.output.dense.weight
> encoder.layer.6.output.dense.bias
> encoder.layer.6.output.LayerNorm.weight
> encoder.layer.6.output.LayerNorm.bias
> encoder.layer.7.attention.self.query.weight
> encoder.layer.7.attention.self.query.bias
> encoder.layer.7.attention.self.key.weight
> encoder.layer.7.attention.self.key.bias
> encoder.layer.7.attention.self.value.weight
> encoder.layer.7.attention.self.value.bias
> encoder.layer.7.attention.output.dense.weight
> encoder.layer.7.attention.output.dense.bias
> encoder.layer.7.attention.output.LayerNorm.weight
> encoder.layer.7.attention.output.LayerNorm.bias
> encoder.layer.7.intermediate.dense.weight
> encoder.layer.7.intermediate.dense.bias
> encoder.layer.7.output.dense.weight
> encoder.layer.7.output.dense.bias
> encoder.layer.7.output.LayerNorm.weight
> encoder.layer.7.output.LayerNorm.bias
> encoder.layer.8.attention.self.query.weight
> encoder.layer.8.attention.self.query.bias
> encoder.layer.8.attention.self.key.weight
> encoder.layer.8.attention.self.key.bias
> encoder.layer.8.attention.self.value.weight
> encoder.layer.8.attention.self.value.bias
> encoder.layer.8.attention.output.dense.weight
> encoder.layer.8.attention.output.dense.bias
> encoder.layer.8.attention.output.LayerNorm.weight
> encoder.layer.8.attention.output.LayerNorm.bias
> encoder.layer.8.intermediate.dense.weight
> encoder.layer.8.intermediate.dense.bias
> encoder.layer.8.output.dense.weight
> encoder.layer.8.output.dense.bias
> encoder.layer.8.output.LayerNorm.weight
> encoder.layer.8.output.LayerNorm.bias
> encoder.layer.9.attention.self.query.weight
> encoder.layer.9.attention.self.query.bias
> encoder.layer.9.attention.self.key.weight
> encoder.layer.9.attention.self.key.bias
> encoder.layer.9.attention.self.value.weight
> encoder.layer.9.attention.self.value.bias
> encoder.layer.9.attention.output.dense.weight
> encoder.layer.9.attention.output.dense.bias
> encoder.layer.9.attention.output.LayerNorm.weight
> encoder.layer.9.attention.output.LayerNorm.bias
> encoder.layer.9.intermediate.dense.weight
> encoder.layer.9.intermediate.dense.bias
> encoder.layer.9.output.dense.weight
> encoder.layer.9.output.dense.bias
> encoder.layer.9.output.LayerNorm.weight
> encoder.layer.9.output.LayerNorm.bias
> encoder.layer.10.attention.self.query.weight
> encoder.layer.10.attention.self.query.bias
> encoder.layer.10.attention.self.key.weight
> encoder.layer.10.attention.self.key.bias
> encoder.layer.10.attention.self.value.weight
> encoder.layer.10.attention.self.value.bias
> encoder.layer.10.attention.output.dense.weight
> encoder.layer.10.attention.output.dense.bias
> encoder.layer.10.attention.output.LayerNorm.weight
> encoder.layer.10.attention.output.LayerNorm.bias
> encoder.layer.10.intermediate.dense.weight
> encoder.layer.10.intermediate.dense.bias
> encoder.layer.10.output.dense.weight
> encoder.layer.10.output.dense.bias
> encoder.layer.10.output.LayerNorm.weight
> encoder.layer.10.output.LayerNorm.bias
> encoder.layer.11.attention.self.query.weight
> encoder.layer.11.attention.self.query.bias
> encoder.layer.11.attention.self.key.weight
> encoder.layer.11.attention.self.key.bias
> encoder.layer.11.attention.self.value.weight
> encoder.layer.11.attention.self.value.bias
> encoder.layer.11.attention.output.dense.weight
> encoder.layer.11.attention.output.dense.bias
> encoder.layer.11.attention.output.LayerNorm.weight
> encoder.layer.11.attention.output.LayerNorm.bias
> encoder.layer.11.intermediate.dense.weight
> encoder.layer.11.intermediate.dense.bias
> encoder.layer.11.output.dense.weight
> encoder.layer.11.output.dense.bias
> encoder.layer.11.output.LayerNorm.weight
> encoder.layer.11.output.LayerNorm.bias
> pooler.dense.weight
> pooler.dense.bias

Instead using the following does give the expected output.

```python
bert = BertModel.from_pretrained('bert-base-uncased')
for name, param in bert.named_parameters():                
    if name.startswith('embeddings'):
        param.requires_grad = False
````


> Nope I haven't. I plan to do that today. @thomwolf do you think we're on the right track? Just need your 2 cents :)

How to check the values of the embedding matrix change or not?

> Correct me if I'm wrong, but setting `model.bert.embeddings.requires_grad = False` does not seem to propagate.
> 
> ```python
> bert = BertModel.from_pretrained('bert-base-uncased')
> bert.embeddings.requires_grad = False
> for name, param in bert.named_parameters():                
>     if param.requires_grad:
>         print(name)
> ```
> 
> Output:
> 
> > embeddings.word_embeddings.weight
> > embeddings.position_embeddings.weight
> > embeddings.token_type_embeddings.weight
> > embeddings.LayerNorm.weight
> > embeddings.LayerNorm.bias
> > encoder.layer.0.attention.self.query.weight
> > encoder.layer.0.attention.self.query.bias
> > encoder.layer.0.attention.self.key.weight
> > encoder.layer.0.attention.self.key.bias
> > encoder.layer.0.attention.self.value.weight
> > encoder.layer.0.attention.self.value.bias
> > encoder.layer.0.attention.output.dense.weight
> > encoder.layer.0.attention.output.dense.bias
> > encoder.layer.0.attention.output.LayerNorm.weight
> > encoder.layer.0.attention.output.LayerNorm.bias
> > encoder.layer.0.intermediate.dense.weight
> > encoder.layer.0.intermediate.dense.bias
> > encoder.layer.0.output.dense.weight
> > encoder.layer.0.output.dense.bias
> > encoder.layer.0.output.LayerNorm.weight
> > encoder.layer.0.output.LayerNorm.bias
> > encoder.layer.1.attention.self.query.weight
> > encoder.layer.1.attention.self.query.bias
> > encoder.layer.1.attention.self.key.weight
> > encoder.layer.1.attention.self.key.bias
> > encoder.layer.1.attention.self.value.weight
> > encoder.layer.1.attention.self.value.bias
> > encoder.layer.1.attention.output.dense.weight
> > encoder.layer.1.attention.output.dense.bias
> > encoder.layer.1.attention.output.LayerNorm.weight
> > encoder.layer.1.attention.output.LayerNorm.bias
> > encoder.layer.1.intermediate.dense.weight
> > encoder.layer.1.intermediate.dense.bias
> > encoder.layer.1.output.dense.weight
> > encoder.layer.1.output.dense.bias
> > encoder.layer.1.output.LayerNorm.weight
> > encoder.layer.1.output.LayerNorm.bias
> > encoder.layer.2.attention.self.query.weight
> > encoder.layer.2.attention.self.query.bias
> > encoder.layer.2.attention.self.key.weight
> > encoder.layer.2.attention.self.key.bias
> > encoder.layer.2.attention.self.value.weight
> > encoder.layer.2.attention.self.value.bias
> > encoder.layer.2.attention.output.dense.weight
> > encoder.layer.2.attention.output.dense.bias
> > encoder.layer.2.attention.output.LayerNorm.weight
> > encoder.layer.2.attention.output.LayerNorm.bias
> > encoder.layer.2.intermediate.dense.weight
> > encoder.layer.2.intermediate.dense.bias
> > encoder.layer.2.output.dense.weight
> > encoder.layer.2.output.dense.bias
> > encoder.layer.2.output.LayerNorm.weight
> > encoder.layer.2.output.LayerNorm.bias
> > encoder.layer.3.attention.self.query.weight
> > encoder.layer.3.attention.self.query.bias
> > encoder.layer.3.attention.self.key.weight
> > encoder.layer.3.attention.self.key.bias
> > encoder.layer.3.attention.self.value.weight
> > encoder.layer.3.attention.self.value.bias
> > encoder.layer.3.attention.output.dense.weight
> > encoder.layer.3.attention.output.dense.bias
> > encoder.layer.3.attention.output.LayerNorm.weight
> > encoder.layer.3.attention.output.LayerNorm.bias
> > encoder.layer.3.intermediate.dense.weight
> > encoder.layer.3.intermediate.dense.bias
> > encoder.layer.3.output.dense.weight
> > encoder.layer.3.output.dense.bias
> > encoder.layer.3.output.LayerNorm.weight
> > encoder.layer.3.output.LayerNorm.bias
> > encoder.layer.4.attention.self.query.weight
> > encoder.layer.4.attention.self.query.bias
> > encoder.layer.4.attention.self.key.weight
> > encoder.layer.4.attention.self.key.bias
> > encoder.layer.4.attention.self.value.weight
> > encoder.layer.4.attention.self.value.bias
> > encoder.layer.4.attention.output.dense.weight
> > encoder.layer.4.attention.output.dense.bias
> > encoder.layer.4.attention.output.LayerNorm.weight
> > encoder.layer.4.attention.output.LayerNorm.bias
> > encoder.layer.4.intermediate.dense.weight
> > encoder.layer.4.intermediate.dense.bias
> > encoder.layer.4.output.dense.weight
> > encoder.layer.4.output.dense.bias
> > encoder.layer.4.output.LayerNorm.weight
> > encoder.layer.4.output.LayerNorm.bias
> > encoder.layer.5.attention.self.query.weight
> > encoder.layer.5.attention.self.query.bias
> > encoder.layer.5.attention.self.key.weight
> > encoder.layer.5.attention.self.key.bias
> > encoder.layer.5.attention.self.value.weight
> > encoder.layer.5.attention.self.value.bias
> > encoder.layer.5.attention.output.dense.weight
> > encoder.layer.5.attention.output.dense.bias
> > encoder.layer.5.attention.output.LayerNorm.weight
> > encoder.layer.5.attention.output.LayerNorm.bias
> > encoder.layer.5.intermediate.dense.weight
> > encoder.layer.5.intermediate.dense.bias
> > encoder.layer.5.output.dense.weight
> > encoder.layer.5.output.dense.bias
> > encoder.layer.5.output.LayerNorm.weight
> > encoder.layer.5.output.LayerNorm.bias
> > encoder.layer.6.attention.self.query.weight
> > encoder.layer.6.attention.self.query.bias
> > encoder.layer.6.attention.self.key.weight
> > encoder.layer.6.attention.self.key.bias
> > encoder.layer.6.attention.self.value.weight
> > encoder.layer.6.attention.self.value.bias
> > encoder.layer.6.attention.output.dense.weight
> > encoder.layer.6.attention.output.dense.bias
> > encoder.layer.6.attention.output.LayerNorm.weight
> > encoder.layer.6.attention.output.LayerNorm.bias
> > encoder.layer.6.intermediate.dense.weight
> > encoder.layer.6.intermediate.dense.bias
> > encoder.layer.6.output.dense.weight
> > encoder.layer.6.output.dense.bias
> > encoder.layer.6.output.LayerNorm.weight
> > encoder.layer.6.output.LayerNorm.bias
> > encoder.layer.7.attention.self.query.weight
> > encoder.layer.7.attention.self.query.bias
> > encoder.layer.7.attention.self.key.weight
> > encoder.layer.7.attention.self.key.bias
> > encoder.layer.7.attention.self.value.weight
> > encoder.layer.7.attention.self.value.bias
> > encoder.layer.7.attention.output.dense.weight
> > encoder.layer.7.attention.output.dense.bias
> > encoder.layer.7.attention.output.LayerNorm.weight
> > encoder.layer.7.attention.output.LayerNorm.bias
> > encoder.layer.7.intermediate.dense.weight
> > encoder.layer.7.intermediate.dense.bias
> > encoder.layer.7.output.dense.weight
> > encoder.layer.7.output.dense.bias
> > encoder.layer.7.output.LayerNorm.weight
> > encoder.layer.7.output.LayerNorm.bias
> > encoder.layer.8.attention.self.query.weight
> > encoder.layer.8.attention.self.query.bias
> > encoder.layer.8.attention.self.key.weight
> > encoder.layer.8.attention.self.key.bias
> > encoder.layer.8.attention.self.value.weight
> > encoder.layer.8.attention.self.value.bias
> > encoder.layer.8.attention.output.dense.weight
> > encoder.layer.8.attention.output.dense.bias
> > encoder.layer.8.attention.output.LayerNorm.weight
> > encoder.layer.8.attention.output.LayerNorm.bias
> > encoder.layer.8.intermediate.dense.weight
> > encoder.layer.8.intermediate.dense.bias
> > encoder.layer.8.output.dense.weight
> > encoder.layer.8.output.dense.bias
> > encoder.layer.8.output.LayerNorm.weight
> > encoder.layer.8.output.LayerNorm.bias
> > encoder.layer.9.attention.self.query.weight
> > encoder.layer.9.attention.self.query.bias
> > encoder.layer.9.attention.self.key.weight
> > encoder.layer.9.attention.self.key.bias
> > encoder.layer.9.attention.self.value.weight
> > encoder.layer.9.attention.self.value.bias
> > encoder.layer.9.attention.output.dense.weight
> > encoder.layer.9.attention.output.dense.bias
> > encoder.layer.9.attention.output.LayerNorm.weight
> > encoder.layer.9.attention.output.LayerNorm.bias
> > encoder.layer.9.intermediate.dense.weight
> > encoder.layer.9.intermediate.dense.bias
> > encoder.layer.9.output.dense.weight
> > encoder.layer.9.output.dense.bias
> > encoder.layer.9.output.LayerNorm.weight
> > encoder.layer.9.output.LayerNorm.bias
> > encoder.layer.10.attention.self.query.weight
> > encoder.layer.10.attention.self.query.bias
> > encoder.layer.10.attention.self.key.weight
> > encoder.layer.10.attention.self.key.bias
> > encoder.layer.10.attention.self.value.weight
> > encoder.layer.10.attention.self.value.bias
> > encoder.layer.10.attention.output.dense.weight
> > encoder.layer.10.attention.output.dense.bias
> > encoder.layer.10.attention.output.LayerNorm.weight
> > encoder.layer.10.attention.output.LayerNorm.bias
> > encoder.layer.10.intermediate.dense.weight
> > encoder.layer.10.intermediate.dense.bias
> > encoder.layer.10.output.dense.weight
> > encoder.layer.10.output.dense.bias
> > encoder.layer.10.output.LayerNorm.weight
> > encoder.layer.10.output.LayerNorm.bias
> > encoder.layer.11.attention.self.query.weight
> > encoder.layer.11.attention.self.query.bias
> > encoder.layer.11.attention.self.key.weight
> > encoder.layer.11.attention.self.key.bias
> > encoder.layer.11.attention.self.value.weight
> > encoder.layer.11.attention.self.value.bias
> > encoder.layer.11.attention.output.dense.weight
> > encoder.layer.11.attention.output.dense.bias
> > encoder.layer.11.attention.output.LayerNorm.weight
> > encoder.layer.11.attention.output.LayerNorm.bias
> > encoder.layer.11.intermediate.dense.weight
> > encoder.layer.11.intermediate.dense.bias
> > encoder.layer.11.output.dense.weight
> > encoder.layer.11.output.dense.bias
> > encoder.layer.11.output.LayerNorm.weight
> > encoder.layer.11.output.LayerNorm.bias
> > pooler.dense.weight
> > pooler.dense.bias
> 
> Instead using the following does give the expected output.
> 
> ```python
> bert = BertModel.from_pretrained('bert-base-uncased')
> for name, param in bert.named_parameters():                
>     if name.startswith('embeddings'):
>         param.requires_grad = False
> ```

Hi 
How to tell the optimizer that freezing the embedding?

> > Nope I haven't. I plan to do that today. @thomwolf do you think we're on the right track? Just need your 2 cents :)
> 
> How to check the values of the embedding matrix change or not?

Hi! You can use the following code in order to check if any layer has been modified (it should work for any pytorch code if I am not wrong, not just BERT):

```python3
import copy
from transformers import BertModel

bert = BertModel.from_pretrained('bert-base-uncased')
layer = bert.embeddings
frozen_parameters = {}

# Copy tensors
for name, p in layer.named_parameters():
    frozen_parameters[name] = copy.deepcopy(p.data) # Freeze in order to be able to compare later

# Do stuff ...

# Check if the value of the tensors have been updated
for name, p in layer.named_parameters():
    updated = (frozen_parameters[name] != p.data).any().cpu().detach().numpy()

    print(f"Layer '{name}' has been updated? {'yes' if updated else 'no'}")
```

It is very similar to the code I use in order to check if a layer has been updated (remember that it won't be updated if `grad_fn` is `None`), but I have not tested this exactly code.

I hope it helps!