ImportError: cannot import name 'BertLayerNorm' when upgrading to latest transformers
Author: gsrivas4
Pull request: no
Labels: name: Migration

# 📚 Migration

## Getting error when upgrading from pytorch-transformers to transformers

<!-- Important information -->

Model I am using (Bert, XLNet ...): Bert

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: Yes
* [ ] my own modified scripts: Yes

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: No
* [ ] my own task or dataset: No

## Details

I am using Oscar repo (https://github.com/microsoft/Oscar), which uses an older version of Huggingface pytorch-transformers (https://github.com/huggingface/transformers/tree/067923d3267325f525f4e46f357360c191ba562e). I am trying to upgrade the repo to use latest version of transformers (https://github.com/huggingface/transformers). However, I am getting following error when I try to use the latest version of transformers:
```
Traceback (most recent call last):
  File "oscar/run_captioning.py", line 22, in <module>
    from oscar.modeling.modeling_bert import BertForImageCaptioning
  File "/home/default/ephemeral_drive/work/image_captioning/Oscar_edited/oscar/modeling/modeling_bert.py", line 15, in <module>
    from transformers.models.bert.modeling_bert import BertLayerNorm
ImportError: cannot import name 'BertLayerNorm'
```
I have tried running an example script given with the latest transformers repo- https://github.com/huggingface/transformers/blob/master/examples/research_projects/movement-pruning/emmental/modeling_bert_masked.py, which uses `BertLayerNorm`, but that gives following error:
```
$ python emmental/modeling_bert_masked.py
Traceback (most recent call last):
  File "emmental/modeling_bert_masked.py", line 29, in <module>
    from emmental import MaskedBertConfig
  File "/home/default/ephemeral_drive/work/image_captioning/Oscar_edited/transformers/examples/research_projects/movement-pruning/emmental/__init__.py", line 3, in <module>
    from .modeling_bert_masked import (
  File "/home/default/ephemeral_drive/work/image_captioning/Oscar_edited/transformers/examples/research_projects/movement-pruning/emmental/modeling_bert_masked.py", line 33, in <module>
    from transformers.models.bert.modeling_bert import ACT2FN, BertLayerNorm, load_tf_weights_in_bert
ImportError: cannot import name 'BertLayerNorm'
```
I tried looking for definition of `BertLayerNorm` in the current version of transformers, but it is not present there. The definition is present in the older version of transformer, here - https://github.com/huggingface/transformers/blob/067923d3267325f525f4e46f357360c191ba562e/pytorch_transformers/modeling_bert.py#L223-L240

How can I import `BertLayerNorm` in my project using the latest transformer?

## Environment info
<!-- You can run the command `python transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
 
- `transformers` version: https://github.com/huggingface/transformers
- Platform: x86_64 GNU/Linux
- Python version: 3.6.8
- PyTorch version (GPU?): 1.7.0+cu101 (GPU)
- Tensorflow version (GPU?): 2.3.0 (GPU)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: No

<!-- IMPORTANT: which version of the former library do you use? -->
* `pytorch-transformers` or `pytorch-pretrained-bert` version (or branch): https://github.com/huggingface/transformers/tree/067923d3267325f525f4e46f357360c191ba562e


## Checklist

- [ Yes] I have read the migration guide in the readme.
 ([pytorch-transformers](https://github.com/huggingface/transformers#migrating-from-pytorch-transformers-to-transformers);
  [pytorch-pretrained-bert](https://github.com/huggingface/transformers#migrating-from-pytorch-pretrained-bert-to-transformers))
- [ yes] I checked if a related official extension example runs on my machine.


Comments:
There is no `BertLayerNorm` anymore since all it was adding has been ported to main PyTorch. The BERT model is now jusing `torch.nn.LayerNorm`.

So to make your code working, instead of trying to import it from transformers, just define it as:
```
BertLayerNorm = torch.nn.LayerNorm
```

@sgugger your suggestion resolved the issue. Thanks!

Closing then :-)