Model conversion from PyTorch to TF2 doesn't work properly for XLM-Roberta
Author: QixinLi
Pull request: no

## Environment info

- `transformers` version: 3.4.0
- Platform: MacOS
- Python version: 3.7
- PyTorch version (GPU?): 1.6.0
- Tensorflow version (GPU?): 2.3.1
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

@LysandreJik 

## Errors

```
Loading PyTorch weights from pytorch_model.bin
PyTorch checkpoint contains 470,547,238 parameters
Loaded 278,295,186 parameters in the TF 2.0 model.
Some weights of the PyTorch model were not used when initializing the TF 2.0 model TFXLMRobertaForMaskedLM: ['lm_head.decoder.bias', 'roberta.embeddings.position_ids', 'lm_head.decoder.weight']
- This IS expected if you are initializing TFXLMRobertaForMaskedLM from a PyTorch model trained on another task or with another architecture (e.g. initializing a TFBertForSequenceClassification model from a BertForPretraining model).
- This IS NOT expected if you are initializing TFXLMRobertaForMaskedLM from a PyTorch model that you expect to be exactly identical (e.g. initializing a TFBertForSequenceClassification model from a BertForSequenceClassification model).
All the weights of TFXLMRobertaForMaskedLM were initialized from the PyTorch model.
If your task is similar to the task the model of the ckeckpoint was trained on, you can already use TFXLMRobertaForMaskedLM for predictions without further training.
All model checkpoint weights were used when initializing XLMRobertaForMaskedLM.

All the weights of XLMRobertaForMaskedLM were initialized from the model checkpoint at None.
If your task is similar to the task the model of the checkpoint was trained on, you can already use XLMRobertaForMaskedLM for predictions without further training.
Traceback (most recent call last):
  File "convert_pytorch_checkpoint_to_tf2.py", line 432, in <module>
    use_cached_models=args.use_cached_models)
  File "convert_pytorch_checkpoint_to_tf2.py", line 297, in convert_pt_checkpoint_to_tf
    assert diff <= 2e-2, "Error, model absolute difference is >2e-2: {}".format(diff)
AssertionError: Error, model absolute difference is >2e-2: 1.0000114440917969
Max absolute difference between models outputs 1.0000114440917969

```
There's some weights didn't initialize correctly from pytorch model.

Comments:
Hello! Could you provide the commands you used to launch the script, and where you obtained the file from? Thanks.

> Hello! Could you provide the commands you used to launch the script, and where you obtained the file from? Thanks.

I found that I made a mistake when saving the pretrained model, after fix this bug, the script converts the pytorch model correctly. 
Thanks for your time!

@QixinLi , how did you converti this one? I am having similar type of problem while converting xlmroberta to tf. 

my code: https://colab.research.google.com/drive/17mOz39gXNHjeGN9tT4oJMBLvSWkgKlDN?usp=sharing

```
AttributeError                            Traceback (most recent call last)
<ipython-input-6-37732b1a66b9> in <module>()
      2                                                               '/content/drive/MyDrive/Colab_Notebooks/models/pytorch_xlmr/pytorch_model.bin',
      3                                                               '/content/drive/MyDrive/Colab_Notebooks/models/pytorch_xlmr/config.json',
----> 4                                                              '/content/drive/MyDrive/Colab_Notebooks/models/pytorch_xlmr')

2 frames
/usr/local/lib/python3.7/dist-packages/transformers/modeling_tf_pytorch_utils.py in load_pytorch_weights_in_tf2_model(tf_model, pt_state_dict, tf_inputs, allow_missing_keys)
    179                     continue
    180 
--> 181             raise AttributeError(f"{name} not found in PyTorch model")
    182 
    183         array = pt_state_dict[name].numpy()

AttributeError: lm_head.bias not found in PyTorch model
```