Size mismatch when loading pretrained model
Author: malmaud
Pull request: no

I'm seeing this:

```
In [1]: import pytorch_transformers                                                                                
In [2]: m=pytorch_transformers.AutoModel.from_pretrained('roberta-base')                                           
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-2-7a33f5ecb345> in <module>
----> 1 m=pytorch_transformers.AutoModel.from_pretrained('roberta-base')
/opt/anaconda3/lib/python3.7/site-packages/pytorch_transformers/modeling_auto.py in from_pretrained(cls, pretrained
_model_name_or_path, *model_args, **kwargs)
    240             return DistilBertModel.from_pretrained(pretrained_model_name_or_path, *model_args, **kwargs)
    241         elif 'roberta' in pretrained_model_name_or_path:
--> 242             return RobertaModel.from_pretrained(pretrained_model_name_or_path, *model_args, **kwargs)
    243         elif 'bert' in pretrained_model_name_or_path:
    244             return BertModel.from_pretrained(pretrained_model_name_or_path, *model_args, **kwargs)
/opt/anaconda3/lib/python3.7/site-packages/pytorch_transformers/modeling_utils.py in from_pretrained(cls, pretraine
d_model_name_or_path, *model_args, **kwargs)
    592         if len(error_msgs) > 0:
    593             raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
--> 594                                model.__class__.__name__, "\n\t".join(error_msgs)))
    595 
    596         if hasattr(model, 'tie_weights'):
RuntimeError: Error(s) in loading state_dict for RobertaModel:
        size mismatch for roberta.embeddings.position_embeddings.weight: copying a param with shape torch.Size([514
, 768]) from checkpoint, the shape in current model is torch.Size([512, 768]).
```

Comments:
I'm having the same problem with RoBERTa, it didn't happen until a few hours.

Hi, thanks for pointing it out, I made a mistake with a config object hosted on our S3. It should be fixed now.

Running the following snippet:
`# Load the model in fairseq`
`from fairseq.models.roberta import RobertaModel`
`roberta = RobertaModel.from_pretrained('./roberta.large', checkpoint_file='model.pt')`
`roberta.eval()  # disable dropout (or leave in train mode to finetune)`

I got the following error:
`RuntimeError: Error(s) in loading state_dict for RobertaModel:
	Missing key(s) in state_dict: "decoder.sentence_encoder.layers.0.self_attn.k_proj.weight", "decoder.sentence_encoder.layers.0.self_attn.k_proj.bias", "decoder.sentence_encoder.layers.0.self_attn.v_proj.weight", "decoder.sentence_encoder.layers.0.self_attn.v_proj.bias", "decoder.sentence_encoder.layers.0.self_attn.q_proj.weight", "decoder.sentence_encoder.layers.0.self_attn.q_proj.bias", "decoder.sentence_encoder.layers.1.self_attn.k_proj.weight", "decoder.sentence_encoder.layers.1.self_attn.k_proj.bias", "decoder.sentence_encoder.layers.1.self_attn.v_proj.weight", "decoder.sentence_encoder.layers.1.self_attn.v_proj.bias", "decoder.sentence_encoder.layers.1.self_attn.q_proj.weight", "decoder.sentence_encoder.layers.1.self_attn.q_proj.bias", "decoder.sentence_encoder.layers.2.self_attn.k_proj.weight", "decoder.sentence_encoder.layers.2.self_attn.k_proj.bias", "decoder.sentence_encoder.layers.2.self_attn.v_proj.weight", "decoder.sentence_encoder.layers.2.self_attn.v_proj.bias", "decoder.sentence_encoder.layers.2.self_attn.q_proj.weight", "decoder.sentence_encoder.layers.2.self_attn.q_proj.bias", "decoder.sentence_encoder.layers.3.self_attn.k_proj.weight", "decoder.sentence_encoder.layers.3.self_attn.k_proj.bias", "decoder.sentence_encoder.layers.3.self_attn.v_proj.weight", "decoder.sentence_encoder.layers.3.self_attn.v_proj.bias", "decoder.sentence_encoder.layers.3.self_attn.q_proj.weight", "decoder.sentence_encoder.layers.3.self_attn.q_proj.bias", "decoder.sentence_encoder....
	Unexpected key(s) in state_dict: "decoder.sentence_encoder.layers.0.self_attn.in_proj_weight", "decoder.sentence_encoder.layers.0.self_attn.in_proj_bias", "decoder.sentence_encoder.layers.1.self_attn.in_proj_weight", "decoder.sentence_encoder.layers.1.self_attn.in_proj_bias", "decoder.sentence_encoder.layers.2.self_attn.in_proj_weight", "decoder.sentence_encoder.layers.2.self_attn.in_proj_bias", "decoder.sentence_encoder.layers.3.self_attn.in_proj_weight", "decoder.sentence_encoder.layers.3.self_attn.in_proj_bias", "decoder.sentence_encoder.layers.4.self_attn.in_proj_weight", "decoder.sentence_encoder.layers.4.self_attn.in_proj_bias", "decoder.sentence_encoder.layers.5.self_attn.in_proj_weight", "decoder.sentence_encoder.layers.5.self_attn.in_proj_bias", "decoder.sentence_encoder.layers.6.self_attn.in_proj_weight", "decoder.sentence_encoder.layers.6.self_attn.in_proj_bias", "decoder.sentence_encoder.layers.7.self_attn.in_proj_weight", "decoder.sentence_encoder.layers.7.self_attn.in_proj_bias", "decoder.sentence_encoder.layers.8.self_attn.in_proj_weight", "decoder.sentence_encoder.layers.8.self_attn.in_proj_bias", "decoder.sentence_encoder.layers.9.self_attn.in_proj_weight", "decoder.sentence_encoder.layers.9.self_attn.in_proj_bias", "decoder.sentence_encoder.layers.10.self_attn.in_proj_weight", "decoder.sentence_encoder.layers.10.self_attn.in_proj_bias", "decoder.sentence_encoder.layers.11.self_attn.in_proj_weight", "decoder.sentence_encoder.layers.11.self_attn.in_proj_bi...`

Is it related to the above error? How can we fix it?

I am seeing the same error as @pbabvey is seeing. I suspect the S3 object is out-of-sync with the code?

I am running into the same issue as @pbabvey while loading the model. Is there any fix available for this?

Hi, if you're running the following code:

```py
# Load the model in fairseq
from fairseq.models.roberta import RobertaModel
roberta = RobertaModel.from_pretrained('./roberta.large', checkpoint_file='model.pt')
roberta.eval() # disable dropout (or leave in train mode to finetune)
```
Then you are not using our library, but [fairseq](https://github.com/pytorch/fairseq).

To use our library you would do it as follows:

```py
from transformers import RobertaModel

model = RobertaModel.from_pretrained("roberta-large")
```

There is one argument called `ignore_mismatched_sizes` in `from_pretrained` method. ISSUE: [#13187](https://github.com/huggingface/transformers/issues/13187)