Error in BertForMaskedLM with add_tokens
Author: emillykkejensen
Pull request: no

## 🐛 Bug

<!-- Important information -->

Model I am using (Bert, XLNet....): Bert / BertForMaskedLM 

Language I am using the model on (English, Chinese....): bert-base-multilingual-cased

The problem arise when using:
* [X] the official example scripts: (give details)
* [X] my own modified scripts: (give details)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [X] my own task or dataset: (give details)

<!-- If you have a code sample, error messages, stack traces, please provide it here as well. -->

## Expected behavior

I am trying to fine tune the BERT language model using the pretrained bert-base-multilingual-cased tokenizer where I add 22 new tokens. I use the pretrained bert-base-multilingual-cased BertForMaskedLM model and run it all using the run_lm_finetuning train script.

Here is what I do:

```
from transformers import BertForMaskedLM, BertTokenizer
tokenizer = BertTokenizer.from_pretrained('bert-base-multilingual-cased')
tokenizer.add_tokens(my_new_tokens_list) #Consisting of 22 new word pieces

model = BertForMaskedLM.from_pretrained(model_name_or_path)
model.resize_token_embeddings(len(tokenizer))
model.to(torch.device(type='cuda'))

from transformers_fromGITHUB.examples import run_lm_finetuning
dataset = run_lm_finetuning.load_and_cache_examples(args, tokenizer, evaluate=False)

global_step, tr_loss = run_lm_finetuning.train(args, train_dataset, model, tokenizer)

```

When I run this last step: `global_step, tr_loss = run_lm_finetuning.train(args, train_dataset, model, tokenizer)` I get the following error:

```
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "mylib/BERTlm/transformers_fromGITHUB/examples/run_lm_finetuning.py", line 304, in train
    outputs = model(inputs, masked_lm_labels=labels) if args.mlm else model(inputs, labels=labels)                                                                               | 0/1 [00:00<?, ?it/s]
  File "mylib/.local/share/virtualenvs/BERTlm-giwpujkO/lib/python3.7/site-packages/torch/nn/modules/module.py", line 541, in __call__                     | 0/6359 [00:00<?, ?it/s]
    result = self.forward(*input, **kwargs)
  File "mylib/.local/share/virtualenvs/BERTlm-giwpujkO/lib/python3.7/site-packages/transformers/modeling_bert.py", line 887, in forward
    prediction_scores = self.cls(sequence_output)
  File "mylib/.local/share/virtualenvs/BERTlm-giwpujkO/lib/python3.7/site-packages/torch/nn/modules/module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File "mylib/.local/share/virtualenvs/BERTlm-giwpujkO/lib/python3.7/site-packages/transformers/modeling_bert.py", line 459, in forward
    prediction_scores = self.predictions(sequence_output)
  File "mylib/.local/share/virtualenvs/BERTlm-giwpujkO/lib/python3.7/site-packages/torch/nn/modules/module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File "mylib/.local/share/virtualenvs/BERTlm-giwpujkO/lib/python3.7/site-packages/transformers/modeling_bert.py", line 449, in forward
    hidden_states = self.decoder(hidden_states) + self.bias
RuntimeError: The size of tensor a (119569) must match the size of tensor b (119547) at non-singleton dimension 2
```
If I run it all without adding new tokens (skipping `tokenizer.add_tokens(my_new_tokens_list)` and `model.resize_token_embeddings(len(tokenizer))`) all works fine!


Having looked a bit around, the only place there are 119547 tokens, are in the tokenizer.vocab_size - all others are 119569:

```
>>> tokenizer.vocab_size
119547
>>> model.config.vocab_size
119569
>>> model.get_input_embeddings()
Embedding(119569, 768)
```
So can I somehow change the vocab_size in the tokenizer?

<!-- A clear and concise description of what you expected to happen. -->

## Environment

* OS: Ubuntu 18.04
* Python version: 3.7.5
* PyTorch version: 1.3.1
* PyTorch Transformers version (or branch): 2.2 / Git repo master comit f599623
* Using GPU ? Yes
* Distributed or parallel setup ? No
* Any other relevant information:

## Additional context

<!-- Add any other context about the problem here. -->


Comments:
I also have the same problem with AlbertForMaskedLM. 
I have tried all version of the git repo as well as pip installs.
Basically I add tokens
`from transformers import AlbertForMaskedLM, AlbertTokenizer
tokenizer = AlbertTokenizer.from_pretrained('albert-base-v2')
tokenizer.add_tokens(myvocab.get_unique_words_to_add()) #add news words from out corpus not in the spiece model. 37 words in total

model = AlbertForMaskedLM.from_pretrained(model_name_or_path)
model.resize_token_embeddings(len(tokenizer))
model.to(torch.device(type='cuda'))`
...
...
`
I receive the error

> RuntimeError: The size of tensor a (30037) must match the size of tensor b (30000) at non-singleton dimension 2

Environment
OS: Ubuntu 16.04
Python version: 3.6.9
PyTorch version: 1.3.1
PyTorch Transformers version (or branch): All Albert compatible branches and pip installs (2.3 as of last test)
Using GPU ? Yes
Distributed or parallel setup ? yes
Any other relevant information:

Hi, I've pushed a fix that was just merged in `master`. Could you please try and install from source:
```py
pip install git+https://github.com/huggingface/transformers
```
and tell me if you face the same error?

Forgot to update this issue - but yes, it now works.

https://github.com/huggingface/transformers/issues/2480#issuecomment-574548989