MaskedLM models doesn't output CLS and  weights not initialized in MaskedLM models
Author: ghost
Pull request: no

### System Info

- `transformers` version: 4.25.1
- Platform: Linux-5.15.0-56-generic-x86_64-with-glibc2.31
- Python version: 3.9.15
- Huggingface_hub version: 0.11.1
- PyTorch version (GPU?): 1.13.0+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help?

@ArthurZucker @younesbelkada 

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

When I try to use a Masked LM model some CLS weights are not initialized, thus when running a text through the model CLS and other special tokens as SEP will not be predicted correctly, producing misleading loss values when evaluating the models.

```python
from transformers import AutoModelForMaskedLM, AutoTokenizer

model_name = 'bert-base-cased'
model = AutoModelForMaskedLM.from_pretrained(model_name)
```
Gives:

> Some weights of the model checkpoint at bert-base-cased were not used when initializing BertForMaskedLM: ['cls.seq_relationship.bias', 'cls.seq_relationship.weight']
> - This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
> - This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
> 

So then when I run:
```python
tokenizer = AutoTokenizer.from_pretrained(model_name)

text = "Hello my name [MASK] Jhon, how can I [MASK] you?"
inputs = tokenizer(text, return_tensors='pt')
out_argmaxes = model(**inputs).logits[0].argmax(-1)

print(inputs['input_ids'])
print(out_argmaxes)
print(tokenizer.decode(out_argmaxes))
```
I got:
```
tensor([[ 101, 8667, 1139, 1271,  103,  147, 8613,  117, 1293, 1169,  146,  103, 1128,  136,  102]])
tensor([ 119,  119, 1139, 1271, 1110,  147, 8613,  117, 1293, 1169,  146, 1494, 1128,  136,  119])
'.. my name is Jhon, how can I help you?.'
```

This happens with different models, not just **bert-base-uncased**, the only case where this is not happening is with a custom **Roberta MaskedLM** model trained with a **custom tokenizer** where CLS and other special tokens are mapped as the first ids in the tokenizer, such as PAD: 0 /  <mask>: 1 / CLS: 2 ecc.

### Expected behavior

I would expect that the model would output logits with correctly predicted CLS and SEP tokens as the first and last tokens of the output, as they should be so that they can be evaluated producing a correct loss value.

```
'[CLS]Hello my name is Jhon, how can I help you?[SEP]'
```

Comments:
Hey! This is expected, you should be using the following : 
```python 
from transformers import pipeline
unmasker = pipeline('fill-mask', model='bert-base-uncased')
unmasker("Hello my name [MASK] Jhon, how can I [MASK] you?")
```

```python 
[[{'score': 0.8831212520599365,
   'sequence': '[CLS] Hello my name is Jhon, how can I [MASK] you? [SEP]',
   'token': 1110,
   'token_str': 'is'},
  {'score': 0.03171379491686821,
   'sequence': '[CLS] Hello my name, Jhon, how can I [MASK] you? [SEP]',
   'token': 117,
   'token_str': ','},
  {'score': 0.020678386092185974,
   'sequence': '[CLS] Hello my name? Jhon, how can I [MASK] you? [SEP]',
   'token': 136,
   'token_str': '?'},
  {'score': 0.013670953921973705,
   'sequence': '[CLS] Hello my name am Jhon, how can I [MASK] you? [SEP]',
   'token': 1821,
   'token_str': 'am'},
  {'score': 0.009090826846659184,
   'sequence': '[CLS] Hello my name was Jhon, how can I [MASK] you? [SEP]',
   'token': 1108,
   'token_str': 'was'}],
 [{'score': 0.9777076244354248,
   'sequence': '[CLS] Hello my name [MASK] Jhon, how can I help you? [SEP]',
   'token': 1494,
   'token_str': 'help'},
  {'score': 0.006017779931426048,
   'sequence': '[CLS] Hello my name [MASK] Jhon, how can I meet you? [SEP]',
   'token': 2283,
   'token_str': 'meet'},
  {'score': 0.00487362127751112,
   'sequence': '[CLS] Hello my name [MASK] Jhon, how can I reach you? [SEP]',
   'token': 2519,
   'token_str': 'reach'},
  {'score': 0.0022672810591757298,
   'sequence': '[CLS] Hello my name [MASK] Jhon, how can I be you? [SEP]',
   'token': 1129,
   'token_str': 'be'},
  {'score': 0.0018145894864574075,
   'sequence': '[CLS] Hello my name [MASK] Jhon, how can I call you? [SEP]',
   'token': 1840,
   'token_str': 'call'}]]
```
Where you have a list of length `2`, with the predictions and their different scores. 
This can be seen in the model cards' how to use. 

@ArthurZucker Thanks for your reply, but my goal here is not to extract the predicted tokens, I would like to extract the raw model logits to calculate loss value with respect to the correct labels, for example using `torch.nn.CrossEntropyLoss`, how can I achieve this without incurring in those wrong logits? 

If I use the bare model without a pipeline I get an unexpected loss value, so maybe I'm using this wrong:
```python
text_original = "Hello my name is Jhon, how can I help you?"
text = "Hello my name [MASK] Jhon, how can I [MASK] you?"
inputs = tokenizer(text, return_tensors='pt')
labels = tokenizer(text_original, return_tensors='pt')['input_ids']
out = model(**inputs, labels=labels)
out.loss
```
```
tensor(3.2894, grad_fn=<NllLossBackward0>)
```
Thanks 

Any suggestion on how to use the model to extract the correct loss value? Thanks

Hey, as mentioned in the [model's documentation](https://huggingface.co/docs/transformers/model_doc/bert#transformers.BertForMaskedLM.forward.example), you should be using the following : 
```python
>>> labels = tokenizer("The capital of France is Paris.", return_tensors="pt")["input_ids"]
# mask labels of non-[MASK] tokens
>>> labels = torch.where(inputs.input_ids == tokenizer.mask_token_id, labels, -100)

>>> outputs = model(**inputs, labels=labels)
>>> round(outputs.loss.item(), 2)
0.81
```
Tell me if this is not fixing the issue 😉 

Thanks a lot, I also used to not mask all the non-masked tokens in the labels during training, this would give me losses in the range of 0.10 - 0.20 while training my MaskedLM, I don't how this was affecting my models, anyway I updated the code accordingly to your suggestion, leaving only the masked tokens in the labels, now the loss is up to 1.6ish during training, so maybe this will produce better gradients to train the network? I also was able to evaluate my models correctly.