Pretrained bare model have weights that are not beein used
Author: cajoek
Pull request: no

## Environment info
- `transformers` version: 4.12.5
- Platform: Linux-5.11.0-40-generic-x86_64-with-glibc2.29
- Python version: 3.8.10
- PyTorch version (GPU?): 1.9.1+cu111 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
Models:
- DistilBERT, BERT @LysandreJik

Documentation: @sgugger

## Information

The problem arises when using:
* [x] the official example scripts: (give details below)

## To reproduce

Hi I want to use the raw hidden-states from a pretrained bert-like model as input to another model.
I don't want a too big models so I tried the bare DistilBERT like this from the [docs](https://huggingface.co/transformers/master/model_doc/distilbert.html#transformers.DistilBertModel):
```python
from transformers import DistilBertTokenizer, DistilBertModel
import torch

tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-uncased')
model = DistilBertModel.from_pretrained('distilbert-base-uncased')
inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")
outputs = model(**inputs)
last_hidden_states = outputs.last_hidden_state
```
But I get the warning about weights that are not used:
```
Some weights of the model checkpoint at distilbert-base-uncased were not used when initializing DistilBertModel: ['vocab_transform.weight', 'vocab_transform.bias', 'vocab_layer_norm.bias', 'vocab_projector.bias', 'vocab_projector.weight', 'vocab_layer_norm.weight']
- This IS expected if you are initializing DistilBertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing DistilBertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
```
Should I see this warning? I didn't expect to.

I also tried with the bare BERT model, also by copying from the [docs](https://huggingface.co/transformers/master/model_doc/bert.html#transformers.BertModel). But this too shows a warning about weights not being used:
```
Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertModel: ['cls.predictions.decoder.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.transform.dense.weight', 'cls.seq_relationship.weight', 'cls.seq_relationship.bias', 'cls.predictions.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.LayerNorm.bias']
- This IS expected if you are initializing BertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing BertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
```
Should I really be seeing this? I assume the weights are matched with the correct model. If this is expected behavior maybe that can be noted in the docs?

Comments:
In both case you are using a checkpoint pretrained on masked language modeling. The warning tells you some weights of the checkpoint are not used, which is the case: you are using the bare model without the masked LM classification head, so you are not using those weights.

Alright so the default checkpoints contain the masked LM head? Is there a checkpoint without it? 

Thanks 

I'm not aware of any pretrained model without any head, so I think the answer is no.

Okay, thanks. I was just not expecting to see any warnings when following the examples.