Optional layers
Author: jplu
Pull request: yes

# What does this PR do?

This PR adds the possibility to have optional layers in the models thanks to the new input/output process. Here the pooling layer is created or not for the BERT/ALBERT/Longformer/MobileBERT/Roberta models. The keys to ignore when loading for these layers has been updated in same time.

Comments:
Thanks for working on this @jplu. I think we should take the opportunity to think about this issue: https://github.com/huggingface/transformers/issues/8793.

The problem with the `add_pooling_layer` option and how it's currently done in PyTorch models is that when doing a training initialized from a model checkpoints that *contains the pooling layer*, like `bert-base-cased`:

```py
model = BertForMaskedLM.from_pretrained("bert-base-cased")
# Fine-tune the model on an MLM task
```

we're losing the pooling layer doing so. It's not a big deal here as we're doing an MLM task, however, if we want to use that model for a downstream task:

```py
model.save_pretrained("bert-base-cased-finetuned-mlm")
classifier_model = BertForSequenceClassification.from_pretrained("bert-base-cased-finetuned-mlm")
```
we're now having a classifier model that has a randomly initialized pooling layer, whereas the weights that were stored in the `bert-base-cased` original checkpoint would have been better than a randomly initialized layer.

The issue is that right now, we have no way of specifying if we want to keep the pooling layer or not in such a setup. I would argue that controlling it from the configuration would really be useful here, rather than setting it to `add_pooling_layer=False` in architectures that do not need it.

cc @jplu @sgugger @patrickvonplaten 

Indeed, it starts to be more complicated than we thought at the beginning, but the case you are raising is a very good one!!

I think that controlling this from the config to have the same behavior would be more flexible, I +1 this proposal!

> Thanks for working on this @jplu. I think we should take the opportunity to think about this issue: #8793.
> 
> The problem with the `add_pooling_layer` option and how it's currently done in PyTorch models is that when doing a training initialized from a model checkpoints that _contains the pooling layer_, like `bert-base-cased`:
> 
> ```python
> model = BertForMaskedLM.from_pretrained("bert-base-cased")
> # Fine-tune the model on an MLM task
> ```
> 
> we're losing the pooling layer doing so. It's not a big deal here as we're doing an MLM task, however, if we want to use that model for a downstream task:
> 
> ```python
> model.save_pretrained("bert-base-cased-finetuned-mlm")
> classifier_model = BertForSequenceClassification.from_pretrained("bert-base-cased-finetuned-mlm")
> ```
> 
> we're now having a classifier model that has a randomly initialized pooling layer, whereas the weights that were stored in the `bert-base-cased` original checkpoint would have been better than a randomly initialized layer.
> 
> The issue is that right now, we have no way of specifying if we want to keep the pooling layer or not in such a setup. I would argue that controlling it from the configuration would really be useful here, rather than setting it to `add_pooling_layer=False` in architectures that do not need it.
> 
> cc @jplu @sgugger @patrickvonplaten

I remember that we were thinking about adding a config param for `add_pooling_layer` for PT: https://github.com/huggingface/transformers/pull/7272 and decided not to. I still think the cleaner solution is to **not** add a config param because it's a very weird use-case IMO. Why wouldn't the user just use a `BertForPreTraining` model for his use case? But I'm also fine with adding a config param instead. It's not a big deal to me...but in this case I'd definitely prefer to not add it to the general `PretrainedConfig`, but to each model's config.

Good point regarding the `BertForPreTraining`. I think this is a use-case (you want to keep a layer from another architecture) where you would want to build your own architectures for that, to have complete control over the layers.

I think we might be missing some documentation on how to do that, and on how creating an architecture that inherits from `PreTrainedModel` works, but this is a discussion for another time.

Ok to keep it this way.

LGTM for me!