adding encoder/decoder normalize_before to BART models
Author: nicola-decao
Pull request: yes

# What does this PR do?

This PR adds `encoder_normalize_before` and `decoder_normalize_before` parameters to BART models. These parameters control whether or not the `nn.LayerNorm` is applied before each block in the encoder/decoder (not used by all models -- often helps the model learn).

These parameters are already in use in `fairseq` see here for reference how it is used for [`TransformerEncoderLayerBase`](https://github.com/facebookresearch/fairseq/blob/f97cdf76d9cd20b16c3ff51f46382c2ed639bd17/fairseq/modules/transformer_layer.py#L164), [`TransformerDecoderLayerBase`](https://github.com/facebookresearch/fairseq/blob/f97cdf76d9cd20b16c3ff51f46382c2ed639bd17/fairseq/modules/transformer_layer.py#L385),  [`TransformerDecoderBase`](https://github.com/facebookresearch/fairseq/blob/f97cdf76d9cd20b16c3ff51f46382c2ed639bd17/fairseq/models/transformer/transformer_encoder.py#L101), and [`TransformerDecoderBase`](https://github.com/facebookresearch/fairseq/blob/f97cdf76d9cd20b16c3ff51f46382c2ed639bd17/fairseq/models/transformer/transformer_decoder.py#L126).

This PR simply implement the same logic that is: when `*_normalize_before` is set to `False` it behave exacly as it was before; when it is set to `True` it applys `nn.LayerNorm` before each block.

This PR is useful to load model trained with `fairseq` with that option active (e.g., [GENRE](https://github.com/facebookresearch/GENRE) -- see [here](https://github.com/facebookresearch/GENRE/blob/115095c3f526a7f12fc23f074f0da3ff7c63a2ad/scripts_mgenre/train.sh#L47)).

## Who can review?

Anyone in the community is free to review the PR once the tests have passed.

@patrickvonplaten
@patil-suraj
@ola13 


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hmmm not really in favor of this PR as it will greatly reduce readability IMO and does not correspond to the official BART implementation. 

@sgugger @LysandreJik what are your thoughts here? 

@patrickvonplaten I'm open to suggestions. Essentially I want to make possible to load some models that currently are not supported because they have been trained with `    --encoder-normalize-before`  and `--decoder-normalize-before` flags in `fairseq`. See [here](https://github.com/facebookresearch/GENRE/blob/1fa162308403808c7f226f792677faf566d5524f/scripts_mgenre/train.sh#L47)

@patrickvonplaten Additionally I want to release an [MBART model](https://dl.fbaipublicfiles.com/GENRE/mbart.cc100.tar.gz) trained on 125 languages that suffers from the exact same problem ie, it has been trained with `--encoder-normalize-before` and `--decoder-normalize-before` flags in `fairseq`.

You can use the [code in the Hub feature](https://huggingface.co/docs/transformers/custom_models) to share a model with custmoized code, but I agree with @patrickvonplaten here. Transformers is not a modular toolbox and this has nothing to do with the original BART model (activating those new config arguments would give bad predictions with a pretrained checkpoint).

@nicola-decao - the MBart modeling code is different from the Bart code in Transformers. I think it should work with the MBart code :-)

@patrickvonplaten Do you suggest incorporating `encoder/decoder normalize_before` directly into the MBart implementation?

I like that option so I can release one of the official FAIR [MBart model](https://dl.fbaipublicfiles.com/GENRE/mbart.cc100.tar.gz) trained on 100 languages.

> @nicola-decao - the MBart modeling code is different from the Bart code in Transformers. I think it should work with the MBart code :-)

@patrickvonplaten Ahhhh I see what you mean now! In MBart the layer normalization is already inverted! Let me try to import my models with that one and if works I'll cancel this PR.

@patrickvonplaten It worked! :D Sorry for the bother!