Add BART DLM PyTorch pretraining example
Author: BramVanroy
Pull request: yes

Implements a pretraining example for BART (denoising language model). Big focus on getting the data denoising as close to the original fairseq as possible but instead of on the dataset level on the dataloader level.

Heavily inspired by the fairseq implementation and the FLAX implementation. (See `HF (Flax), fairseq, and current implementation`.) Looking for some feedback. Please see `Questions/Uncertainties`.

# Some notes 

## Default values
The defaults are set to the [given BART args](https://github.com/facebookresearch/fairseq/issues/1899#issuecomment-1069429320). This differs from the Flax defaults in one respect, namely `poisson_lambda`, which is now set to `3.5` instead of `3.0`.


## HF (Flax), fairseq, and current implementation

There are some differences in implementation between fairseq, the HF FLAX example, and this PyTorch implementation.

- `argwhere` in the Flax example
[in this position](https://github.com/huggingface/transformers/blob/65fb71bc762c46bb067306c1fd083b1cba87a095/examples/flax/language-modeling/run_bart_dlm_flax.py#L319)
is not the same as what is happening in fairseq. [In fairseq](https://github.com/facebookresearch/fairseq/blob/a6a63279422f846a3c2f6c45b9c96d6951cc4b82/fairseq/data/denoising_dataset.py#L230)
we check explicitly that the previous token was not a "full stop" (padding token) but in HF we just check whether the
current token is a full stop. In the current example I also explicitly check that the next token is not a full stop,
in case of padding. (However, in practice that should be a non-issue since all batches/samples should have the
same sequence length and there should not be any padding.)
- I found that the result of sentence permutation was not consistent in terms of where the separating pad token ended
up ([bug report](https://github.com/facebookresearch/fairseq/issues/4695)), so I have reimplemented that method so
that sentences in a sequence are still separated by a padding token, even after permutation.
- In HF FLAX, the token_mask is restricted to [non-special and non-padding tokens](https://github.com/huggingface/transformers/blob/65fb71bc762c46bb067306c1fd083b1cba87a095/examples/flax/language-modeling/run_bart_dlm_flax.py#L361).
In Fairseq, by default, only the first and last tokens are excluded and [all others](https://github.com/facebookresearch/fairseq/blob/1bba712622b8ae4efb3eb793a8a40da386fe11d0/fairseq/data/denoising_dataset.py#L241)
are prone to masking. The HF implementation seems sensible so I follow that. `get_special_tokens_mask` includes the 
padding token, though, so no need to add that separately.
- The Flax example does not include methods to add more noise. I have ported those as well.
- However, I did not adapt `add_insertion_noise` to work well with padded sequences. So the inserted noise may occur
ANYWHERE. It is unclear whether this is intended behavior.

Alternatively, we could implement all this processing on the dataset level and use `Dataset.map`. This has some
advantages:

- more true to fairseq implementation (sample level rather than batch level);
- cached.

... and disadvantages:

- potentially slower (not batched), although we can integrate a batched approach. But as discussed above, this will be
less true to the original fairseq implementation in `add_insertion_noise`
- every sample is always processed the same. So in small datasets which are seen multiple times by the model, the 
same sample will always be processed the same. In a dataloader, that will not be the case because the processing
occurs on every iteration rather than once before training.

## Questions/Uncertainties
- Do the padding tokens still serve a purpose after permutation? (Teaching the model to learn to detect sentence boundaries?) They _can_ get masked and noised.
- It seems that `add_insertion_noise` can insert noise _anywhere_ (also in fairseq), which means that it will also overwrite special
tokens and that sequence don't necessarily end with a EOS token. Is that a problem?
- I have now added auxiliary scripts for config/tokenizer creation when pre-training. Should I remove those? In the FLAX example, these steps are [described inline](https://github.com/huggingface/transformers/tree/main/examples/flax/language-modeling#bart-denoising-language-modeling) but without a given script. So we could also just do that.
- I have explicitly added fingerprints (hashed) because in the past I've come to encounter issues when using spaCy and Dataset.map (every time you load a spaCy model, it has a different hash so the processing will happen every time). I don't see a better way but feel free to share ideas. Maybe someone of the `datasets` team can chime in, too.


# Before submitting
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? https://github.com/huggingface/transformers/issues/5096#issuecomment-1237227809
- [x] Did you make sure to update the documentation with your changes? 



# Who can review?

- bart: @patrickvonplaten @patil-suraj
- maintained examples (not research project or legacy): @patil-suraj
- flax implementation authors: @sanchit-gandhi @duongna21



Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_18904). All of your documentation changes will be reflected on that endpoint.

Hey @BramVanroy! Thanks for making a start on this PR. In general, we aim to mirror the original repo's functionality as closely as possible. In this case, porting from fairseq is the way to go! So great to see your comments regarding consistency with fariseq, and yes to all of them! If indeed these changes are required, we'll need to update the Flax example accordingly.

We can batch samples with datasets.map by passing the `num_workers` arg. To pre-process samples on a specified number of CPU workers concurrently:
```python
dataset = dataset.map(map_fn, num_workers=data_args.preprocessing_num_workers)
```
This I think is the way to go for processing the dataset being the closest to fariseq. 

Adding auxiliary scripts for config/tokenizer creation is a great idea - all for it! Makes it far easier to reproduce and run the example :-)