prepare_seq2seq_batch makes labels/ decoder_input_ids made later.
Author: sshleifer
Pull request: yes

`src/` changes:
- when `tgt_texts` is supplied `prepare_seq_to_seq_batch` calls the tensor that used to be called `decoder_input_ids`, `labels`.
- This change helps metrics for models whose tokenizers do not add bos to the beginning of target sequences, like Marian and Pegasus, without affecting metrics for other models (bart). 
This branch was originally called "Fairseq batch equivalence", because it makes batches that look identical to fairseq's for mbart (and bart).
- tokenization testing file for bart.
- lots of cleanup and testing.

`examples/seq2seq` changes:
- `examples/seq2seq/finetune.py` (and eventually Seq2SeqTrainer) makes decoder_input_ids by shifting tokens right.
- this enables Marian finetuning and distillation, with a few extra changes. 
- add `--label_smoothing` option to seq2seq/distillation.py
- rename `Seq2SeqDataset` -> `LegacySeq2SeqDataset` and `TranslationDataset`-> `Seq2SeqDataset`. The new `Seq2SeqDataset` calls `prepare_seq2seq_batch`. The choice of which dataset to use is determined based on whether the tokenizer has a `prepare_seq2seq_batch` method.



**Problem:**

Previously on master, if the target language sequence was
"Șeful ONU declară că nu există soluții militare în Siria", and the tokenizer was Marian, lm_labels would become "ONU declară că nu există soluții militare în Siria", and the model would learn to skip the first token (or not generate bos).

Generations would then start very strangely, for example:
`", fostul şef al personalului prezidenţial din Brazilia, va participa la un proces"`

now: `"Fostul şef al personalului prezidenţial al Braziliei va fi judecat".`
(same thing is happening for pegasus #6711)

### Metrics
**mbart en-> ro**: no change
marian: master: 23 BLEU, this branch: 25
(en ro distillation/no teacher/3 dec layers)
distilbart-cnn-12-3: no change (within 0.01 ROUGE 2)
master + label smoothing: `{'rouge1': 43.2764,  'rouge2': 20.4969, 'rougeL': 29.9210}`
this branch + label smoothing: `{"rouge1": 43.1997, "rouge2": 20.4879, "rougeL": 30.1607}`


### TODO:
- check t5-base
- check pegasus

If you want to test whether this branch makes truncation go away, the easiest way is to pull the mirror branch with
```bash
git fetch
git checkout batch-parity-cleaner
```

cc @patil-suraj 

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6654?src=pr&el=h1) Report
> Merging [#6654](https://codecov.io/gh/huggingface/transformers/pull/6654?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/4bd7be9a4268221d2a0000c7e8033aaeb365c03b?el=desc) will **decrease** coverage by `0.04%`.
> The diff coverage is `100.00%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/6654/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/6654?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #6654      +/-   ##
==========================================
- Coverage   79.74%   79.70%   -0.05%     
==========================================
  Files         157      157              
  Lines       28479    28477       -2     
==========================================
- Hits        22712    22697      -15     
- Misses       5767     5780      +13     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/6654?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_bart.py](https://codecov.io/gh/huggingface/transformers/pull/6654/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19iYXJ0LnB5) | `95.57% <100.00%> (+0.01%)` | :arrow_up: |
| [src/transformers/modeling\_t5.py](https://codecov.io/gh/huggingface/transformers/pull/6654/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190NS5weQ==) | `83.83% <100.00%> (ø)` | |
| [src/transformers/tokenization\_bart.py](https://codecov.io/gh/huggingface/transformers/pull/6654/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fYmFydC5weQ==) | `100.00% <100.00%> (ø)` | |
| [src/transformers/tokenization\_marian.py](https://codecov.io/gh/huggingface/transformers/pull/6654/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fbWFyaWFuLnB5) | `99.15% <100.00%> (+32.48%)` | :arrow_up: |
| [src/transformers/tokenization\_mbart.py](https://codecov.io/gh/huggingface/transformers/pull/6654/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fbWJhcnQucHk=) | `96.82% <100.00%> (+1.51%)` | :arrow_up: |
| [src/transformers/tokenization\_pegasus.py](https://codecov.io/gh/huggingface/transformers/pull/6654/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fcGVnYXN1cy5weQ==) | `95.23% <100.00%> (+49.92%)` | :arrow_up: |
| [src/transformers/tokenization\_t5.py](https://codecov.io/gh/huggingface/transformers/pull/6654/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fdDUucHk=) | `95.32% <100.00%> (ø)` | |
| [src/transformers/tokenization\_roberta.py](https://codecov.io/gh/huggingface/transformers/pull/6654/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fcm9iZXJ0YS5weQ==) | `87.67% <0.00%> (-10.96%)` | :arrow_down: |
| [src/transformers/tokenization\_utils\_base.py](https://codecov.io/gh/huggingface/transformers/pull/6654/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fdXRpbHNfYmFzZS5weQ==) | `86.58% <0.00%> (-7.19%)` | :arrow_down: |
| [src/transformers/tokenization\_transfo\_xl.py](https://codecov.io/gh/huggingface/transformers/pull/6654/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fdHJhbnNmb194bC5weQ==) | `38.73% <0.00%> (-3.76%)` | :arrow_down: |
| ... and [9 more](https://codecov.io/gh/huggingface/transformers/pull/6654/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/6654?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6654?src=pr&el=footer). Last update [4bd7be9...08ddfd4](https://codecov.io/gh/huggingface/transformers/pull/6654?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Added common tokenizer tests @LysandreJik