Add mbart-large-cc25, support translation finetuning
Author: sshleifer
Pull request: yes
Labels: name: seq2seq

name: translation

TODO:

- [x] fix config (max_length=100)
- [x] split test_modeling_mbart.py to new file
- [x] decoder_input_ids should start with decoder_start_token_id
- [x] AutoTokenizer
- [x] test finetuning. Requires adding `--src_lang` and `--tgt_lang` clargs. 
- [x] Documentation
- [ ] Model Card
- [x] fix mbart tokenizer tests
- [x] tokenizer docs

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5129?src=pr&el=h1) Report
> Merging [#5129](https://codecov.io/gh/huggingface/transformers/pull/5129?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/d2a93991158f15993eba9ab421d82766b892f948&el=desc) will **increase** coverage by `0.25%`.
> The diff coverage is `92.00%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5129/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5129?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #5129      +/-   ##
==========================================
+ Coverage   76.84%   77.09%   +0.25%     
==========================================
  Files         141      141              
  Lines       24685    24702      +17     
==========================================
+ Hits        18969    19044      +75     
+ Misses       5716     5658      -58     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/5129?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/tokenization\_bart.py](https://codecov.io/gh/huggingface/transformers/pull/5129/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fYmFydC5weQ==) | `95.45% <92.00%> (+1.57%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_openai.py](https://codecov.io/gh/huggingface/transformers/pull/5129/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9vcGVuYWkucHk=) | `20.78% <0.00%> (-74.20%)` | :arrow_down: |
| [src/transformers/tokenization\_xlm.py](https://codecov.io/gh/huggingface/transformers/pull/5129/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25feGxtLnB5) | `16.26% <0.00%> (-66.67%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_gpt2.py](https://codecov.io/gh/huggingface/transformers/pull/5129/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9ncHQyLnB5) | `65.69% <0.00%> (-29.45%)` | :arrow_down: |
| [src/transformers/tokenization\_openai.py](https://codecov.io/gh/huggingface/transformers/pull/5129/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fb3BlbmFpLnB5) | `71.21% <0.00%> (-12.88%)` | :arrow_down: |
| [src/transformers/tokenization\_transfo\_xl.py](https://codecov.io/gh/huggingface/transformers/pull/5129/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fdHJhbnNmb194bC5weQ==) | `33.56% <0.00%> (-8.93%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5129/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `82.99% <0.00%> (-6.13%)` | :arrow_down: |
| [src/transformers/tokenization\_auto.py](https://codecov.io/gh/huggingface/transformers/pull/5129/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fYXV0by5weQ==) | `95.55% <0.00%> (-2.23%)` | :arrow_down: |
| [src/transformers/modeling\_openai.py](https://codecov.io/gh/huggingface/transformers/pull/5129/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19vcGVuYWkucHk=) | `79.72% <0.00%> (-1.38%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_pytorch\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5129/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9weXRvcmNoX3V0aWxzLnB5) | `88.05% <0.00%> (-1.26%)` | :arrow_down: |
| ... and [4 more](https://codecov.io/gh/huggingface/transformers/pull/5129/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5129?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5129?src=pr&el=footer). Last update [d2a9399...3685a81](https://codecov.io/gh/huggingface/transformers/pull/5129?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


> Could we test the MBart tokenizer? Or do you think it's too similar to the XLM-R tokenizer to deserve its own test suite?

<3 that idea. Will do in this PR. Thanks for reminding me.

This does not need review at the moment. I need to fix the tests first.

Oh, and you have the `build_doc` failing (warnings make the build fail) with the following error:

```
tokenization_bart.py:docstring of transformers.MBartTokenizer.build_inputs_with_special_tokens:4:Unexpected indentation.
```

Hi Sam, 
Thanks for your great work to add MBART-cc25.
I've been following issues and threads regarding MBART generation problems for a while now.
Are those problems fixed in this PR or is the model still generating English?

The model is still generating english, but I'm not sure whether that's a bug. I can finetune it to achieve good scores on WMT en-ro, and nobody has replied to https://github.com/pytorch/fairseq/issues/2258, so I moved on.

Sad :(  I was really looking forward to trying it on language pairs that Marian does not support (e.g. English->Chinese, English->Arabic). Is there any way to get MBART to work without any fine-tuning?
The problem I'm facing is it always generates BOS tokens at the first decoding step even though the tgt_lang id is provided as the starting token. I tried forcing zero probability on BOS token at each step but then it generates gibberish text...

I don't know how the preprocessing is done in the original setup, but I found a potential problems in generating decoder_input_ids (i.e target sentence) for training. According to the documentation, it's supposed to have the format:
```
language code - tokens - eos
```
(which is supposedly different from the format of the input_ids), but the tokenizer currently generates for both source (input_ids) and target (decoder_input_ids) the current format:
```
tokens - eos - language code
```
Maybe this is why you are unable to generate? Could sb verify this?

This was a bug, but is fixed on master (by this PR) if you use `prepare_translation_batch`.

For example
```python
from transformers import MBartTokenizer
model_name = 'facebook/mbart-large-cc25'
tok = MBartTokenizer.from_pretrained(model_name)
src_text = ['I think I fixed the tokenizer']
tgt_text = ['Cred că am rezolvat tokenizatorul']
batch = tok.prepare_translation_batch(src_text, tgt_texts=tgt_text)
batch.input_ids # (*tokens, eos, lang_code)
=> tensor([[    87,   5351,     87, 188347,     70,     47,   1098,  52825,      2, 250004]])
batch.decoder_input_ids # (lang_code, *tokens, eos)
=> tensor([[250020,  68523,   1362,    444, 102432,     18,     47,   1098, 164077, 202,      2]])
```

More info: mbart knows how to switch between src and tgt modes using the `set_lang` method: https://github.com/huggingface/transformers/blob/d6eab53058015483e9cbcbfee4bf900c3a8ab772/src/transformers/tokenization_bart.py#L186