[Marian Fixes] prevent predicting pad_token_id before softmax, support language codes, name multilingual models
Author: sshleifer
Pull request: yes

(this should probably be 3 PRs, sorry).

TLDR: This gets us to equal or better BLEU for 99+% of the converted models.

### Issue 1: non-breaking Generation Issue
`pad_token_id` is predicted with score 0, but we never want to generate it.
Previously, we set its score to inf  after softmax, so probabilities summed to something much less than 1. At the end of long generations, the probabilities got infinitesimally small, and BLEU suffered.

Solution: set `logits[:, pad_token_id] = -float('inf')` before calling `log_softmax`
Also, renames `prepare_scores_for_generation` -> `prepare_logits_for_generation` because it is now called on logits.

- Bart outputs are unchanged, other models unaffected.
- This could also be done for other filters, like bad_words_ids, but results may change slightly.
- This works fine with fp16
- Improves BLEU scores for 50% of Marian models, often by a huge amount for those with long examples in test set. No change for others models.

### Issue 2 Multilingual Models require language codes.

`>>pt<< Don't spend so much time watching TV.`
The language code at the beginning, which tells the model to generate in portuguese, is not known to sentencepiece, so `MarianTokenizer` must manually extract it and add it as it's own piece.
We do this using `re.match` inside `_tokenize`.


### Issue 3: Opus' Multilingual models have very long names
We remap them to much shorter names, like `Helsinki-NLP/opus-mt-en-ROMANCE` in `convert_marian_to_pytorch.py`, and document the naming scheme in `docs/`.




### Working snippet:
```
from transformers import MarianMTModel, MarianTokenizer
text = ['>>fr<< this is a sentence in english that we want to translate to french', 
        '>>pt<< This should go to portuguese',
        '>>es<< And this to Spanish'
       ]
model_name = 'Helsinki-NLP/opus-mt-en-ROMANCE'
tokenizer = MarianTokenizer.from_pretrained(model_name)
model = MarianMTModel.from_pretrained(model_name)
translated = model.generate(**tokenizer.prepare_translation_batch(text))
[tokenizer.decode(t, skip_special_tokens=True) for t in translated]

=> ["c'est une phrase en anglais que nous voulons traduire en français",
 'Isto deve ir para o português.',
 'Y esto al español']
```

Comments:
The slow bart summarization tests all pass with this? 

yes!

LGTM

Correct, @LysandreJik not a breaking change.


# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4290?src=pr&el=h1) Report
> Merging [#4290](https://codecov.io/gh/huggingface/transformers/pull/4290?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/7d7fe4997f83d6d858849a659302b9fdc32c3337&el=desc) will **increase** coverage by `0.06%`.
> The diff coverage is `95.65%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4290/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4290?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #4290      +/-   ##
==========================================
+ Coverage   78.16%   78.22%   +0.06%     
==========================================
  Files         120      120              
  Lines       20005    20013       +8     
==========================================
+ Hits        15636    15655      +19     
+ Misses       4369     4358      -11     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4290?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_marian.py](https://codecov.io/gh/huggingface/transformers/pull/4290/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19tYXJpYW4ucHk=) | `88.88% <75.00%> (+26.38%)` | :arrow_up: |
| [src/transformers/modeling\_bart.py](https://codecov.io/gh/huggingface/transformers/pull/4290/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19iYXJ0LnB5) | `96.79% <100.00%> (+0.56%)` | :arrow_up: |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4290/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `91.08% <100.00%> (+0.12%)` | :arrow_up: |
| [src/transformers/tokenization\_marian.py](https://codecov.io/gh/huggingface/transformers/pull/4290/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fbWFyaWFuLnB5) | `87.14% <100.00%> (+6.19%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4290/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `92.77% <0.00%> (+0.16%)` | :arrow_up: |
| [src/transformers/activations.py](https://codecov.io/gh/huggingface/transformers/pull/4290/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9hY3RpdmF0aW9ucy5weQ==) | `90.00% <0.00%> (+5.00%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4290?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4290?src=pr&el=footer). Last update [7d7fe49...54d704c](https://codecov.io/gh/huggingface/transformers/pull/4290?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Merging in advance of release tomorrow. Feel free to leave comments and I will fix them!