MarianMTModel.from_pretrained('Helsinki-NLP/opus-marian-en-de')
Author: sshleifer
Pull request: yes
Labels: name: seq2seq

name: translation

Adds support for `opus/marian-en-de` translation models:
- There are 900 models with this `MarianSentencePieceTokenizer`, `MarianMTModel` setup. - This PR only adds en-de to avoid massive S3 maintenance if names/other things change.
- There is no formal connection to the bart authors, but the bart code is well-tested and fast and I didn't want to rewrite it. You can still read everything in one file: modeling_bart.py.
- unittest failures are from isort.

The only differences from BART are:
- static (sinusoid) positional embeddings (`config.static_position_embeddings`)
- a new `final_logits_bias` (`config.add_bias_logits`)
- no `layernorm_embedding`  (`config.normalize_embedding`)

I could have also used T5 but that has more prefix token logic and relative position bias.
Solutions that split the changes across two files were much harder to read.


Why MarianSentencePieceTokenizer instead of MarianTokenizer?
- about 10% of the models (another 100) use BPE instead of SentencePiece. We will use that tokenizer (or integrate it with this one and rename) in a future PR.

### Future PRs
- load in many models
- support BPE tokenizer in some way.
- make lm_labels on the fly for training loop, like T5.
- `TranslationPipeline`

`TODO[Code]`:

- [x] investigate discrepancies in generation parameters: `length_penalty`, `decoder_start_token_id`, etc. These may explain small differences in generations (detailed in comment below). 
- [x] isort

`TODO[Docs]`:

- [x] Naming: `MarianMTModel`, `MarianSentencePieceTokenizer`
- [x] put en-de model in S3, update integration tests.
- [ ] docstring examples
- [ ] notebook to show?
- [ ] marian.rst
- [ ] README.md
- [ ] `AutoModelWithLMHead`


Comments:
I think that's awesome! Such little code changes to include that many models is amazing :-) 
Does the model with the pre-trained weights produce 1-to-1 the same results as the official Marian models in C++?

@patrickvonplaten the generations are very sensible, but not identical to marian, due to slightly different generation parameters: 

Equivalent (4/7):

C+: Ich bin ein kleiner Frosch.
HF: Ich bin ein kleiner Frosch.

C+: Tom bat seinen Lehrer um Rat.
HF: Tom bat seinen Lehrer um Rat.

C+: Tom bewunderte Marias Mut wirklich.
HF: Tom bewunderte Marias Mut wirklich.

C+: Umdrehen und die Augen schließen.
HF: Umdrehen und die Augen schließen.

Not Equivalent (3/7):
C+: Jetzt kann ich die 100 Wörter vergessen, die ich kenne.
HF: Jetzt kann ich die 100 Wörter **des Deutschen** vergessen, die ich kenne.

C+: **O**  (Input="O")
HF: 

C+: So würde ich das **machen**.
HF: So würde ich das **tun**.

I'm investigating.

> @patrickvonplaten the generations are very sensible, but not identical to marian, due to slightly different generation parameters:
> 
> Equivalent (4/7):
> 
> C+: Ich bin ein kleiner Frosch.
> HF: Ich bin ein kleiner Frosch.
> 
> C+: Tom bat seinen Lehrer um Rat.
> HF: Tom bat seinen Lehrer um Rat.
> 
> C+: Tom bewunderte Marias Mut wirklich.
> HF: Tom bewunderte Marias Mut wirklich.
> 
> C+: Umdrehen und die Augen schließen.
> HF: Umdrehen und die Augen schließen.
> 
> Not Equivalent (3/7):
> C+: Jetzt kann ich die 100 Wörter vergessen, die ich kenne.
> HF: Jetzt kann ich die 100 Wörter **des Deutschen** vergessen, die ich kenne.
> 
> C+: **O** (Input="O")
> HF:
> 
> C+: So würde ich das **machen**.
> HF: So würde ich das **tun**.
> 
> I'm investigating.

Sounds all very coherent :-) 

Merging. Will add docs in next PR, when I add tons of models.

@sshleifer, can you elaborate on "investigate discrepancies in generation parameters: length_penalty, decoder_start_token_id, etc. These may explain small differences in generations (detailed in comment below)"? I see https://github.com/huggingface/transformers/pull/3908#issuecomment-618473222 but was this resolved?

_TLDR Was not resolved, the investigation continues._
Our lib has many parameters to control generations, `length_penalty`, `repetition_penalty`, `bad_word_ids`, `decoder_start_token_id`.
They are documented [here](https://huggingface.co/transformers/model_doc/bart.html?#transformers.BartForConditionalGeneration.generate).
The marian lib has `normalize` and `word-penalty`, which are not identical, but similar. Part of the discrepancy may involve configuring our parameters to be closer to theirs.



I see, thanks!

Could anyone help with this issue: #5040 ?