High Quality EN-DE/EN-FR Translators
Author: sshleifer
Pull request: no
Labels: name: Help wanted

name: New model

name: translation

Download instructions from torchub/fairseq: [here](https://github.com/pytorch/fairseq/blob/f03392d11faf1588cb571d19835d6a61ab0d9ca6/examples/wmt19/README.md#L1)
the BART conversion script should be reusable.


## Open source status

* [ x] the model implementation is available: (give details)
* [ x] the model weights are available: (give details)
* [ x] who are the authors: (mention them, if possible by @gh-username)
Sergey Edunov, @myleott  Michael Auli, David Grangier


Paper: https://arxiv.org/pdf/1808.09381.pdf


### Spec

Desired API:

```python
mname = 'facebook/wmt-en-de'



model = FairseqTranslator.from_pretrained(mname)
tokenizer = FairseqBPETokenizer.from_pretrained(mname)  # AutoTokenizer should also work
batch = tokenizer.prepare_seq2seq_batch(['Maschinelles Lernen ist großartig!'])
translated = model.generate(**batch) # determine 
assert tokenizer.batch_decode(translated)[0] == 'Machine Learning is great'
```

- add .rst docs, (see adding a new model instructions, but don't follow them too religiously if something seems suboptimal).
- check timing, memory vs fairseq.
- if lots of modeling code is added, common tests should pass.



### Steps

1. Get tokenizer equivalence (The fairseq object should have an encode method, and there should be wgettable links of fairseq to get the relevant tokenizer files).  
1b. Upload tokenizer to s3 so your tokenizer tests work on CI. You can work out of the `stas/fairseq-en-de` namespace on your modelhub account and then move everything over (or not) at the end.
2. Get model.forward/ "logits" equivalence (ignore differences less than 1e-6). This usually doesn't work the first time and you have to go line by line with two ipdb sessions (one fairseq, one hf) until you can find the line that's different. At this stage you should worry very little about code quality and just try to get integration tests passing.
3. Get model.generate/ "translation" equivalence. There may be small beam search discrepancies. For this you will need to figure out `decoder_start_token_id`, `num_beams`, and other config settings.
4. Upload Everything to S3.
5. Go through [template](https://github.com/huggingface/transformers/blob/master/templates/adding_a_new_model/README.md#typical-workflow-for-including-a-model)
 and make sure most of the reasonable things are done.
At this point a full integration test (as above) should pass.
6. Check memory, time and BLEU against fairseq (ideally in collab). Improve/document results in PR description. 
7. test the scary parts: special tokens, padding insensitivity.
8. Docs/AutoConfig Etc.
Helpful: https://huggingface.co/transformers/model_sharing.html

Assigned to: @stas00 

Comments:
Excuse me.
Will this model be added in the future, how long will it take?
Is currently only T5 and Bart can do machine translation?

I would guess that I get around to this by the end of July, but I can't be sure.

We also have `MarianMTModel` and 1000+ pretrained weights from `Helsinki-NLP/` that do translation. Here is the list:
https://huggingface.co/Helsinki-NLP


I will work on this one. 

Here is a lazy man's implementation that uses a simple proxy to the fairseq implementation and makes the spec test pass:
```
import torch

class FairseqProxy():
    def __init__(self, module):
        self.module = module
        
    @classmethod
    def from_pretrained(cls, mname): 
        return cls(module=torch.hub.load('pytorch/fairseq', mname, checkpoint_file='model1.pt:model2.pt:model3.pt:model4.pt', tokenizer='moses', bpe='fastbpe'))

class FairseqTranslator(FairseqProxy):
    
    def generate(self, **tokenized_sentences):
        return self.module.generate(tokenized_sentences['data'])
    
class FairseqBPETokenizer(FairseqProxy):

    def prepare_seq2seq_batch(self, sentences): # encode
        return {'data': [self.module.encode(sentence) for sentence in sentences]}
    
    def batch_decode(self, batched_hypos):
        return [self.module.decode(hypos[0]['tokens']) for hypos in batched_hypos]
```

```
# Look ma, I cheated and the test passes ;)
mname = 'transformer.wmt19.ru-en'
model = FairseqTranslator.from_pretrained(mname)
tokenizer = FairseqBPETokenizer.from_pretrained(mname)
batch = tokenizer.prepare_seq2seq_batch(["Машинное обучение - это здорово!"])
translated = model.generate(**batch)
assert tokenizer.batch_decode(translated)[0] == 'Machine learning is great!'
```

Now to the real work of porting...

mostly done: https://github.com/huggingface/transformers/pull/6940

once https://github.com/huggingface/transformers/pull/6940 is merged this issue is to be closed

FYI, Linked Pull requests automatically close the linked issue.

I noticed that you already did the linking after leaving the comment, but decided to leave it as the previous comment of mine wasn't certain ;)