MBART/Marian for low resource/backtranslation
Author: tuhinjubcse
Pull request: no

Are there any finetuned checkpoints of mBART on any low resource language release by hugging face. I can see English- Romanian but was wondering if there is a way to access other languages. Did facebook release them actually?

@sshleifer 

Comments:
Nope, use Helsinki-NLP!

LMK if you have trouble finding your language.

**Swahili / Uzbek / Telugu / Gujrati** any of them will do. Also, I just need to do back-translation, do u have a code snippet to do translation using  Helsinki-NLP

```python
mname = 'Helsinki-NLP/opus-mt-en-sw'
src_text = ['I am a small frog with tiny legs.']
torch_device = 'cpu'#'cuda' if torch.cuda.is_available() else 'cpu'
model = AutoModelForSeq2SeqLM.from_pretrained(mname)
tok = AutoTokenizer.from_pretrained(mname)
translated = model.generate(**tok(src_text, return_tensors='pt'))
tok.batch_decode(translated, skip_special_tokens=True)
# ['Mimi ni chura mdogo mwenye miguu midogo.']
```
Full list of models [here](https://huggingface.co/Helsinki-NLP)

let's move to [here](https://discuss.huggingface.co/t/marian-language-discovery-questions/739/3) for further language discovery questions! Thanks!

Backtranslation snippet:
```python
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
mname_fwd = 'Helsinki-NLP/opus-mt-en-ceb'
mname_bwd = 'Helsinki-NLP/opus-mt-ceb-en'
src_text = ['I am a small frog with tiny legs.']
torch_device = 'cpu'#'cuda' if torch.cuda.is_available() else 'cpu'
fwd = AutoModelForSeq2SeqLM.from_pretrained(mname_fwd).to(torch_device)
fwd_tok = AutoTokenizer.from_pretrained(mname_fwd)
bwd_tok = AutoTokenizer.from_pretrained(mname_bwd)
bwd = AutoModelForSeq2SeqLM.from_pretrained(mname_bwd).to(torch_device)
if torch_device == 'cuda':
    fwd = fwd.half()
    bwd = bwd.half()

fwd_batch = fwd_tok(src_text, return_tensors='pt').to(torch_device)
translated = fwd.generate(**fwd_batch)
translated_txt = fwd_tok.batch_decode(translated, skip_special_tokens=True)
bwd_batch = bwd_tok(translated_txt, return_tensors='pt').to(torch_device)
backtranslated = bwd.generate(**bwd_batch)
bwd_tok.batch_decode(backtranslated, skip_special_tokens=True)
# ['I am a small toad with small feet.']
```

```
import ast
import torch
import os
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
os.environ["CUDA_VISIBLE_DEVICES"]="2"
model1 =  'Helsinki-NLP/opus-mt-en-ceb'
model2 =  'Helsinki-NLP/opus-mt-ceb-en'
torch_device = 'cuda' if torch.cuda.is_available() else 'cpu'
fwd = AutoModelForSeq2SeqLM.from_pretrained(model1).to(torch_device)
fwd_tok = AutoTokenizer.from_pretrained(model1)
bwd_tok = AutoTokenizer.from_pretrained(model2)
bwd = AutoModelForSeq2SeqLM.from_pretrained(model2).to(torch_device)
if torch_device == 'cuda':
        fwd = fwd.half()
        bwd = bwd.half()


for line in open('train_rep.txt'):
        line = ast.literal_eval(line.strip())
        fwd_batch = fwd_tok(line, return_tensors='pt').to(torch_device)
        translated = fwd.generate(**fwd_batch)
        translated_txt = fwd_tok.batch_decode(translated, skip_special_tokens=True)
        bwd_batch = bwd_tok(translated_txt, return_tensors='pt').to(torch_device)
        backtranslated = bwd.generate(**bwd_batch)
        orginal_text = bwd_tok.batch_decode(backtranslated, skip_special_tokens=True)
        print(orginal_text)
        break

```

got



 ```
 File "/nas/home/tuhinc/miniconda3/envs/advattack/lib/python3.6/site-packages/transformers/tokenization_utils_base.py", line 555, in convert_to_tensors
    tensor = as_tensor(value)
ValueError: expected sequence of length 27 at dim 1 (got 47)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "bt.py", line 25, in <module>
    fwd_batch = fwd_tok(line, return_tensors='pt').to(torch_device)
  File "/nas/home/tuhinc/miniconda3/envs/advattack/lib/python3.6/site-packages/transformers/tokenization_utils_base.py", line 1954, in __call__
    **kwargs,
  File "/nas/home/tuhinc/miniconda3/envs/advattack/lib/python3.6/site-packages/transformers/tokenization_utils_base.py", line 2139, in batch_encode_plus
    **kwargs,
  File "/nas/home/tuhinc/miniconda3/envs/advattack/lib/python3.6/site-packages/transformers/tokenization_utils.py", line 548, in _batch_encode_plus
    verbose=verbose,
  File "/nas/home/tuhinc/miniconda3/envs/advattack/lib/python3.6/site-packages/transformers/tokenization_utils.py", line 614, in _batch_prepare_for_model
    batch_outputs = BatchEncoding(batch_outputs, tensor_type=return_tensors)
  File "/nas/home/tuhinc/miniconda3/envs/advattack/lib/python3.6/site-packages/transformers/tokenization_utils_base.py", line 186, in __init__
    self.convert_to_tensors(tensor_type=tensor_type, prepend_batch_axis=prepend_batch_axis)
  File "/nas/home/tuhinc/miniconda3/envs/advattack/lib/python3.6/site-packages/transformers/tokenization_utils_base.py", line 572, in convert_to_tensors
    "Unable to create tensor, you should probably activate truncation and/or padding "
ValueError: Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the same length.


```


It works for single sentences but fails for an array or sentences

Got it had to do padding=True

`padding='longest'` is the best setting IMO.