[example scripts] disambiguate language specification API
Author: stas00
Pull request: no

Currently in example scripts like `run_seq2seq.py` we have:

1. for t5
```
--task translation_en_to_ro
--source_prefix "translate English to Romanian: "
```

2. Also these 2:
```
--target_lang ro_RO
--source_lang en_XX
```
are used only for MBart and are ignored for other models. Which means that people will unknowingly try to use these two as well when they aren't need.
 
The problem in both situations is that we provide error-prone API where a user wants to change the language and forgets that there is more than one of the same and changes only one of the sets of languages, but not the other, which leads to broken outcome.

If such an error is made the specification supplied by the user becomes ambiguous, because one can't tell which of the multiple inputs takes precedence.

Proposal: There should be only one way to input a set of languages and not multiple ways.

Specifically:

- in case 1, probably the easiest is to leave `--task translation_en_to_ro` and auto-generate `--source_prefix "translate English to Romanian: "`
- in case 2, assert if `--target_lang` or `--source_lang` are passed and the model is not MBart.

Thinking more about it, case 1 is a must to solve, because if a user misses `--source_prefix` or makes a typo in it - the train/eval won't fail, but will mysteriously produce really bad outcome. This is not user-friendly.

@sgugger, @patrickvonplaten, @patil-suraj 

Comments:
Regarding "case 1", only the "old" T5 models: `t5-small`, `t5-base`, `t5-large`, `t5-3b` and `t5-11b` were trained with the `source_prefix` and not the new T5 models. Also IMO, there is a very legitimate case that people might want to fine-tune `t5-small`, `t5-base`, ... on translation, but don't want to condition the model on the prefix. In the second case, I agree more that we should probably raise if `--target_lang` and/or `--source_lang` are not given. For the first case, I'm fine with adding a warning

So if it is one of the 4 models that you listed print a warning to set `--source_prefix` if it's an exact match for model name and the flag wasn't passed, right?

Is it just `run_seq2seq.py` or are there any other scripts that need these 2 special supports?

Yeah, I think this would be a good idea! Think it's only T5 so only `run_seq2seq.py`

I can take this on #10611, what I'd do is remove `source_prefix` and map `(src,tgt)` pairs to matching `source_prefix` values when the `model_name` matches the "older" T5 models.
But I'd need said mapping, do you have pointers?

Skimming through the T5 paper it seems the mapping is quite small, `en->{fr,de,ro}`? 
If so no need to build an exhaustive mapping of 2 letter ISO codes to capitalized language names, and I can issue a warning when `{src,tgt}_lang` is out of "supported" language-pairs.

This is for the pre-trained models, but if a user provides their own model it could be any language.

Plus you have https://github.com/google-research/multilingual-t5.

I wonder if there is a python module that comes with such a map.

https://gist.github.com/carlopires/1262033/c52ef0f7ce4f58108619508308372edd8d0bd518

Language mapping:
Here you go: https://github.com/LuminosoInsight/langcodes
or another alternative: https://github.com/janpipek/iso639-python

Thanks, I had found a couple of mappings in json as well, should we hardcode them or use an external dependency?

we require running `pip install -r examples/seq2seq/requirements.txt` already, so why not follow suite.

This issue hasn't been resolved. @theo-m solved it initially and linked to it, but then the group didn't like the solution and it was reverted. So a user is still required to enter the language pair twice. Not a great example.

well, it looks like the overall agreement is that examples don't have to be perfect, they are just examples.