New run_seq2seq script
Author: sgugger
Pull request: yes

# What does this PR do?

This PR adds a new version of `finetune_trainer` using datasets and completely self-contained (not using anything in the `utils` module or any other python script of the seq2seq folder). I renamed a few args from the old script, mainly:
- `n_train` -> `max_train_samples`
- `n_val` -> `max_val_samples`
- `src_lang` -> `source_lang`
- `tgt_lang` -> `target_lang`
because they were really too short and uninformative. I didn't touch the other ones for backward compatibility (but since the name of the script will change, we can change more if we feel like it). In any case, the way the dataset main arguments is a breaking change compared to the old script.

The following are the features from the old script not implemented yet and will follow either in this PR or in follow-up PRs:
- [x ] Add a small test on some dummy data (Do not merge before this one is ticked)
- [ ] Ability to freeze the encoder / embeddings
- [ ] Pass a test set for predictions

Comments:
Could we discuss the naming of this script and others?

The description goes:

> Fine-tuning the library models for sequence to sequence.

`run_seq2seq.py` is much less descriptive or intuitive than `finetune_trainer.py `- why not go back to `finetune.py` to replace the script that was PL-based and moved to experiments? 

1. If we are cleaning up the naming, just as well we could drop any `run_` prefices that we now have in many `examples/*/run_*.py` - they are all scripts, they all get to **run**. The names are great when they are focused on their purpose and not how they are executed. 

2. This script is already inside `seq2seq` - So in `examples/seq2seq/run_seq2seq.py` - how does it help to repeat it twice? I can see where you'd want to uniquely identify each script if they are taken out of context of their `examples/*/` subdir - perhaps this is the intention? perhaps if you open them all in the editor and end up with 10 `finetune.py`? If that's the case, then I can see your point of repeating the "domain" in the name of the script.

If the 2nd item is trying to solve the uniqueness issue, then repetition works just fine, but I strongly recommend replacing `run_` with `finetune_` to at the very least have some mnemonics about what it does.



Also, since you will need to update README.md to show users how to run the new script - could we have some of it in this PR? Even just the basic command lines - that would help testing this PR and not needing to figure out the new args? 

If possible that is?

Thank you!

The scripts are all named `run_xxx` precisely for reason 2, the same way we didn't rename `modeling_xxx` files to just `modeling.py` when restructuring the repo. I have no strong objection to changing `run` to `finetune` but it will break lots of links in the documentation and may confuse users, so not sure if it's worth it. I'll let @LysandreJik and @patrickvonplaten chime in on that subject.

I'll add command examples in the README (this PR is not quite ready to be merged yet, there is also the small test to add), first I wanted to grab comments on the actual script before finishing :-) I don't expect it to work fully (though if it's magically the case I'll be happy :-) ) which is why this PR does not delete the old script, so we can make some tests and make sure there is no regression, then progressively fix this new script as needed.

> The scripts are all named run_xxx precisely for reason 2 [...]

I understand. Thank you for clarifying that part. Easy editing is a strong pro for sure.

Thinking more about it perhaps `finetune` isn't the right name either because it does finetuning plus prediction, so perhaps `run` actually is somewhat of a better choice, as it's less committing to anything ;)

I think what I'm experiencing here is the pain of pattern breaking. First I was using "finetune.py", then I switched to "finetune_trainer.py" and now "run_seq2seq.py" - say, what? :)

> I'll add command examples in the README  [...]

I'd like to contribute with the review, but I need context to do such things and there is neither diff nor a way to run it, I'm just not sure how to approach such type of review. So perhaps I will be able to do that at a later stage when I can test the new script, or if you'd like me to look at a particular part of it I'm game too.

I am merging as a first step. @stas00 I know it's missing examples of use and that there still is the memory regression, I plan to address those in follow-up PRs (also anyone should feel free to suggest improvements to the new scripts).

It's a good plan, @sgugger! I know you won't forget these. Thank you for considering my concerns.

The only thing I am not sure about is that nobody commented on the new script's naming.