[Seq2Seq] Fix a couple of bugs and clean examples
Author: patrickvonplaten
Pull request: yes

# What does this PR do?

## **IMPORTANT** - BREAKING CHANGES 
This PR changes the behavior of **T5** and **TFT5** due to 3 bugs and 1 small change in forward API to support onnx and torchscript.
It also slighlty changes the behavior of **Bart** and **EncoderDecoderModel**

## Description 

_1st Bug_: Due to a sloppy review from my part these lines got merged into the PyTorch T5 model: https://github.com/huggingface/transformers/pull/5518/files#r496058908, which set `decoder_input_ids = input_ids` if `decoder_input_ids` were not provided. This is misleading and also just wrong. Never should `decoder_input_ids=input_ids` in T5. This is not done during training nor during inference, so these lines don't make much sense. Because T5 is mostly either used with `.generate()` or in training with `model(input_ids=input_ids, labels=labels)`, in which cases the change has no effect we only received one issue recently about it #7358. The change was done to make T5 work with onnx, but is just wrong IMO.

_2nd Bug_: T5 was implemented with a small bug regarding the relative distance bias calculation for the cross-attention layer. It was spotted here: #7323. The correction leads to slightly different results when doing beam search. @sshleifer - if it's easy for you could maybe run a quick eval on WMT to see if bleu improves in this PR? 

_3rd Bug_: T5 currently cuts the `input_ids` to the last token when `past` is used. This is a convenient function for the user, but has the potential to lead to bugs as mentioned here: https://github.com/huggingface/transformers/issues/4368#issuecomment-630244541. It's not really in the spirit of the library to do some magic under-the-hood which make certain use-cases easier for the user, but prevents other edge cases as shown in the issue above.

Feature request: support torchscript and onnx. This PR allows to use T5 with torchscript and onnx.

Now the difficult part: **This PR has breaking changes!**

For once, all three bugs lead to breaking changes. Then in order to solve the torchscript, onnx problem we are having with T5 (and we actually have with all Seq2Seq models), I had to change the positional ordering of T5's forward pass slightly, which should have minimal breaking changes because I doubt anybody has used T5 with positional arguments as follows:

`tf_model(input_ids, None, None, decoder_input_ids)`.

We had a couple of issues *e.g.* #5647 about supporting torchscript and onnx for Bart/T5. If we ever want to support onnx and torchscript in the future, I think we need to do this positional reordering. As shown by @mfuntowicz onnx can lead to great speed improvements and we also know now that `torchscript` can give ~30% speed improvement on dynamic input sizes. => I would be really happy if we could accept this slight breaking change here. 

I thought about this quite a bit and I think it's very important that we agree on ONE positional argument ordering for the forward pass of Seq2Seq models. At the moment the ordering of Bart, EncoderDecoder, T5, ... is not coherent and done in a way that does not support onnx and torchscript. At the moment no seq2seq model really supports torchscript (Bart does in the test, but one cannot provide `decoder_input_ids` when using torchscript which effectively makes torchscript useless for inference). The ordering should be as follows IMO:

`input_ids`
`attention_mask`
`decoder_input_ids`
`decoder_attention_mask`
`encoder_outputs` 
...,

meaning that all `required` inputs should come first to comply with onnx and torchscript and optional ones should come after.

I changed the ordering of all seq2seq models to comply with this format even though we have some positional ordering breaking changes for `T5`, `Bart` and `EncoderDecoder`.

## UPDATE:

Added tests that the encoder decoder forward signature stays the same
Applied changes to all Seq2Seq models
Cleaned docs
Fixed TF slow tests

Comments:
You got a little zero-shot BLEU boost it seems!

This Branch: 34.433 (on en-de)
Master: 34.4052



@patrickvonplaten 

Is the docstring still wrong? [T5ForConditionalGeneration](https://huggingface.co/transformers/model_doc/t5.html?highlight=t5forconditional#transformers.T5ForConditionalGeneration), under `decoder_input_ids` says "if both decoder_input_ids and decoder_inputs_embeds are both unset, decoder_input_ids takes the value of input_ids"

> @patrickvonplaten
> 
> Is the docstring still wrong? [T5ForConditionalGeneration](https://huggingface.co/transformers/model_doc/t5.html?highlight=t5forconditional#transformers.T5ForConditionalGeneration), under `decoder_input_ids` says "if both decoder_input_ids and decoder_inputs_embeds are both unset, decoder_input_ids takes the value of input_ids"

I can't find this docstring, can you link to it?

> > @patrickvonplaten
> > Is the docstring still wrong? [T5ForConditionalGeneration](https://huggingface.co/transformers/model_doc/t5.html?highlight=t5forconditional#transformers.T5ForConditionalGeneration), under `decoder_input_ids` says "if both decoder_input_ids and decoder_inputs_embeds are both unset, decoder_input_ids takes the value of input_ids"
> 
> I can't find this docstring, can you link to it?

In T5, it's under method `forward` and argument `decoder_input_ids`. Here's [code link](https://github.com/huggingface/transformers/blob/eb3bd73ce35bfef56eeb722d697f2d39a06a8f8d/src/transformers/modeling_t5.py#L869)

Oh yeah you're right that was a bad copy & past probably! Do you feel like opening a PR to fix it / delete it? That would be amazing :-)