[examples (seq2seq)] fix preparing decoder_input_ids for T5 
Author: patil-suraj
Pull request: yes

possible fix for #5987
`
` should be added at the end of target text

@sshleifer 

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5994?src=pr&el=h1) Report
> Merging [#5994](https://codecov.io/gh/huggingface/transformers/pull/5994?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/33d7506ea10ca92886fd1bb3b5306a1a720c58fe&el=desc) will **decrease** coverage by `0.23%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5994/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5994?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #5994      +/-   ##
==========================================
- Coverage   78.48%   78.25%   -0.24%     
==========================================
  Files         146      146              
  Lines       26230    26230              
==========================================
- Hits        20587    20525      -62     
- Misses       5643     5705      +62     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/5994?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_openai.py](https://codecov.io/gh/huggingface/transformers/pull/5994/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9vcGVuYWkucHk=) | `20.27% <0.00%> (-74.92%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_distilbert.py](https://codecov.io/gh/huggingface/transformers/pull/5994/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9kaXN0aWxiZXJ0LnB5) | `64.90% <0.00%> (-33.90%)` | :arrow_down: |
| [src/transformers/generation\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5994/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9nZW5lcmF0aW9uX3RmX3V0aWxzLnB5) | `84.71% <0.00%> (-1.51%)` | :arrow_down: |
| [src/transformers/modeling\_openai.py](https://codecov.io/gh/huggingface/transformers/pull/5994/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19vcGVuYWkucHk=) | `81.02% <0.00%> (-1.29%)` | :arrow_down: |
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5994/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `81.49% <0.00%> (+0.29%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_roberta.py](https://codecov.io/gh/huggingface/transformers/pull/5994/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9yb2JlcnRhLnB5) | `93.36% <0.00%> (+49.37%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_electra.py](https://codecov.io/gh/huggingface/transformers/pull/5994/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9lbGVjdHJhLnB5) | `95.53% <0.00%> (+69.51%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5994?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5994?src=pr&el=footer). Last update [33d7506...fbf4b03](https://codecov.io/gh/huggingface/transformers/pull/5994?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


I will let you know ASAP!

As far as I know, @mrm8488  hasn't used finetune.py for T5. He has linked his training colabs in model cards and they just pass `labels` directly and `decoder_input_ids` are created by the `T5ForConditionalGeneration`

 I usually follow this one https://github.com/patil-suraj/exploring-T5/blob/master/T5_on_TPU.ipynb

Also I think #5866 should be merged before this, because if `
` is not added then it might miss the last token 

> can we unittest this behavior somehow?

if we move this logic to dataset or collate function then we can unit test `__getitem__`  or `collat_fn`. Would this be a good idea ?

Yes, great idea. Whichever seems more natural.

Hi @sshleifer , does this PR needs any other changes before merging ?

sorry for being slow!