Generate up to max_target_length sequences
Author: tetsef
Pull request: yes

* Modifies the generate() call to allow for generation of sequences up to and including max_target_length number of tokens.
* Previous to this commit, implementation caps generation at 20 tokens and may result in poor performance. 
* See related recent generation_utils.py commit: https://github.com/huggingface/transformers/blob/c4d4e8bdbd25d9463d41de6398940329c89b7fb6/src/transformers/generation_utils.py#L139

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5618?src=pr&el=h1) Report
> Merging [#5618](https://codecov.io/gh/huggingface/transformers/pull/5618?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/fa5423b1695cd24856bcff47214172e0f540d924&el=desc) will **decrease** coverage by `0.91%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5618/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5618?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #5618      +/-   ##
==========================================
- Coverage   77.79%   76.88%   -0.92%     
==========================================
  Files         145      145              
  Lines       25355    25355              
==========================================
- Hits        19726    19495     -231     
- Misses       5629     5860     +231     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/5618?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_mobilebert.py](https://codecov.io/gh/huggingface/transformers/pull/5618/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9tb2JpbGViZXJ0LnB5) | `23.38% <0.00%> (-73.39%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_distilbert.py](https://codecov.io/gh/huggingface/transformers/pull/5618/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9kaXN0aWxiZXJ0LnB5) | `64.90% <0.00%> (-33.90%)` | :arrow_down: |
| [src/transformers/generation\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5618/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9nZW5lcmF0aW9uX3RmX3V0aWxzLnB5) | `86.46% <0.00%> (+0.50%)` | :arrow_up: |
| [src/transformers/modeling\_openai.py](https://codecov.io/gh/huggingface/transformers/pull/5618/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19vcGVuYWkucHk=) | `81.09% <0.00%> (+1.37%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_roberta.py](https://codecov.io/gh/huggingface/transformers/pull/5618/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9yb2JlcnRhLnB5) | `93.36% <0.00%> (+49.37%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_openai.py](https://codecov.io/gh/huggingface/transformers/pull/5618/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9vcGVuYWkucHk=) | `95.18% <0.00%> (+74.91%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5618?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5618?src=pr&el=footer). Last update [fa5423b...db13af1](https://codecov.io/gh/huggingface/transformers/pull/5618?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


@sshleifer 

This should depend on `config.max_length`, no?

Config is here:
https://s3.amazonaws.com/models.huggingface.co/bert/t5-small-config.json
and we have the line:
https://github.com/huggingface/transformers/blob/353b8f1e7a7361c0afd9e391381bc226b4a5ca8f/examples/seq2seq/finetune.py#L66

so I think we should continue to let the config determine the generation length.
The cost of this proposal is that people often set `max_target_length` shorter than optimal to make training run faster for training data, but leave `val_max_target_length` long to get a more accurate approximation of Rouge.

Were your summaries getting truncated like #5656 ?

That seems like a better solution. I had been using a different prefix, without an associated config, so the max_length must have defaulted to 20.