Truncated Translations with mT5 model
Author: sumanthd17
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.2
- Platform: GCP
- Python version: 3.7
- PyTorch version (GPU?): 1.7
- Using GPU in script?: V100
- Using distributed or parallel set-up in script?: using pytorch-lightning `dp` setup

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 TextGeneration: @TevenLeScao
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten @TevenLeScao
 Blenderbot: @patrickvonplaten
 Bart: @patrickvonplaten
 Marian: @patrickvonplaten
 Pegasus: @patrickvonplaten
 mBART: @patrickvonplaten
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 RAG: @patrickvonplaten, @lhoestq
 FSMT: @stas00
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 ray/raytune: @richardliaw @amogkam
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

## Information

Model I am using (Bert, XLNet ...): I'm trying to fine-tune mT5 for neural machine translation. I trained the model on 3M data points with a `max_seq_len` of `200`. But when I perform `model.generate` I'm getting truncated outputs. 

The input sequence which I'm inferencing is not a very long input and no where close to 200 tokens.

The problem arises when using:
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] my own task or dataset: (give details below)

## To reproduce

I'm using my own dataset and training the model by wrapping the code around with pytorch-lightning.

I played around with the parameters in `generate` method but all combinations still endup giving truncated outputs



<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behaviour

An example output generated by the model 

Generated output:
`I hope that representatives from abroad will get some time to see Delhi's`

The quality of translation is good but the sentence ends abruptly.

Expected output: 
`I hope that delegates from abroad will have some time to see the history and pride of Delhi.` (generated with google translate)

@patrickvonplaten tagging you for help as this is a T5 related model

Thanks in Advance

<!-- A clear and concise description of what you would expect to happen. -->


Comments:
I've read these similar issue

[#5654](https://github.com/huggingface/transformers/issues/5656)
[#7500](https://github.com/huggingface/transformers/issues/7500)

but couldn't get the information needed.

@sshleifer @patil-suraj Any idea why this is happening? (Sorry for tagging everyone, I've been facing this issue for a while, looking for answers 🙂)

Hi @sumanthd17 ,
I had this issue with standard T5 in which it was ignoring the `min_length` flag. I fixed this simply by upping the `num_beams` to 4.

Could you test this and see if it's quick fix for you?

@FL33TW00D Thanks. Yeah the `min_length` flag is being ignored. 

Thanks for the quick fix. I think we can close this issue for now, But it might be a good idea to know why the min_length is being ignored.

@sumanthd17 
This blog post from Patrick explains why min_length may not be satisfied by the beam_search:
https://huggingface.co/blog/how-to-generate

Although I do agree perhaps it should throw a warning message when it is unable to satisfy the min_length flag with the current generation parameters.