Small Bug: text2text_generation and text_generation result in duplicate results
Author: FrankDataAnalystPython
Pull request: no

- `transformers` version: 4.17.0
- Platform: Darwin-20.1.0-x86_64-i386-64bit
- Python version: 3.7.2
- PyTorch version (GPU?): 1.7.1 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>


Library:
- Text generation: @patrickvonplaten @narsil
- Pipelines: @Narsil

## Information

I would like to report a small bug inside the text2text_generation.py and the text_generation.py. When I am using 'return_tensors=True' and 'do_sample=True' inside the __call__, it will result in duplicated tensors.

The bug locates at the following lines:
Line 229: transformers.pipelines.text_generation.py
Line 172: transformers.pipelines.text2text_generation.py

For example at Line 229
```python
generated_sequence = generated_sequence.numpy().tolist()
records = []
for sequence in generated_sequence:
    if return_type == ReturnType.TENSORS:
        record = {"generated_token_ids": generated_sequence}
        # record = {"generated_token_ids": sequence}  # should be this?
    elif return_type in {ReturnType.NEW_TEXT, ReturnType.FULL_TEXT}:
      	...
```

The generated_sequence has length which equals to num_return_sequences. And For each sequence, it would either do nothing and add the tensor into the records, or do decoding. However, the whole generated_sequence is added into the record at each loop. This also happens in Line 172: transformers.pipelines.text2text_generation.py

```python
records = []
for output_ids in model_outputs["output_ids"][0]:
    if return_type == ReturnType.TENSORS:
        record = {f"{self.return_name}_token_ids": model_outputs}
        # record = {f"{self.return_name}_token_ids": output_ids} # should be this?
    elif return_type == ReturnType.TEXT:
```

Model I am using (GPT2):

## To reproduce

Steps to reproduce the behavior:

```python
from transformers import pipeline
text_generator = pipeline("text-generation")
result = text_generator(
    "As far as I am concerned, I will", 
    max_length=10, 
    do_sample=True, 
    num_return_sequences=3, 
    return_tensors=True
)
```

```
[{'generated_token_ids': 
 [[1722, 1290, 355, 314, 716, 5213, 11, 314, 481, 307],
  [1722, 1290, 355, 314, 716, 5213, 11, 314, 481, 307],
  [1722, 1290, 355, 314, 716, 5213, 11, 314, 481, 691]]},
{'generated_token_ids': 
 [[1722, 1290, 355, 314, 716, 5213, 11, 314, 481, 307],
  [1722, 1290, 355, 314, 716, 5213, 11, 314, 481, 307],
  [1722, 1290, 355, 314, 716, 5213, 11, 314, 481, 691]]},
{'generated_token_ids': 
 [[1722, 1290, 355, 314, 716, 5213, 11, 314, 481, 307],
  [1722, 1290, 355, 314, 716, 5213, 11, 314, 481, 307],
  [1722, 1290, 355, 314, 716, 5213, 11, 314, 481, 691]]}]
```

## Expected behavior
```
[{'generated_token_ids': 
  [1722, 1290, 355, 314, 716, 5213, 11, 314, 481, 307]},
 {'generated_token_ids': 
  [1722, 1290, 355, 314, 716, 5213, 11, 314, 481, 307]},
 {'generated_token_ids': 
  [1722, 1290, 355, 314, 716, 5213, 11, 314, 481, 691]}]
```

This is the first time I am contributing to the great transformers, very excited. Should I make a pull request?

Regrads
Xiangyang Ni



Comments:
Hey @FrankDataAnalystPython,

Great job discovering the bug :slightly_smiling_face: ! 
It would be great if you could open a PR to fix it

@FrankDataAnalystPython 
Thanks for the report, opened a PR for the fix ! 

https://github.com/huggingface/transformers/pull/16828

Dear @Narsil:
 I saw you already finished the PR. Should I close the current issue now?

Many Thanks!
Regards
Xiangyang Ni