Text2TextGeneration Pipeline : Batch size and num_return_sequences are not working together
Author: ierezell
Pull request: no
Labels: name: bug

### System Info

```shell
- `transformers` version: 4.16.2
- Platform: Linux-5.13.0-40-generic-x86_64-with-glibc2.31
- Python version: 3.9.12
- PyTorch version (GPU?): 1.11.0+cu113 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No
```


### Who can help?

Hello @Narsil,

When using a pipeline I wanted to speed up the generation and thus use the `batch_size` parameter. 
In a `Text2TextGenerationPipeline`, with `num_return_sequences` of 1 everything works fine and I have a x3 speedup when using a `batch_size` of 8 !

However, I would like to use `num_return_sequences > 1`. Setting this still leads to the same amount of output utterances of input (and not twice as much (2 per input utterance if `num_return_sequences` is =2 )) and after investigation, I realized that the Text2TextPipeline has a `__call__` method with `[res[0] for res in results]` so I decided to remove it (with a custom class) to be able to have the "num_return_sequences * input_len".
```python
class MultipleText2TextGenerationPipeline(Text2TextGenerationPipeline):
    def __call__(self, *args, **kwargs) -> list[str]:
        result: list[list[dict[Literal["generated_text"], str]]] = super(Text2TextGenerationPipeline, self).__call__(*args, **kwargs)
        flatten_results: list[str] = []
        for result_list in result:
            for result_dict in result_list:
                flatten_results.append(result_dict["generated_text"].replace("question: ", ""))
        return flatten_results
```

When, using `batch_size` with `num_return_sequences > 1` lead to weird output like having 24 outputs with `batch_size = 8` and `num_return_sequences=3`.... When using `num_return_sequences=1` I have the good output meaning 60 sentences if I places 20 uttrances as input with `num_return_sequences=3` but not for `batch_size>1`

Thanks in advance for any help, 
Have a great day. 


### Information

- [X] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction
```python
from time import perf_counter 
from torch import bfloat16
from transformers import Text2TextGenerationPipeline
from transformers.modeling_utils import PreTrainedModel
from transformers.models.auto.modeling_auto import AutoModelForSeq2SeqLM
from transformers.models.auto.tokenization_auto import AutoTokenizer
from transformers.tokenization_utils import PreTrainedTokenizer

class MultipleText2TextGenerationPipeline(Text2TextGenerationPipeline):
    def __call__(self, *args, **kwargs) -> list[str]:
        result: list[list[dict[Literal["generated_text"], str]]] = super(Text2TextGenerationPipeline, self).__call__(*args, **kwargs)
        flatten_results: list[str] = []
        for result_list in result:
            for result_dict in result_list:
                flatten_results.append(result_dict["generated_text"].replace("question: ", ""))
        return flatten_results

tokenizer: PreTrainedTokenizer = AutoTokenizer.from_pretrained("mrm8488/t5-base-finetuned-question-generation-ap")
model: PreTrainedModel = AutoModelForSeq2SeqLM.from_pretrained("mrm8488/t5-base-finetuned-question-generation-ap")
pipeline = MultipleText2TextGenerationPipeline(model=model,tokenizer=tokenizer, device=0)

input_texts = [f"answer: {ans} context: I like to eat bananas in the morning" for ans in ['I', 'bananas', 'morning', 'yes', 'no']]

DEFAULT_GENERATOR_OPTIONS = {
    "max_length": 128,
    "min_length": 2,
    "early_stopping": True,
    "num_beams": 3,
    "temperature": 1.0,
    "top_k": 0,
    "top_p": 0.92,
    "repetition_penalty": 2.0,
    "length_penalty": 1.0,
}

start=perf_counter()
print(len(input_texts))
print(f"expecting {len(input_texts)} got : ", end=" ")
print(len(pipeline(input_texts, **DEFAULT_GENERATOR_OPTIONS, num_return_sequences=1))) # 20
print(perf_counter()-start)

start=perf_counter()
print(len(input_texts))
print(f"expecting {len(input_texts)*3} got : ", end=" ")
print(len(pipeline(input_texts, **DEFAULT_GENERATOR_OPTIONS, num_return_sequences=3))) # 60
print(perf_counter()-start)


start=perf_counter()
print(len(input_texts))
print(f"expecting {len(input_texts)*3} got : ", end=" ")
print(len(pipeline(input_texts, **DEFAULT_GENERATOR_OPTIONS, num_return_sequences=3, batch_size=8))) # 24
print(perf_counter()-start)
```
### Expected behavior

```shell
If setting the number of return sequence of a list of string, return a list of len(list)*num_return_sequences
```


Comments:
Solved with huggingface 4.19.2. 

Sorry for all the fuss. Maybe it will help someone someday.