"text2text-generation" pipeline fails when setting return_dict_in_generate=True
Author: KMFODA
Pull request: no

### System Info

- `transformers` version: 4.23.1
- Platform: macOS-12.5.1-arm64-arm-64bit
- Python version: 3.9.4
- Huggingface_hub version: 0.10.1
- PyTorch version (GPU?): 1.12.1 (False)
- Tensorflow version (GPU?): 2.10.0 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@Narsil

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Setting `return_dict_in_generate` to `True` in the `text2text-generation` pipeline returns the following error:

```
Traceback (most recent call last):
  File "/Users/karimfoda/.asdf/installs/python/3.9.4/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/Users/karimfoda/.asdf/installs/python/3.9.4/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Users/karimfoda/.vscode/extensions/ms-python.python-2022.20.2/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/__main__.py", line 39, in <module>
    cli.main()
  File "/Users/karimfoda/.vscode/extensions/ms-python.python-2022.20.2/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 430, in main
    run()
  File "/Users/karimfoda/.vscode/extensions/ms-python.python-2022.20.2/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 284, in run_file
    runpy.run_path(target, run_name="__main__")
  File "/Users/karimfoda/.vscode/extensions/ms-python.python-2022.20.2/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 321, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/Users/karimfoda/.vscode/extensions/ms-python.python-2022.20.2/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 135, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/Users/karimfoda/.vscode/extensions/ms-python.python-2022.20.2/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 124, in _run_code
    exec(code, run_globals)
  File "/Users/karimfoda/Documents/STUDIES/PYTHON/KAIZAN/DATASETS/scripts/debug_return_dict_in_generate_error.py", line 6, in <module>
    print(sentiment_t5_model("hello this is a test", return_dict_in_generate=True, output_scores = True))
  File "/Users/karimfoda/Documents/STUDIES/PYTHON/KAIZAN/DATASETS/_env/lib/python3.9/site-packages/transformers/pipelines/text2text_generation.py", line 148, in __call__
    result = super().__call__(*args, **kwargs)
  File "/Users/karimfoda/Documents/STUDIES/PYTHON/KAIZAN/DATASETS/_env/lib/python3.9/site-packages/transformers/pipelines/base.py", line 1074, in __call__
    return self.run_single(inputs, preprocess_params, forward_params, postprocess_params)
  File "/Users/karimfoda/Documents/STUDIES/PYTHON/KAIZAN/DATASETS/_env/lib/python3.9/site-packages/transformers/pipelines/base.py", line 1081, in run_single
    model_outputs = self.forward(model_inputs, **forward_params)
  File "/Users/karimfoda/Documents/STUDIES/PYTHON/KAIZAN/DATASETS/_env/lib/python3.9/site-packages/transformers/pipelines/base.py", line 990, in forward
    model_outputs = self._forward(model_inputs, **forward_params)
  File "/Users/karimfoda/Documents/STUDIES/PYTHON/KAIZAN/DATASETS/_env/lib/python3.9/site-packages/transformers/pipelines/text2text_generation.py", line 173, in _forward
    output_ids = output_ids.reshape(in_b, out_b // in_b, *output_ids[0].shape[1:])
AttributeError: 'GreedySearchEncoderDecoderOutput' object has no attribute 'reshape'
```

Running the following code reproduces this error:

```
from transformers import pipeline, AutoTokenizer, AutoModelWithLMHead, AutoModelForCausalLM

sentiment_t5_model = pipeline("text2text-generation", model = "mrm8488/t5-base-finetuned-imdb-sentiment")

print(sentiment_t5_model("hello this is a test", return_dict_in_generate=True, output_scores = True))
```

### Expected behavior

The expected output is:

`[{'generated_text': 'positive', 'scores': (tensor([[-19.5107, -12.7762, -13.3044,  ..., -41.9292, -41.8459, -41.9196]]), tensor([[-69.0289,  -8.4889, -31.7621,  ..., -75.5579, -75.6114, -75.5323]]))}]`

I was able to produce this output and fix this issue by changing:

https://github.com/huggingface/transformers/blob/6d67664380c09a1e9e1e3771f2124cd49b72f6be/src/transformers/pipelines/text2text_generation.py#L188-L192

to:

```
out_b = output_ids['sequences'].shape[0]
if self.framework == "pt":
    output_ids['sequences'] = output_ids['sequences'].reshape(in_b, out_b // in_b, *output_ids['sequences'].shape[1:])
elif self.framework == "tf":
    output_ids['sequences'] = tf.reshape(output_ids, (in_b, out_b // in_b, *output_ids['sequences'].shape[1:]))
```

and

https://github.com/huggingface/transformers/blob/6d67664380c09a1e9e1e3771f2124cd49b72f6be/src/transformers/pipelines/text2text_generation.py#L201-L206

to:

```
record = {
    f"{self.return_name}_text": self.tokenizer.decode(
        output_ids,
        skip_special_tokens=True,
        clean_up_tokenization_spaces=clean_up_tokenization_spaces,
    ),
    "scores":model_outputs["output_ids"]['scores']
}
```

if this is an acceptable fix happy to submit a PR for these changes.

Comments:
This is only valid if we indeed have the argument `return_dict_in_generate`. Otherwise the pipeline will also fail because `output_ids` will not be a dictionary. Pipelines in general currently don't support outputting anything else than the text prediction. See #21274. @Narsil do you think we could support something like `output_generate_dict` which would just output everything? (might be useful for people who want to use all in one tokenizer, feature extractor and model but still post process)

> want to use all in one tokenizer, feature extractor and model but still post process

Feels a bit power usery to me.

Two options :

- Subclass pipeline and use it instead `pipeline(..., pipeline_class=MyOwnClass)` which will use your subclass where everything is free to modify (and still benefit from batching and such).
- Make it shareable to the world with a custom pipeline: https://huggingface.co/docs/transformers/v4.26.0/en/add_new_pipeline#how-to-create-a-custom-pipeline

There are **many** things that could be done within text-generation pipeline, but I fear we should be very sparse in what we agree to add and maintain. The main goal of the pipeline is to be useable by non-ML people, meaning, we need to refrain from adding many use cases which include understanding how the tokens work. Advances usage is always possible with lower level tools and I feel that's where they belong.

Does that make sense ?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.