Run TextGenerationPipeline in FP16
Author: bofenghuang
Pull request: no

### System Info

- `transformers` version: 4.26.0.dev0
- Platform: Linux-4.19.0-22-cloud-amd64-x86_64-with-glibc2.17
- Python version: 3.8.15
- Huggingface_hub version: 0.11.1
- PyTorch version (GPU?): 1.13.0+cu117 (True)
- Tensorflow version (GPU?): 2.11.0 (False)
- Flax version (CPU?/GPU?/TPU?): 0.5.3 (cpu)
- Jax version: 0.3.6
- JaxLib version: 0.3.5
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help?

Hi @Narsil,

I just found some other pipelines (e.g., `TextGenerationPipeline`, `Text2TextGenerationPipeline`) can't run fp16 inference any more due to the change in this PR https://github.com/huggingface/transformers/pull/20864.

In fact, the added `torch_dtype` attribute will be unexpectedly thrown into `forward_params` by `_sanitize_parameters()`, then raises an error in `generate()` function.

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Below is a code snippet to reproduce the behavior.

```python
import torch
from transformers import pipeline

generator = pipeline(model="gpt2", device=0, torch_dtype=torch.float16)
generator("I can't believe you did such a ")
```

When running this we see the following stack trace:

```
╭──────────────────────────── Traceback (most recent call last) ────────────────────────────╮
│ <ipython-input-1-f80bcf5e17e1>:6 in <module>                                              │
│ /home/bhuang/transformers/src/transformers/pipelines/text_generation.py:210 in __call__   │
│                                                                                           │
│   207 │   │   │   - **generated_token_ids** (`torch.Tensor` or `tf.Tensor`, present when  │
│   208 │   │   │     ids of the generated text.                                            │
│   209 │   │   """                                                                         │
│ ❱ 210 │   │   return super().__call__(text_inputs, **kwargs)                              │
│   211 │                                                                                   │
│   212 │   def preprocess(self, prompt_text, prefix="", handle_long_generation=None, **gen │
│   213 │   │   inputs = self.tokenizer(                                                    │
│                                                                                           │
│ /home/bhuang/transformers/src/transformers/pipelines/base.py:1074 in __call__             │
│                                                                                           │
│   1071 │   │   elif is_iterable:                                                          │
│   1072 │   │   │   return self.iterate(inputs, preprocess_params, forward_params, postpro │
│   1073 │   │   else:                                                                      │
│ ❱ 1074 │   │   │   return self.run_single(inputs, preprocess_params, forward_params, post │
│   1075 │                                                                                  │
│   1076 │   def run_multi(self, inputs, preprocess_params, forward_params, postprocess_par │
│   1077 │   │   return [self.run_single(item, preprocess_params, forward_params, postproce │
│                                                                                           │
│ /home/bhuang/transformers/src/transformers/pipelines/base.py:1081 in run_single           │
│                                                                                           │
│   1078 │                                                                                  │
│   1079 │   def run_single(self, inputs, preprocess_params, forward_params, postprocess_pa │
│   1080 │   │   model_inputs = self.preprocess(inputs, **preprocess_params)                │
│ ❱ 1081 │   │   model_outputs = self.forward(model_inputs, **forward_params)               │
│   1082 │   │   outputs = self.postprocess(model_outputs, **postprocess_params)            │
│   1083 │   │   return outputs                                                             │
│   1084                                                                                    │
│                                                                                           │
│ /home/bhuang/transformers/src/transformers/pipelines/base.py:990 in forward               │
│                                                                                           │
│    987 │   │   │   │   inference_context = self.get_inference_context()                   │
│    988 │   │   │   │   with inference_context():                                          │
│    989 │   │   │   │   │   model_inputs = self._ensure_tensor_on_device(model_inputs, dev │
│ ❱  990 │   │   │   │   │   model_outputs = self._forward(model_inputs, **forward_params)  │
│    991 │   │   │   │   │   model_outputs = self._ensure_tensor_on_device(model_outputs, d │
│    992 │   │   │   else:                                                                  │
│    993 │   │   │   │   raise ValueError(f"Framework {self.framework} is not supported")   │
│                                                                                           │
│ /home/bhuang/transformers/src/transformers/pipelines/text_generation.py:252 in _forward   │
│                                                                                           │
│   249 │   │   │   in_b = input_ids.shape[0]                                               │
│   250 │   │   prompt_text = model_inputs.pop("prompt_text")                               │
│   251 │   │   # BS x SL                                                                   │
│ ❱ 252 │   │   generated_sequence = self.model.generate(input_ids=input_ids, attention_mas │
│   253 │   │   out_b = generated_sequence.shape[0]                                         │
│   254 │   │   if self.framework == "pt":                                                  │
│   255 │   │   │   generated_sequence = generated_sequence.reshape(in_b, out_b // in_b, *g │
│                                                                                           │
│ /home/bhuang/anaconda3/envs/asr/lib/python3.8/site-packages/torch/autograd/grad_mode.py:2 │
│ 7 in decorate_context                                                                     │
│                                                                                           │
│    24 │   │   @functools.wraps(func)                                                      │
│    25 │   │   def decorate_context(*args, **kwargs):                                      │
│    26 │   │   │   with self.clone():                                                      │
│ ❱  27 │   │   │   │   return func(*args, **kwargs)                                        │
│    28 │   │   return cast(F, decorate_context)                                            │
│    29 │                                                                                   │
│    30 │   def _wrap_generator(self, func):                                                │
│                                                                                           │
│ /home/bhuang/transformers/src/transformers/generation/utils.py:1145 in generate           │
│                                                                                           │
│   1142 │   │                                                                              │
│   1143 │   │   generation_config = copy.deepcopy(generation_config)                       │
│   1144 │   │   model_kwargs = generation_config.update(**kwargs)  # All unused kwargs mus │
│ ❱ 1145 │   │   self._validate_model_kwargs(model_kwargs.copy())                           │
│   1146 │   │                                                                              │
│   1147 │   │   # 2. Set generation parameters if not already defined                      │
│   1148 │   │   logits_processor = logits_processor if logits_processor is not None else L │
│                                                                                           │
│ /home/bhuang/transformers/src/transformers/generation/utils.py:973 in                     │
│ _validate_model_kwargs                                                                    │
│                                                                                           │
│    970 │   │   │   │   unused_model_args.append(key)                                      │
│    971 │   │                                                                              │
│    972 │   │   if unused_model_args:                                                      │
│ ❱  973 │   │   │   raise ValueError(                                                      │
│    974 │   │   │   │   f"The following `model_kwargs` are not used by the model: {unused_ │
│    975 │   │   │   │   " generate arguments will also show up in this list)"              │
│    976 │   │   │   )                                                                      │
╰───────────────────────────────────────────────────────────────────────────────────────────╯
ValueError: The following `model_kwargs` are not used by the model: ['torch_dtype'] (note: 
typos in the generate arguments will also show up in this list)
```


Comments: