add torch.compile in pipeline
Author: jiqing-feng
Pull request: yes

Hi @Narsil . Since torch compile is supported in pytorch2.0, see [here](https://pytorch.org/tutorials/intermediate/torch_compile_tutorial.html). 
However, users usually use pipeline and don't know which model function (`forward` or `generate`) should be compiled. 

I was thinking add `torch.compile` in pipeline to make users easier to use it. Would like to hear your opinion. Thank!

BTW, do you have any idea about how to determine which model function should be compiled? I compiled both model functions because I don't know which function will be used by the model.

Comments:
cc @gante @sywangyi 

Hi @jiqing-feng, thanks for opening this PR! 

I don't think this is a feature we want to add at the moment. Users can already pass compiled models to the pipeline to use, and there they can control which parts will or won't be compiled. There's many optimization choices one might make when loading a model e.g. whether to quantize the weights. To keep the API of the pipeline as simple as possible - we can leave additional configuration of the model outside of the pipeline.

I'll let @Narsil's opinion weigh more heavily on whether this should be added to the pipeline over mine. 


The main idea is that users may not know if the `pipeline` uses `model.forward` or `model.generate`. If we integrate `torch.compile` in the `pipeline`, we can decide which function should be compiled. We only need 2 extra params: `torch_compile` and `torch_compile_config`, or we can just pass `torch_compile_backend` to `kwargs`. It will help users to apply torch compile and avoid compiling useless functions easily. 

For example, in [ASR pipeline](https://github.com/huggingface/transformers/blob/main/src/transformers/pipelines/automatic_speech_recognition.py#L541-L601), it uses `model.generate` if `model_type in {"seq2seq", "seq2seq_whisper"}`, and use `model.forward` otherwise. 

I think users won't check the `pipeline` code in such detail, so the best way is we define which model function needs to be compiled in `pipeline`.

BTW, it is important to clarify that `model=torch.compile(model)` won't work on `model.generate.forward`, the model in generation will still use the original forward function to inference.

yes. user may not be aware of the calling API in each Task of Pipeline in __call__. we could do it for user in each task if user want to use torch.compile for acceleration. for some task, we see 20%-30% perf boost.

Hi @Narsil . Do you think that we should add `torch.compile` in the pipeline since users may not know which function of model should be compiled?

No you can and should use ` pipeline(model=torch.compile(model)`.
If that doesn't work with generate it's up to the caller to know what to do.

Just as a reminder `pipeline` is meant to make ML models easy to use, while have *decent* performance (shouldn't be 10x slower than it should). However, complexifying the codebase for a potential 10-20% is not really worth it imo (especially when sending directly a compiled model WILL work, you just need to know when you can and when you cannot, but that' s more of a torch.compile limitation than anything related to pipelines... pipelines also can work with Tensorflow models for instance)

> No you can and should use ` pipeline(model=torch.compile(model)`. If that doesn't work with generate it's up to the caller to know what to do.
> 
> Just as a reminder `pipeline` is meant to make ML models easy to use, while have _decent_ performance (shouldn't be 10x slower than it should). However, complexifying the codebase for a potential 10-20% is not really worth it imo (especially when sending directly a compiled model WILL work, you just need to know when you can and when you cannot, but that' s more of a torch.compile limitation than anything related to pipelines... pipelines also can work with Tensorflow models for instance)

Thanks for the clarification!