Pipeline for Conditional Generation (T5 type models)
Author: enzoampil
Pull request: no

As text-to-text models (like T5) increase the accessibility of multi-task learning, it also makes sense to have a flexible "Conditional Generation" pipeline.

For example, I should be able to use this pipeline for a multitude of tasks depending on how I format the text input (examples in Appendix D of the [T5 paper](https://arxiv.org/pdf/1910.10683.pdf)). As a baseline, this should be able to work on `T5ForConditionalGeneration` and allow for any of the tasks that are learned by the open sourced T5 model.

Since T5 isn't usable for `TextGenerationPipeline`, I propose we add a `ConditionalGenerationPipeline`.

Please do let me know if there is an existing way to perform the above via pipelines, or if adding a pipeline doesn't makes sense for this; otherwise, I can submit a PR for the above `ConditionalGenerationPipeline` 🙂

Comments:
Yes having a "Conditional Generation" pipeline makes sense given that variety of tasks can be solved using it. We can use T5, BART for these tasks as well as the new Encoder-Decoder. I would like to call it `TextToTextPipeline` though, since we can solve non-generative tasks also as demonstrated in the T5 paper. I think this pipeline will be really useful.

Technically, any task using Text-To-Text is generative in nature right? But yeah, agree `TextToTextPipeline` will make the use case clearer :smile:

Hoping to get feedback from @patrickvonplaten  before attempting this

Yeah. To be honest, I'm not sure whether this is a good idea. The pipelines are supposed to be directly related to a task such as `translation`, `summarization` which are specific cases of `text2text` applications. 

I think for every task we should introduce a new `pipeline` before starting to have different levels of abstractions in `pipelines`. A `TextToTextPipeline could become quite a mess regarding different possible input formats, different prefixes (for T5), etc...For general tasks such as these ones I'd prefer to just implement your own code using the `.generate()` function. 

@LysandreJik - what do you think? 

I think from a high level, more than just thinking about `text2text`, I'm foreseeing the future where multi-task learning becomes a standard way of deploying ML models. Having a pipeline to introduce this can be one step to accelerating that future.

Although, I do understand that `text2text` is just one approach to doing this, but in my opinion, it's the most promising one at the moment, so it's a good interface to start with for a multi task model pipeline.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


I'm not sure that T5 is the most promising place to do a multi-task pipeline, since their results in that paper suggested it was hard to significantly beat the baseline of just fine tuning on the target task. 

The recent AdapterHub library built off of HuggingFace seems a better place for building out multitask systems/pipelines imo. But of course the library designers have more intuition on this.

I'm don't think anyone is arguing for the T5 model specifically, just that there is a trend towards `text2text` as a common method of doing multitask learning for NLP (GPT-3 frames tasks like this too for example).

> I'm don't think anyone is arguing for the T5 model specifically, just that there is a trend towards `text2text` as a common method of doing multitask learning for NLP (GPT-3 frames tasks like this too for example).

Fair enough. I'm not one to argue against a feature, even if I wouldn't use it much myself. I've been using `text2text` myself for multiple tasks. 

Mostly I just meant the multitask part of `text2text` is going to be a little tricky to abstract away conveniently into a pipeline. The main complexity there is mixing the proportion of each task / batch correctly. The T5 paper suggests performance and weights are very specific to the multitask learning, and if its not tuned properly the performance will be hurt by using multitasks. Uniform mixing for example performs quite poorly. I suspect that problem would apply to most `text2text` paradigms.

 What I've been doing myself is using a custom DataLoader class that handles the mixing of batch proportions of each task. A pipeline that can integrate something like that would be terrific to have.

Hey everybody, after thinking a bit more about it, I think it does make sense to add a `ConditionalTextGeneration` pipeline which will be the equivalent of `TextGenerationPipeline` for all models in `AutoModelForSeq2Seq`. It should look very similar to the `TextGenerationPipeline` (probably we more or less the same at the moment), but it will give us more freedom in the future (for example when we add `decoder_input_ids` to the generation). 
@sshleifer , @yjernite , @LysandreJik - what are your thoughts on this?

@patrickvonplaten happy to work on a PR for this if team agrees it makes sense :smile:

I think we definitely need something like that.

I'd probably go with a more explicit name though: e.g. `TextToTextPipeline` or `Text2TextGenerationPipeline`. `ConditionalTextGeneration` might cover other uses in the future (e.g. multiple input texts or multimodal inputs)

Such a pipeline would be very welcome, indeed!

Awesome, will send a PR in the next week or so :smile:

I also want to work on this, @enzoampil  let me know if you want to collaborate on the PR :)

Sure thing, maybe we can collab on the same fork? :)