T5 models
Author: peter-sk
Pull request: yes
Labels: name: Model on the Hub

# What does this PR do?

Add models for queston answering, sequence classification, and token classification with T5

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [X] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [X] Did you write any new necessary tests?


## Who can review?

@ArthurZucker
@younesbelkada


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_23821). All of your documentation changes will be reflected on that endpoint.

Hello. Could you update the PR name and description to better reflect the content of the PR? 
Moreover, could you explain the motivation behind adding this to transformers? Only adding it for the encoder seems like specific usecase on your par  that can be overcome buy having your own version of the code. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

@ArthurZucker
Sorry for the late reply. I got caught up in exam corrections, conference travels, and other pre-holiday stuff. The use case for just using the encoder part is the following:

f you want to evaluate or use T5 for Natural Language Understanding (NLU) tasks such as Named Entity Recognition or Extractive Question Answering, there are two possible avenues:
1) Fine tune the model with in-context learning using few-shots prompting, i.e., train T5 to hopefully decode the desired result to a query, typically prefixing the prompt with "task: ". This is how the model is already trained for some tasks.
2) Use either the encoder, the decoder, or both stacks with different heads and fine-tune the weights of these heads.

While avenue 1) is the bread and butter of seq2seq language models, avenue 2) has some distinct advantages, including but not limited to:
a) No need to post process unexpected answers. Seq2seq models that have been fine-tuned with few-shots prompting are prone to produce spurious/non-sensical/unexpected output sequences on some inputs. If the desired output for example is a sequence of word classes, we might expect that a prompt of "word classes:  I like my fish raw." would return the desired output "pronoun[personal] verb[present] pronoun[possessive] noun[singular] adjective[descriptive]". But the output might be "pronouns are words that are used instead of nouns" or "dishwasher[soap] melon[hat]".
b) Integration of the NLU model into larger models. A Grammatical Error Correction (GEC) model a la Grammarly's GECToR might directly use the outputs of an NER model for word classes as parts of its inputs. Having to analyze/parse/sanitize the output sequence puts a stopper to jointly train/fine-tune such models. (This is by the way not a hypothetical use case.)

Now, why would one just want to use the encoder part and not both the encoder and the decoder part? First, for NLU tasks, the encoder should by all means be the part that represents the "understanding" part of the seq2seq model. Second, adding the decoder additionally should not hurt in principle, but it is unlikely to improve the performance significantly, blows up the model size, and provides some interesting implementation challenges.

Regarding the implementation challenges, I have implemented such encoder+decoder for NLU tasks models, but getting them to clear the tests of the transformers library is not trivial, as these models use the concatenation of the encoder's and decoder's output as input to the head (i.e., classification layer etc). An alternative is to just use the decoder outputs, but this feels less meaningful for NLU tasks.

How do we move forward? Would it help if I provided some benchmarks of using (i) encoder part only, (ii) decoder part only, (iii) encoder+decoder part concatenated, and (iv) encoder+decoder but only decoder part as input for the head?

Chers,
Peter

Oh, well. I just saw that there is now a T5ForQuestionAnswering model. I will also review that.

Hey! I think the best idea is to put your modifications on the hub! This would prevent you from having to go through all the hassle of passing the CI, and since it is your specific usage, it makes more sense. I invite you to follow[ this tutorial ](https://huggingface.co/docs/transformers/custom_models). Hope this will fit your usage! 🤗 

I can see that this pull request has been closed and is not updating.

Hi Arthur,

Let’s do that for now. And if it turns out that a particular model is wildly successful, I’ll make a pull request for it.

Cheers,
Peter

From: Arthur ***@***.***>
Date: Friday, 21 July 2023 at 10.49
To: huggingface/transformers ***@***.***>
Cc: Peter Schneider-Kamp ***@***.***>, Author ***@***.***>
Subject: Re: [huggingface/transformers] T5 models (PR #23821)

Hey! I think the best idea is to put your modifications on the hub! This would prevent you from having to go through all the hassle of passing the CI, and since it is your specific usage, it makes more sense. I invite you to follow this tutorial <https://huggingface.co/docs/transformers/custom_models> . Hope this will fit your usage! 🤗

—
Reply to this email directly, view it on GitHub<https://github.com/huggingface/transformers/pull/23821#issuecomment-1645234058>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABPCCTHXFIYBHNQQQYQAW23XRI67ZANCNFSM6AAAAAAYRQKCUI>.
You are receiving this because you authored the thread.Message ID: ***@***.***>


Perfect! 👍🏻 🤗