[`T5`, `MT5`, `UMT5`] Add [T5, MT5, UMT5]ForSequenceClassification
Author: sjrl
Pull request: yes

# What does this PR do?

This adds a sequence classification head to the PyTorch implementation of T5 and MT5, following the pattern of BartForSequenceClassification since it is also an encoder-decoder sequence classification model.

I have trained and uploaded a flan-t5-base for MNLI [here](https://huggingface.co/sjrhuschlee/flan-t5-base-mnli) which has shown promising results on the dataset.

I've updated the model tests to include the new model and I believe I found hopefully most of the additional imports and compatibility with the text-classification and zero-shot classification pipelines.

**NOTE:** 
- [x] Help with failing tests

- I found a number of tests are failing and I have linked it to the fact that `T5ForSequenceClassification` (and also `BartForSequenceClassification`) expect the `input_ids` and `decoder_input_ids` to have the same sequence length which they do not for the T5 tests (shown below) https://github.com/huggingface/transformers/blob/abaca9f9432a84cfaa95531de4c72334f38a42f2/tests/models/t5/test_modeling_t5.py#L104-L106
where `encoder_seq_length != decoder_seq_length`

- Whereas they do have the same sequence length for the `BartModelTest` (shown below) https://github.com/huggingface/transformers/blob/abaca9f9432a84cfaa95531de4c72334f38a42f2/tests/models/bart/test_modeling_bart.py#L126-L133

- Here are the lines of code in `BartForSequenceClassification` (and the `T5` versions) that cause an error when the encoder and decoder sequence lengths are different
https://github.com/huggingface/transformers/blob/caf5e369fc7b4755d9f98568cbe5e36a0898c96c/src/transformers/models/bart/modeling_bart.py#L1546-L1554
The `eos_mask` has the wrong shape to be properly cast onto the `hidden_states` since the `eos_mask` shape is linked to the encoder sequence length and the `hidden_states` shape is linked to the decoder sequence length.

Would it be okay to change the T5 tests such that the decoder and encoder input_ids have the same sequence length to get the tests to pass?


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

Hey @ArthurZucker and @younesbelkada I would greatly appreciate a review on this when you have a chance.


Comments:
_The documentation is not available anymore as the PR was closed or merged._

> Quick comment, we can probably add this to UMT5 too (you don't have to if it take too much time)

For sure, I'd be happy to!

Hey @sgugger thanks for the feedback!

> Hi, thanks for your PR! This does not follow the pattern of BartForSequenceClassification, or any other classification model of the library: 

I agree! I ran into the same problem when implementing `T5ForQuestionAnswering`. I opted to follow the implementation used for `T5ForConditionalGeneration` which also does not use the `BaseModel` and instead reimplements the encoder and decoder. 

I can go ahead and try and use the BaseModel for the SequenceClassification model, but its probably worth doing a refactor of T5 to use the BaseModel for the other models as well (e.g. ConditionalGeneration and QuestionAnswering). What do you think? 

We can't change existing models without risking massive breaking changes (users wouldn't be able to re-use their checkpoints directly, I think `from_pretrained` would still work though). But that doesn't mean we shouldn't do the right thing for new models! So if you could try it the usual way, that would be great!

And thanks for the feedback! 

> # What does this PR do?
> This adds a sequence classification head to the PyTorch implementation of T5 and MT5, following the pattern of BartForSequenceClassification since it is also an encoder-decoder sequence classification model.
> 
> I have trained and uploaded a flan-t5-base for MNLI [here](https://huggingface.co/sjrhuschlee/flan-t5-base-mnli) which has shown promising results on the dataset.
> 
> I've updated the model tests to include the new model and I believe I found hopefully most of the additional imports and compatibility with the text-classification and zero-shot classification pipelines.
> 
> **NOTE:**
> 
> * [x]  Help with failing tests
> * I found a number of tests are failing and I have linked it to the fact that `T5ForSequenceClassification` (and also `BartForSequenceClassification`) expect the `input_ids` and `decoder_input_ids` to have the same sequence length which they do not for the T5 tests (shown below) https://github.com/huggingface/transformers/blob/abaca9f9432a84cfaa95531de4c72334f38a42f2/tests/models/t5/test_modeling_t5.py#L104-L106
>   
>   where `encoder_seq_length != decoder_seq_length`
> * Whereas they do have the same sequence length for the `BartModelTest` (shown below) https://github.com/huggingface/transformers/blob/abaca9f9432a84cfaa95531de4c72334f38a42f2/tests/models/bart/test_modeling_bart.py#L126-L133
> * Here are the lines of code in `BartForSequenceClassification` (and the `T5` versions) that cause an error when the encoder and decoder sequence lengths are different
>   https://github.com/huggingface/transformers/blob/caf5e369fc7b4755d9f98568cbe5e36a0898c96c/src/transformers/models/bart/modeling_bart.py#L1546-L1554
>   
>   The `eos_mask` has the wrong shape to be properly cast onto the `hidden_states` since the `eos_mask` shape is linked to the encoder sequence length and the `hidden_states` shape is linked to the decoder sequence length.
> 
> Would it be okay to change the T5 tests such that the decoder and encoder input_ids have the same sequence length to get the tests to pass?
> 
> ## Before submitting
> * [ ]  This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
> * [x]  Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
>   Pull Request section?
> * [ ]  Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
>   to it if that's the case.
> * [x]  Did you make sure to update the documentation with your changes? Here are the
>   [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
>   [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
> * [x]  Did you write any new necessary tests?
> 
> ## Who can review?
> Anyone in the community is free to review the PR once the tests have passed. Feel free to tag members/contributors who may be interested in your PR.
> 
> Hey @ArthurZucker and @younesbelkada I would greatly appreciate a review on this when you have a chance.

Thanks for your work! Could you please give an example on how to use T5ForSequenceClassification on sst-2/sst-5 dataset,especially how to use the tokenizer? I have try but cannot make it. Thanks!

@sjrl Thanks for your work! Could you please give an example on how to use T5ForSequenceClassification on sst-2/sst-5 dataset,especially how to use the tokenizer? I have try but cannot make it. Thanks!

Hi @dongdongzhaoUP, 

This is a question best placed in our [forums](https://discuss.huggingface.co/). We try to reserve the github issues for feature requests and bug reports.