Add TokenClassification for Mistral, Mixtral and Qwen2
Author: josephenguehard
Pull request: yes

# What does this PR do?

This PR adds token classification for Llama and its extensions, where code from Llama was copied : Gemma, Mistral, Mixtral, Persimmon, Qwen2, Qwen2MoE, StableLm and StarCoder2:

- `GemmaForTokenClassification`
- `LlamaForTokenClassification`
- `MistralForTokenClassification`
- `MixtralForTokenClassification`
- `PersimmonForTokenClassification`
- `Qwen2ForTokenClassification`
- `Qwen2MoeForTokenClassification`
- `StableLmForTokenClassification`
- `StarCoder2ForTokenClassification`

Note: The original PR was intended for Mistral only. Llama has been added as a request from @KoichiYasuoka, and the others models have been added as copying most of Llama setting.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

- text models: @ArthurZucker and @younesbelkada

Comments:
Thank you @josephenguehard for 

* `MistralForTokenClassification`

since I've already released https://huggingface.co/KoichiYasuoka/Swallow-MS-7b-upos (using `trust_remote_code=True`). And would you kindly include

* `LlamaForTokenClassification`

from #26521 for my https://huggingface.co/KoichiYasuoka/Swallow-7b-plus-upos ?

Hi Both! 

Thank you for your replies! I will open a separate issue to discuss adding these classes.

@KoichiYasuoka I'd be happy to add `LlamaForTokenClassification`! I'll try to have this done by the end of the week.

The feature request issue linked to this PR is #29940

@KoichiYasuoka I've added Llama support as well as its extensions (models where code from Llama was copied), and I think the PR is ready for review.

One point where I'd like to have a discussion is on the Dropout. I see in [your implementation](https://github.com/huggingface/transformers/issues/26521#issuecomment-1868284434) that you include a Dropout layer, which follows the [BertForTokenClassification](https://github.com/huggingface/transformers/blob/17b06e2c6650de162e7954babf6224c1975c2852/src/transformers/models/bert/modeling_bert.py#L1691).
However, I argue that this implementation actually follows [BertForSequenceClassification](https://github.com/huggingface/transformers/blob/17b06e2c6650de162e7954babf6224c1975c2852/src/transformers/models/bert/modeling_bert.py#L1494), which also has a Dropout layer. Moreover, [LlamaForSequenceClassification](https://github.com/huggingface/transformers/blob/17b06e2c6650de162e7954babf6224c1975c2852/src/transformers/models/llama/modeling_llama.py#L1343) does not have such Dropout layer, so I think we should not include it for `LlamaForTokenClassification` either. But happy to get other's opinions!

Well, Dropout layer... My Implementation follows [GPT2ForTokenClassification](https://github.com/huggingface/transformers/blob/v4.10.0/src/transformers/models/gpt2/modeling_gpt2.py#L1344) which was released with #13290 at Transformers [v4.10.0](https://github.com/huggingface/transformers/releases/tag/v4.10.0). But I'm vague how GPT2 and LLaMA differ at that point...

It's very easy for me to add Dropout if there is a consensus on it. 
FYI I'm away and won't have access to my laptop until April 15th, will pick this up then.

Yesterday @SeanLee97 of @WhereIsAI released two models:

* https://huggingface.co/WhereIsAI/billm-llama-7b-conll03-ner
* https://huggingface.co/WhereIsAI/billm-mistral-7b-conll03-ner

They use their own package [BiLLM](https://pypi.org/project/BiLLM/) with [LlamaForTokenClassification](https://github.com/WhereIsAI/BiLLM/blob/main/src/billm/modeling_llama.py#L230) and [MistralForTokenClassification](https://github.com/WhereIsAI/BiLLM/blob/main/src/billm/modeling_mistral.py#L204).

> Yesterday @SeanLee97 of @WhereIsAI released two models:
> 
> * https://huggingface.co/WhereIsAI/billm-llama-7b-conll03-ner
> * https://huggingface.co/WhereIsAI/billm-mistral-7b-conll03-ner
> 
> They use their own package [BiLLM](https://pypi.org/project/BiLLM/) with [LlamaForTokenClassification](https://github.com/WhereIsAI/BiLLM/blob/main/src/billm/modeling_llama.py#L230) and [MistralForTokenClassification](https://github.com/WhereIsAI/BiLLM/blob/main/src/billm/modeling_mistral.py#L204).

Thank you for mentioning our released models. Happy to see that official 🤗 transformers supports token classification. 
However, based on our experiments in the paper (https://arxiv.org/abs/2310.01208), LLMs with the causal mask cannot perform well on token classification tasks. Thus, we convert the causal attention mask from uni- to bi-directional. This change can improve the token classification performance significantly. Hopefully this feature can be added to the official transformers in the future.

Thank you @SeanLee97 but **bi-directional** seems not causal... IMHO we need `causal` option in LlamaConfig as used in XLMConfig, where `causal=True` (default) means **uni-directional** attention in contrast of `causal=False` bi-directional.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

I'm waiting for your review @ArthurZucker 

also make sure to resolve the conflics

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29878). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@ArthurZucker I've added `Copied from` for the tests and resolved the conflicts.

Done @younesbelkada 👍