feat: add debertav2 fast tokenizer
Author: mingboiz
Pull request: yes

# What does this PR do?

I think the changes made by @alcinos in the current PR #14928 are ready to be merged into main. Please let me know if this is alright, but the PR has gone stale and I'm just helping to move the needle here by collating all the changes so sorry for the one 300 LoC commit - Full credits should go to @alcinos for his great work!

This PR implements a fast tokenizer for DeBERTaV2 and all related models:
- DeBERTav2
- DeBERTav3
- mDeBERTav3

Fixes # (issue)
#11529
#14712 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

There was one initial test `test_right_and_left_truncation` that failed initially on my local when I was running tests on Tokenizers 10.3. https://github.com/huggingface/transformers/blob/39b5d1a63a07d60e496a6bd98c3a60d32e8b9e6d/tests/test_tokenization_common.py#L1521
This test failure wasn’t present in the previous PR when it was still transformers 4.16.0[dev] - I received 'Ignored unknown kwarg option direction' in my logs which appears to be related to https://github.com/huggingface/transformers/pull/15266


Updating my Tokenizers to 11.4 fixed the test locally, and it seems that it passes all tests now - but not super clear to me why this fixes the failing test and it is now able to pass: Should anything be done regarding this issue like introduce some form of backward compatibility to Tokenizers < 11.0?

```
from transformers import DebertaV2Tokenizer, DebertaV2TokenizerFast
tokenizer = DebertaV2Tokenizer('./tests/fixtures/spiece.model')
tokenizer_fast = DebertaV2TokenizerFast('./tests/fixtures/spiece.model')

def test_right_and_left_truncation(tokenizer, sequence="This is a test sequence"):
    truncation_size = 3
    # RIGHT PADDING
    tokenizer.truncation_side = "right"
    encoded_sequence = tokenizer.encode(sequence, add_special_tokens=False)
    sequence_length = len(encoded_sequence)
    truncated_sequence = tokenizer.encode(
        sequence, max_length = sequence_length - truncation_size, truncation=True, add_special_tokens=False
    )
    truncated_sequence_length = len(truncated_sequence)
    print(f"sequence length: {sequence_length}, {truncated_sequence_length + truncation_size}")
    print(f"encoded sequence: {encoded_sequence[:-truncation_size]}, {truncated_sequence}")
    
    # LEFT PADDING
    tokenizer.truncation_side = "left"
    sequence_length = len(encoded_sequence)
    truncated_sequence = tokenizer.encode(
        sequence, max_length=sequence_length - truncation_size, truncation=True, add_special_tokens=False
    )
    truncated_sequence_length = len(truncated_sequence)
    print(f"sequence length: {sequence_length}, {truncated_sequence_length + truncation_size}")
    print(f"encoded sequence: {encoded_sequence[truncation_size:]}, {truncated_sequence}")
```
In Tokenizers 11.4:
```
>>> test_right_and_left_truncation(tokenizer)
sequence length: 7, 7
encoded sequence: [13, 1, 4398, 25], [13, 1, 4398, 25]
sequence length: 7, 7
encoded sequence: [25, 21, 1289, 4030], [25, 21, 1289, 4030]

>>> test_right_and_left_truncation(tokenizer_fast)
sequence length: 7, 7
encoded sequence: [13, 1, 4398, 25], [13, 1, 4398, 25]
sequence length: 7, 7
encoded sequence: [25, 21, 1289, 4030], [25, 21, 1289, 4030]
```
In Tokenizers 10.3:
```
>>> test_right_and_left_truncation(tokenizer)
sequence length: 7, 7
encoded sequence: [13, 1, 4398, 25], [13, 1, 4398, 25]
sequence length: 7, 7
encoded sequence: [25, 21, 1289, 4030], [25, 21, 1289, 4030]

>>> test_right_and_left_truncation(tokenizer_fast)
Ignored unknown kwarg option direction
sequence length: 7, 7
encoded sequence: [13, 1, 4398, 25], [13, 1, 4398, 25]
Ignored unknown kwarg option direction
sequence length: 7, 7
encoded sequence: [25, 21, 1289, 4030], [13, 1, 4398, 25]
```


## Who can review?
@SaulLu @stefan-it

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_15529). All of your documentation changes will be reflected on that endpoint.

Thank you so much for taking the relay on this PR!

It seems to me that [this point](https://github.com/huggingface/transformers/pull/14928#discussion_r775445739) of the PR was the last point that remained to be addressed. What do you think?

Also, I see that you no longer have the commits from @alcinos's original PR. Is there any particular reason? Otherwise, I think it would be nice to at least put all the co-authors of the PR back in the commit message. If it's easier, I could do it in the merge message of this PR on master when it's ready.

About the error, you raise a good point we should not allow versions of tokenizers before v.11 (at least) because they did not allow truncate left. So the error is not related to this PR.  :blush: 

> Thank you so much for taking the relay on this PR!
> 
> It seems to me that [this point](https://github.com/huggingface/transformers/pull/14928#discussion_r775445739) of the PR was the last point that remained to be addressed. What do you think?
> 
> Also, I see that you no longer have the commits from @alcinos's original PR. Is there any particular reason? Otherwise, I think it would be nice to at least put all the co-authors of the PR back in the commit message. If it's easier, I could do it in the merge message of this PR on master when it's ready.
> 
> About the error, you raise a good point we should not allow versions of tokenizers before v.11 (at least) because they did not allow truncate left. So the error is not related to this PR. 😊

@SaulLu No I agree with you, it wasn't my intention to create a new PR for this - but I've tried and couldn't push to the original branch in the original PR created... so I just ended up pushing to my own repo instead, is there a way for me to push to the original PR without write access to @alcinos repo? Or I can modify my commit message - do I need to follow any convention for the commit message when I add the co-authors?

And I've missed that point in the original PR, can I just confirm there needs to be:
- an additional test to check the behavior of backend tokenizer and fast tokenizer for all combinations of arguments (`do_lower_case`, `split_by_punct`)
- If behaviors are differing: for example if the spilt_by_punct behavior differs, then modify the behavior of the Slow Tokenizer to match that of the Fast Tokenizer

Thanks a lot for your kindness @mingboiz ! 

Concerning the mention of co-authors, don't worry too much about it, I'll take care of it when I merge the PR on main :slightly_smiling_face:  - but indeed it is necessary to follow a template cf the corresponding documentation here.

Concerning the `do_lower_case` and `split_by_punct` arguments, you are right, I also think that the best thing to do is to add a test first! However, this test will most probably fail and you will have to add some code in the `__init__` method of `DebertaV2TokenizerFast` (as explained briefly [here](https://github.com/huggingface/transformers/pull/14928#discussion_r777623922)). 

Then, if behaviors are differing between the slow and the fast version, the tokenizer we need to modify is the fast one :relaxed: .

Don't hesitate to ping me if you have any difficulty, these last 2 points are not the easiest to handle.


Hi @SaulLu, CI has been failing after the addition of `split_by_punct` and `do_lower_case` arguments and tests. The failed jobs seem unrelated to the PR, requesting your help in pointing out why the CI is failing and how I can fix it, thank you!

Thanks for your last additions. 

Regarding the failing CI, I think I have a little idea where it might have come from. Do you remember the last time you rebased your branch on master? If it's been a long time, I think that the quicker fix might be to rebase your branch on master: :relaxed: 

Hey @mingboiz and @SaulLu! What is the status of this PR? Do you plan to merge this? The tests seem to pass on the CI.

Thanks for the reminder @bogdankostic ! 

Indeed, it would be nice if we could merge @mingboiz and @alcinos' amazing work! We'd just have to do a new PR review with the latest changes - I think the tests for `split_by_punct` and `do_lower_case` arguments have been added. To review this PR, it would be really great if we could rebase this branch on main (or merge main into this branch).  Let me know if you have any difficulty to do that @mingboiz (I've tried to do it for you [here](https://github.com/SaulLu/transformers/tree/deberta-v2-fast-tokenizer) but I can't open a PR to your fork).

_The documentation is not available anymore as the PR was closed or merged._

Sorry for the lack of updates @bogdankostic it has been hectic for me recently (quarterly releases 🥲 ) Thank you for catching it @nbroad1881, I wasn't as careful with my tests as I would have liked - maybe would you and @SaulLu let me know what you think of the new change and this PR? 😄 

FYI, I just used this PR to train `DebertaV2ForTokenClassification` via `microsoft/deberta-v3-base` on the `conll2003` dataset (didn't play with the training arguments, just a simple test). Previously, this wasn't possible due to a missing fast tokenizer (maybe it was, but my script is written around this feature).

Here are the results and they look great:

Dev
```
{'eval_loss': 0.039761029183864594,
 'eval_precision': 0.9413465012646032,
 'eval_recall': 0.954218044194848,
 'eval_f1': 0.9477385716017946,
 'eval_accuracy': 0.9896374761497467,
 'eval_runtime': 8.4867,
 'eval_samples_per_second': 382.952,
 'eval_steps_per_second': 3.064,
 'epoch': 5.0}
```

Test
```
{'LOC': {'precision': 0.9063653136531366,
  'recall': 0.9317211948790897,
  'f1': 0.9188683656768764,
  'number': 2109},
 'MISC': {'precision': 0.7502590673575129,
  'recall': 0.7479338842975206,
  'f1': 0.7490946714950854,
  'number': 968},
 'ORG': {'precision': 0.8810440735986307,
  'recall': 0.9216651745747538,
  'f1': 0.9008969590899146,
  'number': 2234},
 'PER': {'precision': 0.9716033202271734,
  'recall': 0.9590340664079344,
  'f1': 0.9652777777777778,
  'number': 2319},
 'overall_precision': 0.8985694032736178,
 'overall_recall': 0.9137614678899083,
 'overall_f1': 0.9061017609981157,
 'overall_accuracy': 0.9797932835020207}
```

Thanks a lot! @mingboiz 🙏