fix overflow when training mDeberta in fp16
Author: sjrl
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes https://github.com/microsoft/DeBERTa/issues/77 (issue about transformers opened in Microsoft repo)

- This issue was originally raised in the https://github.com/microsoft/DeBERTa repo which had to do with mDeberta not being able to be trained using fp16. A fix for this was implemented in the Microsoft repo by @BigBird01 but did not yet make it to HuggingFace. I was interested in training mDeberta models on small hardware (e.g. a 3070, T4) so I updated the HF implementation with the changes from the Microsoft repo. I tried to only bring over the minimal changes needed to get the fp16 training to work. 

- I checked that existing tests passed and also used this code to successfully train an mDeberta model in fp16 on Squad2 that can be found [here](https://huggingface.co/sjrhuschlee/mdeberta-v3-base-squad2) which is not currently possible with the main branch of transformers. I'm unsure if there is a good way to add an additional test to make sure mDeberta-V3 training works in fp16 in the CI.

## Before submitting
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

Hey, based on the recommendations from the PR template (and git blame) I decided to tag @ArthurZucker and @sgugger in case you may be interested.

Comments:
cc @younesbelkada @ArthurZucker 

_The documentation is not available anymore as the PR was closed or merged._

I used this code block to check results.
This was run on:
- Ubuntu 20.04.4 LTS
- NVIDIA 3070
- CUDA Version: 11.7
```python
import torch
from transformers import AutoTokenizer, AutoModelForQuestionAnswering
from transformers.pipelines import QuestionAnsweringPipeline

tokenizer = AutoTokenizer.from_pretrained("sjrhuschlee/mdeberta-v3-base-squad2")
model = AutoModelForQuestionAnswering.from_pretrained(
    "sjrhuschlee/mdeberta-v3-base-squad2",
#     torch_dtype=torch.float16,
#     torch_dtype=torch.bfloat16,
#     load_in_8bit=True,
)
pipe = QuestionAnsweringPipeline(model, tokenizer, device=torch.device("cuda:0"))  # device=... was removed for 8bit
```

**Running on Main Branch**

Running the above code using `torch.float16` on the main branch gives me no answer
```python
# with torch.float16
pipe = QuestionAnsweringPipeline(model, tokenizer, device=torch.device("cuda:0"))
# []
```
Running with `torch.bfloat16` and `torch.float32` gives me the expected answer
```python
# with torch.bfloat16
pipe = QuestionAnsweringPipeline(model, tokenizer, device=torch.device("cuda:0"))
# {'score': 0.98369300365448, 'start': 33, 'end': 41, 'answer': ' Berlin.'}

# with torch.float32
pipe = QuestionAnsweringPipeline(model, tokenizer, device=torch.device("cuda:0"))
# {'score': 0.9850791096687317, 'start': 33, 'end': 41, 'answer': ' Berlin.'}
```
Also running in `8bit` works
```python
# with load_in_8bit=True
pipe = QuestionAnsweringPipeline(model, tokenizer)
# {'score': 0.9868391752243042, 'start': 33, 'end': 41, 'answer': ' Berlin.'}
```

**Running on the PR**
The change in this PR also enables mDeberta models to run at inference in `torch.float16` which wasn't possible before. And it doesn't look to affect any of the other dtypes.
```python
# with torch.float16
pipe = QuestionAnsweringPipeline(model, tokenizer, device=torch.device("cuda:0"))
# {'score': 0.9848804473876953, 'start': 33, 'end': 41, 'answer': ' Berlin.'}

# with torch.bfloat16
pipe = QuestionAnsweringPipeline(model, tokenizer, device=torch.device("cuda:0"))
# {'score': 0.9841369986534119, 'start': 33, 'end': 41, 'answer': ' Berlin.'}

# with torch.float32
pipe = QuestionAnsweringPipeline(model, tokenizer, device=torch.device("cuda:0"))
# {'score': 0.9850791096687317, 'start': 33, 'end': 41, 'answer': ' Berlin.'}

# with load_in_8bit=True
pipe = QuestionAnsweringPipeline(model, tokenizer)
# {'score': 0.9870386719703674, 'start': 33, 'end': 41, 'answer': ' Berlin.'}
```

I also noticed that the TF implementation in DebertaV2 has the same line https://github.com/huggingface/transformers/blob/2e2088f24b60d8817c74c32a0ac6bb1c5d39544d/src/transformers/models/deberta_v2/modeling_tf_deberta_v2.py#L678-L679
I'm not too familiar with TF though so I'm not sure if this change should be made there as well. 

@sjrl To the best of my knowledge, we don't support training in fp16 in TF, so less of a risk here. I'd be pro updating in TF, so that the implementations are aligned and it's potentially safer. cc @Rocketknight1 for his thoughts. 

Yes, we support mixed-precision float16/bfloat16 training in TensorFlow, but in general we still expect a 'master' copy of the weights to remain in float32. We're planning some exploration to see if we can get Keras to accept full (b)float16 training, but it might require some refactoring!

@Rocketknight1 should I go ahead and update the TF implementation as well then? 

@sjrl Yes please! Better numerical stability will be nice to have once we've enabled full float16 training

@sjrl  - Are there any other changes to add? Otherwise I think we're good to merge :) 

@amyeroberts You're welcome, and that's it for the changes!