Add type hints for PyTorch Models
Author: karthikrangasai
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

Add type hints to as many PyTorch models as possible.

This PR targets the following models to type hint entire files:
- Albert
- Bart
- Bert
- BertGeneration
- BigBird
- BigBirdPegasus
- Canine
- ConvBert
- ConvNext
- CTRL
- Data2VecText
- Data2VecAudio
- Hubert
- Marian
- MBart
- Nystromformer
- Wav2Vec2
- WavLM
- XGLM
- XLMRobertaXL
- Yoso

Any other file that has been edited is a result of running `make fix-copies`.

In the next PR, I will target few other models to type hint complete files.

<!-- Remove if not applicable -->

Fixes #16059 


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->
@Rocketknight1 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_16425). All of your documentation changes will be reflected on that endpoint.

Hello all,

I changed the cokkiecutter template code because the `check_copies` script couldn't correct code that got changed to multiple lines.

Changing a method in Bart model:
```py
##########
## From ##
##########
# Copied from transformers.models.bart.modeling_bart.BartDecoder._prepare_decoder_attention_mask
def _prepare_decoder_attention_mask(self, attention_mask, input_shape, inputs_embeds, past_key_values_length):

########
## To ##
########
# Copied from transformers.models.bart.modeling_bart.BartDecoder._prepare_decoder_attention_mask
def _prepare_decoder_attention_mask(
    self,
    attention_mask: torch.Tensor,
    input_shape: torch.Size,
    inputs_embeds: torch.FloatTensor,
    past_key_values_length: int,
) -> Optional[torch.Tensor]:
```

The code correcter script would make the following change:
```py
# Copied from transformers.models.bart.modeling_bart.BartDecoder._prepare_decoder_attention_mask
def _prepare_decoder_attention_mask(self, attention_mask, input_shape, inputs_embeds, past_key_values_length):
    self,
    attention_mask: torch.Tensor,
    input_shape: torch.Size,
    inputs_embeds: torch.FloatTensor,
    past_key_values_length: int,
) -> Optional[torch.Tensor]:
```

Which caused the errors in [one of the test runs](https://github.com/huggingface/transformers/runs/5709841712?check_suite_focus=true).


Just commenting the reason here in case someone is taking a look at this PR later.

Hello all,

I have updated the code base with type hints for a few models. 
I will open a new PR for the remaining models after this one is merged, since this PR is getting bigger.

Thanks

cc: @Rocketknight1 


Hello all,

The `Add model like runner` test is failing with an `ImportError` when starting to run the `Run all PyTorch modeling test` section of the tests with the following error:

```bash
ImportError: cannot import name 'get_current_traceback' from 'werkzeug.debug.tbtools' (/home/runner/.local/lib/python3.8/site-packages/werkzeug/debug/tbtools.py)
```
I am unsure as to what is causing the error and any leads on how to resolve this issue would be appreciated.

Thank you

Wow, this is a huge PR! Did you do this manually, or have you figured out some kind of tool for it?

Hello @Rocketknight1 ,

Yeah, I made all this manually. This was how I spent my weekend 😛.


That's amazing! I'll try to review now.

This is a huge and very impressive PR, thank you! The main suggestion I have is that bools are not annotated in some cases, e.g. `output_attentions=False` should be `output_attentions: bool = False`, or `output_attentions: Optional[bool] = None` when the default is `None`. I'll try to recruit a couple of people from Huggingface to help me review the whole thing once that's resolved!

Hello, 

Sure, I can also a take a look once again to fix the missing ones.
I had some doubts with a few of the types and I will post them here later to get the types for them and later update.

Thanks for the update and glad you liked the work.

Absolutely! I saw in some cases `past` was missing annotations - if you're unsure about annotations like that, you can usually check the docstrings for the `past` or `past_key_values` argument for that model - it'll be something like `Tuple[torch.Tensor]`

Sure, later in the process I figured out the type and I had added for a few files. Fill fix for others as well.

Note that `past`/`past_key_values` can have different structure in different models!

Ohhh, thanks for the heads up.

@karthikrangasai The best way to make sure the type hints are correct is to check the [Model Name]_INPUTS_DOCSTRING, right before the first user interfaced forward method

Hello @Tegzes ,
I checked that for all forward methods. But it might be possible that I missed it for a few files.

I have type hinted the entire file, from first function to last class. So i might have missed something in other places.

Hi @karthikrangasai ! This is totally my bad - other PRs came in and I reviewed them without realizing they would create conflicts with your one. Would it be possible to break this PR up into a few separate ones and submit them one at a time? That greatly reduces the chances of conflicts for each one, and it'll make it possible for me to add specific comments/suggestions, whereas at this size I really can just give general advice!

Hello @Rocketknight1 ,

Yeah sure.
I would like to completely work on the typing issues if that's fine with you ( for all PyTorch Models - complete file).

I will break the PR into multiple ones based on the corrections made or the model that was type hinted.

Should I close this one then ?

Hi @karthikrangasai, sorry for the delay! Yeah, it's probably easiest to close this one, make new ones and just tag me in them. Thank you!