Merge type hints from microsoft/python-type-stubs
Author: Avasam
Pull request: yes

# What does this PR do?

Merge type definitions from https://github.com/microsoft/python-type-stubs/tree/main/transformers-stubs so it can be removed from Pylance.
This is also work towards #16059 
I cross-checked the types with what I got at runtime.
I also ran `pyright --pythonversion=3.7` on both files to sanity check I'm not writing anything that will obviously break at runtime under Python 3.7

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section? (yes but make commands are not working on my machine)
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case. (no)
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation). (I guess? type hints are docs, docstrings and external docs should stay the same from my PR)
- [ ] Did you write any new necessary tests? (no, if you test using mypy/pyright this should already be picked up. Unit tests should naturally break if using syntax or imports incompatible with 3.7)


## Who can review?

🤷 The list below doesn't mention typing / type hints
I guess @Rocketknight1 who opened #16059 
<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @sgugger

Integrations:

- deepspeed: HF Trainer: @stas00, Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam

Documentation: @sgugger, @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: @sgugger
- TensorFlow: @Rocketknight1

 -->


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_23093). All of your documentation changes will be reflected on that endpoint.

> the tokenizers imported for Fnet, T5 or Pegasus are wrong

I copied exactly what I obtained from runtime inspection (see image below). `FNetTokenizer`, `PegasusTokenizer`, `T5Tokenizer` and `RagTokenizer` have no common base class with `PreTrainedTokenizer`. Unless there's a `Protocol` I could use (or add) instead, or if you're fine with hiding that these are real potential results, or simplify the `Union` by throwing in a `type` or even an `Any` (with a comment about incomplete type),
![image](https://user-images.githubusercontent.com/1350584/235555329-70e467e6-2071-4bf3-8452-1c43dc13536f.png)

> not interested in creating dependencies on the auto module over all those new modules

Updated to not have runtime dependencies

Python is not a statically typed language and your runtime inspection will be different form another user's runtime inspection depending on the packages installed. Again this is way more headache that what we want to deal with and the benefits of adding type hints, so we won't merge any type hints in the auto module.

Since it's not possible to get accurate and useful inline generic type hints without changing the base class to an alias due to Python 3.8 support: To be reconsidered once python 3.8 support is dropped.

I'll backport this to https://github.com/microsoft/python-type-stubs so they're at least accurate. Which may or may not be migrated to typeshed at some point.