Add overloads for PretrainedModel.from_pretrained
Author: ringohoffman
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes #23980

This PR fixes the type hints that users see when calling `PretrainedModel.from_pretrained`

before:
```python
import transformers

bert_model = transformers.BertForSequenceClassification.from_pretrained("...")
reveal_type(bert_model)  # Type of "bert_model" is "tuple[Unknown | BertForSequenceClassification, dict[str, Unbound | Unknown] | dict[str, Unknown | list[Unknown]] | Unknown] | Unknown | BertForSequenceClassification"

bert_model_and_loading_info = transformers.BertForSequenceClassification.from_pretrained("...", output_loading_info=True)
reveal_type(bert_model_and_loading_info)  # Type of "bert_model_and_loading_info" is "tuple[Unknown | BertForSequenceClassification, dict[str, Unbound | Unknown] | dict[str, Unknown | list[Unknown]] | Unknown] | Unknown | BertForSequenceClassification"
```

after:
```python
import transformers

bert_model = transformers.BertForSequenceClassification.from_pretrained("...")
reveal_type(bert_model)  # Type of "bert_model" is "BertForSequenceClassification"

bert_model_and_loading_info = transformers.BertForSequenceClassification.from_pretrained("...", output_loading_info=True)
reveal_type(bert_model_and_loading_info)  # Type of "bert_model_and_loading_info" is "Tuple[BertForSequenceClassification, LoadingInfo]"
```

1. move `output_loading_info` from variadic kwargs to be an explicit kwarg
2. create overloaded signature for `from_pretrained` based on its value
3. add `LoadingInfo` `TypedDict`

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Documentation: @sgugger, @stevhliu and @MKhalusova

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_24035). All of your documentation changes will be reflected on that endpoint.

> you can search the source code, it's not present anywhere

Searching in the source code, you can see it here: https://github.com/search?q=repo%3Ahuggingface%2Ftransformers+%40overload&type=code

Is there a specific part of this `overload` that makes it difficult to merge in?

This overload makes it obvious to users that you only get the `LoadingInfo` when you pass in `output_loading_info=True` and that otherwise they will only get the model they expected.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

@amyeroberts RE: https://github.com/huggingface/transformers/pull/26125#issuecomment-1803725683

> Previously a similar proposal was very strongly rejected (https://github.com/huggingface/transformers/issues/23980#issuecomment-1576991446 and https://github.com/huggingface/transformers/pull/24035)

This PR was rejected with little explanation and without any discussion. I would be so grateful if you would be willing to hear me out.

> I'm generally against introducing overloads.
> 
> However, this does seem to be an issue which is raised by many independent community members and finding an easy to maintain alternative we can agree on would be good. Perhaps we can open an issue to discuss addressing this more generally?

`overload` describes polymorphic functions--that is, functions whose output types depend on their input types. This is a very natural and common pattern, and `transformers` already uses polymorphic functions extensively.

The alternative would be to rewrite these functions as separate, distinct functions for each input/output type combination (not practical for existing functions). You will end up with the same number of signatures as if you just used `overload`, but with the added burden of having to test and maintain these as separate functions, as well as cluttering your namespace with a bunch of variants of the same function.

Of course, I have to mention that the most important and ubiquitous function in all of `transformers`, `PreTrainedModel.from_pretrained` is polymorphic, and is inferred to have a monster return type: 

```python
import transformers

llama = transformers.LlamaForCausalLM.from_pretrained(...)
reveal_type(llama)  # tuple[Unknown | LlamaForCausalLM, dict[str, Unbound | Unknown] | dict[str, Unknown | list[Unknown]] | Unknown] | Unknown | LlamaForCausalLM
```

I'm really not certain on what basis the claim that `overload` signatures bloat the code is made. `PreTrainedModel.from_pretrained` is [1106 lines long](https://github.com/huggingface/transformers/blob/63fb253df0d976b95d9b4b9a7b0012e5f8a37896/src/transformers/modeling_utils.py#L2749-L3855)! My suggestion in this PR is [40 lines long](https://github.com/huggingface/transformers/pull/24035/files#diff-6b72b98c4c2dcfc6cc606843917733f5d858374fbc22a735ff483bbc0c1e63eaR1928-R1959) and spells out exactly how the output type varies with the input, which is currently not at all obvious (see the inferred type above).

[PyTorch makes considerable use of `overload`, many of them having gone years since they were last touched](https://github.com/search?q=repo%3Apytorch%2Fpytorch%20%40overload&type=code). How many more return type variants are planned to be added to `PreTrainedModel.from_pretrained`? In my experience of maintaining codebases that use `overload`, the number of `overload` signatures tends to remain the same over time. As a testament to this, this PR could still be merged in today without changes from what was originally proposed 1 year ago. If you account for the different parameter names, it could have been merged in as is [since v1.0.0, 5 years ago](https://github.com/huggingface/transformers/blob/b33a385091de604afb566155ec03329b84c96926/pytorch_transformers/modeling_utils.py#L469-L473).

Finally, and maybe most importantly, whether or not Hugging Face type checks its code, users are going to continue run type checkers on code that uses Hugging Face code. This is why the community keeps suggesting these changes. Refusal to adopt well-established Python typing features like `overload` is directly harmful to the user experience. Type checking is the tool that powers IDE auto-completion and code navigation, and helps users to reason about code and ensure correctness.

In light all of this, how do you feel about revisiting this PR? I am sure so many users would be delighted to make use of such type hinting at such little cost to Hugging Face.

Thank you for your consideration.

---

For what it's worth, I think your choice of type checker makes a huge difference. I don't use or recommend mypy at all. I use the type checker built into VS Code, [`pyright`](https://pypi.org/project/pyright/). It is a lot faster, has fewer bugs, and fixes issues much, much faster than mypy. As of right now, it has 6 open issues compared to mypy's 2628.

> This PR was rejected with little explanation and without any discussion. I would be so grateful if you would be willing to hear me out.

As commented here - https://github.com/huggingface/transformers/pull/26125#issuecomment-1803725683 - please feel free to open an issue where a solution (not necessarily relating to overloads) can be discussed. 

> However, this does seem to be an issue which is raised by many independent community members and finding an easy to maintain alternative we can agree on would be good. Perhaps we can open an issue to discuss addressing this more generally?

Since #26125 around a year ago, I'm not aware of any issues or PRs opened addressing this. Given the number of issues we have opened daily, it doesn't appear to be something commonly raised or requested. Most of what I see is through github, so it's possible there are active discussions elsewhere e.g. on the forum. If so, please share links as this will help guide the discussion and decisions.

To be clear, we're not saying that this doesn't provide any utility - it definitely does - the question is whether the additional code and its maintenance is worth it. 

> The alternative would be to rewrite these functions as separate, distinct functions for each input/output type combination (not practical for existing functions). 

This is assumping we want to enable this feature!

To provide some context, we previously had a push to increase complicance with type checkers about 2 years ago. This resulted in many issues and users complaining, so we eventually reverted this support #18485.

> I'm really not certain on what basis the claim that overload signatures bloat the code is made. PreTrainedModel.from_pretrained is 1106 lines long! My suggestion in this PR is 40 lines long and spells out exactly how the output type varies with the input, which is currently not at all obvious (see the inferred type above).

As you point out - the modeling file is already huge (something we want to address!). Doing this adds even more code. And, as @Narsil mentions, [overloads can introduce bugs and make it harder to know what code is being called.](https://github.com/huggingface/transformers/pull/26125#issuecomment-1729007149)  

> PyTorch makes considerable use of overload, many of them having gone years since they were last touched. How many more return type variants are planned to be added to PreTrainedModel.from_pretrained? In my experience of maintaining codebases that use overload, the number of overload signatures tends to remain the same over time. As a testament to this, this PR could still be merged in today without changes from what was originally proposed 1 year ago. If you account for the different parameter names, it could have been merged in as is since v1.0.0, 5 years ago.

PyTorch is an amazing, but different library and which has different considerations and engineering choices. For one thing, the transformers Hugging Face team is small! For core logic like this there's really only 2 - 3 of us responsible for maintenance, and that's alongside our other commitments.   

> How many more return type variants are planned to be added to PreTrainedModel.from_pretrained? 

transformers is a very dynamic library - we don't plan these things out in advance! 

> Finally, and maybe most importantly, whether or not Hugging Face type checks its code, users are going to continue run type checkers on code that uses Hugging Face code. This is why the community keeps suggesting these changes. Refusal to adopt well-established Python typing features like overload is directly harmful to the user experience. Type checking is the tool that powers IDE auto-completion and code navigation, and helps users to reason about code and ensure correctness.

You can't build software based on what people are doing with it downstream. People are of course welcome to run type checkers, but it's something we've decided not to support and so make no guarantees about how well it works. 

Python is ultimately not a great language for making typing assumptions. If the library was created and written now, fresh, then having more explicit and full coverage would be ideal. Sadly, that's not the case.

> at such little cost to Hugging Face.

This is a strong assumption.