Mypy errors since v4.51.0
Author: jc-louis
Pull request: no
Labels: name: bug

@cyyever this is a direct result of https://github.com/huggingface/transformers/pull/37022 which marked the library as typed.

```py
from typing import reveal_type

from transformers import Trainer, PreTrainedModel

def mypy_errors(trainer: Trainer, model: PreTrainedModel) -> None:
    reveal_type(trainer)
    reveal_type(trainer.train())
    trainer.train()
    model.dequantize()
```

Using `mypy test.py`, I get the following errors:

```
script.py:5: note: Revealed type is "def (*args: Any, **kwargs: Any) -> transformers.utils.dummy_pt_objects.Trainer"
script.py:8: note: Revealed type is "transformers.utils.dummy_pt_objects.Trainer"
script.py:9: error: "Trainer" has no attribute "train"  [attr-defined]
script.py:9: note: Revealed type is "Any"
script.py:10: error: "Trainer" has no attribute "train"  [attr-defined]
script.py:11: error: "PreTrainedModel" has no attribute "dequantize"  [attr-defined]
```

Theses errors are false positives, it seems like there is a config issue, all types are pulled from [dummy_pt_objects.py](https://github.com/huggingface/transformers/blob/main/src/transformers/utils/dummy_pt_objects.py)

---


- `transformers` version: 4.51.0
- Platform: macOS-14.7.3-arm64-arm-64bit
- Python version: 3.12.3


Comments:
Try
```
from transformers.trainer import Trainer    
from transformers.modeling_utils  import PreTrainedModel
```

@cyyever this does indeed fix the issue, thanks!

However, doing `from transformers import Trainer ` is the recommended way [in the docs](https://huggingface.co/docs/transformers/main/en/trainer#trainer), is there any fix planed?

We should add `__all__` to all exported symbols.

@jc-louis I submitted a PR for this issue. Can you install it and check?

@cyyever I confirm it does fix the issue 👍 

Does someone know if mypy lets you ignore errors related to a specific python object ?

Like when I get those errors:

```
error: "PreTrainedModel" has no attribute "generate"  [attr-defined]
error: "PreTrainedModel" has no attribute "eval"  [attr-defined]
error: "PreTrainedModel" not callable  [operator]
```

I would simply like to ignore the errors "attr-defined", "operator" related to "PreTrainedModel"

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

This issue isn't stale, the latest release to date (4.51.3) still contains this bug.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Still a problem with current release (4.52.4). Here is to hoping #37653 will solve it as [was discussed elsewhere](https://github.com/huggingface/transformers/pull/37340#issuecomment-2909725620).

@LysandreJik is there something we can help with for your aforementioned PR?

FTR release 4.53.0 still shows the same problem.

Wait for #37340 to get merged.

My problem isn't solved with 4.54.0: mypy still cannot resolve `tokenizer` on `DonutProcessor`.

I resorted to an explicit type cast as a workaround because I don't have more time to spend on this.

In case this is of any help to future readers:
```python
# Typing shenanigans
tokenizer = cast(XLMRobertaTokenizer | XLMRobertaTokenizerFast, self.processor.tokenizer)  # type: ignore[attr-defined]
```
- Explicitly `typing.cast` to whatever tokenizer the model supports (see the model docs)
- Explicitly disable the missing argument error (`error: "DonutProcessor" has no attribute "tokenizer"  [attr-defined]`)

@bibz For the fix, you have to wait the next release or try the master version.

That's surprising given [the merged commit](https://github.com/huggingface/transformers/commit/61163099f1b95e9f8960f6382bff4aa31db52fc9) is part of the last few releases already: it is marked as belonging to the tags [v4.54.0](https://github.com/huggingface/transformers/releases/tag/v4.54.0)
[v4.54-release](https://github.com/huggingface/transformers/releases/tag/v4.54-release)
[v4.53.2-Ernie-4.5-preview](https://github.com/huggingface/transformers/releases/tag/v4.53.2-Ernie-4.5-preview)

@bibz Sorry, let me check it. Could you give me a minimal reproducible script?

Packages:
- mypy: 1.17.0
- transformers: 4.54.0

Obviously not runnable, but enough to trigger `mypy`.

```python
# example.py
from transformers import DonutProcessor

processor = DonutProcessor.from_pretrained("")
assert processor.tokenizer
```

```shell
% mypy example.py
example.py:4: error: "DonutProcessor" has no attribute "tokenizer"  [attr-defined]
Found 1 error in 1 file (checked 1 source file)
```

[The docs for this processor](https://huggingface.co/docs/transformers/main/en/model_doc/donut#notes) do use the tokenizer directly (as a callable), so I think I am doing the right thing here.

@bibz Unfortunately, `tokenizer` is a dynamic attribute in `ProcessorMixin`, so mypy can't infer it. See line 511 of `transformers/processing_utils.py`

Yes, indeed. I guess I conflated my problem with the fix proposed and accepted here. It started happening with 4.51.0 too but that might be unrelated.
Thanks for the troubleshooting.