Repository: typeddjango/django-stubs

Issue:
Custom managers broken with mypy 1.5.0
# Bug report

## What's wrong

Mypy 1.5.0 was just released and breaks lots of django-stubs tests. Obviously that’s semi-expected since 1.5.0 is outside the `django-stubs[compatible-mypy]` bound of `mypy==1.4.*`. But upon investigating anyway, I think mypy is correct that there’s a django-stubs problem here. The failures began with

- python/mypy#14988

which fixes mypy to understand that metaclass attributes take precedence over attributes defined in regular classes, matching Python’s runtime behavior. Unfortunately, django-stubs was relying on the previous misunderstanding:

https://github.com/typeddjango/django-stubs/blob/325006ccd72a9b838cf841b1c61ba4092b084f99/django-stubs/db/models/base.pyi#L21-L29

https://github.com/typeddjango/django-stubs/blob/325006ccd72a9b838cf841b1c61ba4092b084f99/django-stubs/contrib/contenttypes/models.pyi#L14-L18

mypy 1.4.1 thinks `ContentType.objects` has type `ContentTypeManager`, but mypy 1.5.0 thinks `ContentType.objects` has type `BaseManager[ContentType]` from the metaclass.

This affects real projects too, such as Zulip (after bumping its pinned version of mypy):

```
corporate/models.py:80: error: "BaseManager[ContentType]" has no attribute "get_for_model"  [attr-defined]
zerver/actions/invites.py:458: error: "BaseManager[ContentType]" has no attribute "get_for_model"  [attr-defined]
zerver/actions/invites.py:469: error: "BaseManager[ContentType]" has no attribute "get_for_model"  [attr-defined]
zerver/tests/test_users.py:769: error: "BaseManager[ContentType]" has no attribute "clear_cache"  [attr-defined]
zerver/tests/test_signup.py:929: error: "BaseManager[ContentType]" has no attribute "clear_cache"  [attr-defined]
corporate/views/webhook.py:70: error: "BaseManager[ContentType]" has no attribute "get_for_model"  [attr-defined]
corporate/views/webhook.py:85: error: "BaseManager[ContentType]" has no attribute "get_for_model"  [attr-defined]
corporate/views/webhook.py:100: error: "BaseManager[ContentType]" has no attribute "get_for_model"  [attr-defined]
```

## How is that should be

We need to somehow convince mypy 1.5.0 that `ContentType.objects` has type `ContentTypeManager`.

## System information

- OS: Linux (NixOS 23.11 x86-64)
- `python` version: 3.10.2
- `django` version: 4.2.4
- `mypy` version: 1.5.0
- `django-stubs` version: 4.2.3
- `django-stubs-ext` version: 4.2.2



Discussion:

Resolving patch:
diff --git a/mypy_django_plugin/lib/helpers.py b/mypy_django_plugin/lib/helpers.py
--- a/mypy_django_plugin/lib/helpers.py
+++ b/mypy_django_plugin/lib/helpers.py
@@ -379,7 +379,9 @@ def check_types_compatible(
     api.check_subtype(actual_type, expected_type, ctx.context, error_message, "got", "expected")
 
 
-def add_new_sym_for_info(info: TypeInfo, *, name: str, sym_type: MypyType, no_serialize: bool = False) -> None:
+def add_new_sym_for_info(
+    info: TypeInfo, name: str, sym_type: MypyType, *, no_serialize: bool = False, is_classvar: bool = False
+) -> None:
     # type=: type of the variable itself
     var = Var(name=name, type=sym_type)
     # var.info: type of the object variable is bound to
@@ -387,6 +389,7 @@ def add_new_sym_for_info(info: TypeInfo, *, name: str, sym_type: MypyType, no_se
     var._fullname = info.fullname + "." + name
     var.is_initialized_in_class = True
     var.is_inferred = True
+    var.is_classvar = is_classvar
     info.names[name] = SymbolTableNode(MDEF, var, plugin_generated=True, no_serialize=no_serialize)
 
 
diff --git a/mypy_django_plugin/transformers/models.py b/mypy_django_plugin/transformers/models.py
--- a/mypy_django_plugin/transformers/models.py
+++ b/mypy_django_plugin/transformers/models.py
@@ -70,8 +70,13 @@ def create_new_var(self, name: str, typ: MypyType) -> Var:
         var.is_inferred = True
         return var
 
-    def add_new_node_to_model_class(self, name: str, typ: MypyType, no_serialize: bool = False) -> None:
-        helpers.add_new_sym_for_info(self.model_classdef.info, name=name, sym_type=typ, no_serialize=no_serialize)
+    def add_new_node_to_model_class(
+        self, name: str, typ: MypyType, *, no_serialize: bool = False, is_classvar: bool = False
+    ) -> None:
+        # TODO: Rename to signal that it is a `Var` that is added..
+        helpers.add_new_sym_for_info(
+            self.model_classdef.info, name=name, sym_type=typ, no_serialize=no_serialize, is_classvar=is_classvar
+        )
 
     def add_new_class_for_current_module(self, name: str, bases: List[Instance]) -> TypeInfo:
         current_module = self.api.modules[self.model_classdef.info.module_name]
@@ -311,7 +316,7 @@ def reparametrize_dynamically_created_manager(self, manager_name: str, manager_i
         assert manager_info is not None
         # Reparameterize dynamically created manager with model type
         manager_type = Instance(manager_info, [Instance(self.model_classdef.info, [])])
-        self.add_new_node_to_model_class(manager_name, manager_type)
+        self.add_new_node_to_model_class(manager_name, manager_type, is_classvar=True)
 
     def run_with_model_cls(self, model_cls: Type[Model]) -> None:
         manager_info: Optional[TypeInfo]
@@ -336,7 +341,7 @@ def run_with_model_cls(self, model_cls: Type[Model]) -> None:
                 continue
 
             manager_type = Instance(manager_info, [Instance(self.model_classdef.info, [])])
-            self.add_new_node_to_model_class(manager_name, manager_type)
+            self.add_new_node_to_model_class(manager_name, manager_type, is_classvar=True)
 
         if incomplete_manager_defs:
             if not self.api.final_iteration:
@@ -351,7 +356,9 @@ def run_with_model_cls(self, model_cls: Type[Model]) -> None:
                 # setting _some_ type
                 fallback_manager_info = self.get_or_create_manager_with_any_fallback()
                 self.add_new_node_to_model_class(
-                    manager_name, Instance(fallback_manager_info, [Instance(self.model_classdef.info, [])])
+                    manager_name,
+                    Instance(fallback_manager_info, [Instance(self.model_classdef.info, [])]),
+                    is_classvar=True,
                 )
 
                 # Find expression for e.g. `objects = SomeManager()`
@@ -433,7 +440,7 @@ def run_with_model_cls(self, model_cls: Type[Model]) -> None:
             default_manager_info = generated_manager_info
 
         default_manager = Instance(default_manager_info, [Instance(self.model_classdef.info, [])])
-        self.add_new_node_to_model_class("_default_manager", default_manager)
+        self.add_new_node_to_model_class("_default_manager", default_manager, is_classvar=True)
 
 
 class AddRelatedManagers(ModelClassInitializer):
@@ -623,6 +630,10 @@ def adjust_model_class(cls, ctx: ClassDefContext) -> None:
         ):
             del ctx.cls.info.names["MultipleObjectsReturned"]
 
+        objects = ctx.cls.info.names.get("objects")
+        if objects is not None and isinstance(objects.node, Var) and not objects.plugin_generated:
+            del ctx.cls.info.names["objects"]
+
         return
 
     def get_exception_bases(self, name: str) -> List[Instance]:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -33,7 +33,7 @@ def find_stub_files(name: str) -> List[str]:
 
 # Keep compatible-mypy major.minor version pinned to what we use in CI (requirements.txt)
 extras_require = {
-    "compatible-mypy": ["mypy==1.4.*"],
+    "compatible-mypy": ["mypy==1.5.*"],
 }
 
 setup(


Test patch:
diff --git a/tests/typecheck/managers/test_managers.yml b/tests/typecheck/managers/test_managers.yml
--- a/tests/typecheck/managers/test_managers.yml
+++ b/tests/typecheck/managers/test_managers.yml
@@ -194,7 +194,16 @@
 
 -   case: managers_inherited_from_abstract_classes_multiple_inheritance
     main: |
-        from myapp.models import Child
+        from myapp.models import AbstractBase1, AbstractBase2, Child
+        reveal_type(Child.manager1)
+        reveal_type(Child.restricted)
+        reveal_type(AbstractBase1.manager1)
+        reveal_type(AbstractBase2.restricted)
+    out: |
+        main:2: note: Revealed type is "myapp.models.CustomManager1[myapp.models.Child]"
+        main:3: note: Revealed type is "myapp.models.CustomManager2[myapp.models.Child]"
+        main:4: note: Revealed type is "myapp.models.CustomManager1[myapp.models.AbstractBase1]"
+        main:5: note: Revealed type is "myapp.models.CustomManager2[myapp.models.AbstractBase2]"
     installed_apps:
         - myapp
     files:
@@ -297,12 +306,13 @@
         -   path: myapp/__init__.py
         -   path: myapp/models.py
             content: |
+                from typing import ClassVar
                 from django.db import models
                 class ParentOfMyModel4(models.Model):
                     objects = models.Manager()
 
                 class MyModel4(ParentOfMyModel4):
-                    objects = models.Manager['MyModel4']()
+                    objects: ClassVar[models.Manager["MyModel4"]] = models.Manager["MyModel4"]()
 
 # TODO: make it work someday
 #-   case: inheritance_of_two_models_with_custom_objects_manager
@@ -570,7 +580,7 @@
         -   path: myapp/__init__.py
         -   path: myapp/models.py
             content: |
-                from typing import TypeVar
+                from typing import ClassVar, TypeVar
                 from django.db import models
 
                 T = TypeVar("T", bound="MyModel")
@@ -585,7 +595,7 @@
                     pass
 
                 class MySubModel(MyModel):
-                    objects = MySubManager()
+                    objects: ClassVar[MySubManager["MySubModel"]] = MySubManager()
 
 -   case: subclass_manager_without_type_parameters_disallow_any_generics
     main: |
@@ -598,14 +608,14 @@
         [mypy-myapp.models]
         disallow_any_generics = true
     out: |
-        main:2: note: Revealed type is "myapp.models.MySubManager[myapp.models.MySubModel]"
+        main:2: note: Revealed type is "myapp.models.MySubManager"
         main:3: note: Revealed type is "Any"
         myapp/models:9: error: Missing type parameters for generic type "MyManager"
     files:
         -   path: myapp/__init__.py
         -   path: myapp/models.py
             content: |
-                from typing import TypeVar
+                from typing import ClassVar, TypeVar
                 from django.db import models
 
                 T = TypeVar("T", bound="MyModel")
@@ -620,7 +630,7 @@
                     pass
 
                 class MySubModel(MyModel):
-                    objects = MySubManager()
+                    objects: ClassVar[MySubManager] = MySubManager()
 
 -   case: nested_manager_class_definition
     main: |
diff --git a/tests/typecheck/models/test_abstract.yml b/tests/typecheck/models/test_abstract.yml
--- a/tests/typecheck/models/test_abstract.yml
+++ b/tests/typecheck/models/test_abstract.yml
@@ -51,7 +51,7 @@
     Recursive(parent=Recursive(parent=None))
     Concrete(parent=Concrete(parent=None))
   out: |
-    main:4: error: Unexpected attribute "parent" for model "Recursive"
+    main:4: error: "Type[Recursive]" has no attribute "objects"
     main:5: error: Cannot instantiate abstract class "Recursive" with abstract attributes "DoesNotExist" and "MultipleObjectsReturned"
     main:5: error: Unexpected attribute "parent" for model "Recursive"
   installed_apps:
@@ -209,4 +209,4 @@
 
             def create_animal_generic(klass: Type[T], name: str) -> T:
                 reveal_type(klass)  # N: Revealed type is "Type[T`-1]"
-                return klass.objects.create(name=name)
+                return klass._default_manager.create(name=name)
diff --git a/tests/typecheck/models/test_contrib_models.yml b/tests/typecheck/models/test_contrib_models.yml
--- a/tests/typecheck/models/test_contrib_models.yml
+++ b/tests/typecheck/models/test_contrib_models.yml
@@ -31,9 +31,11 @@
 
 -   case: can_override_abstract_user_manager
     main: |
-        from myapp.models import User
-        reveal_type(User.objects)  # N: Revealed type is "myapp.models.UserManager[myapp.models.User]"
-        reveal_type(User.objects.all())  # N: Revealed type is "django.db.models.query._QuerySet[myapp.models.User, myapp.models.User]"
+        from myapp.models import MyBaseUser, MyUser
+        reveal_type(MyBaseUser.objects)  # N: Revealed type is "myapp.models.MyBaseUserManager[myapp.models.MyBaseUser]"
+        reveal_type(MyBaseUser.objects.all())  # N: Revealed type is "django.db.models.query._QuerySet[myapp.models.MyBaseUser, myapp.models.MyBaseUser]"
+        reveal_type(MyUser.objects)  # N: Revealed type is "myapp.models.MyUserManager"
+        reveal_type(MyUser.objects.all())  # N: Revealed type is "django.db.models.query._QuerySet[myapp.models.MyUser, myapp.models.MyUser]"
     installed_apps:
         - django.contrib.auth
         - myapp
@@ -42,8 +44,15 @@
         - path: myapp/models.py
           content: |
               from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager
-              class UserManager(BaseUserManager["User"]):
+              from django.contrib.auth.models import AbstractUser, UserManager
+              from typing import ClassVar
+              class MyBaseUserManager(BaseUserManager["MyBaseUser"]):
                   ...
 
-              class User(AbstractBaseUser):
-                  objects = UserManager()
+              class MyBaseUser(AbstractBaseUser):
+                  objects = MyBaseUserManager()
+
+              class MyUserManager(UserManager["MyUser"]):
+                  ...
+              class MyUser(AbstractUser):
+                  objects: ClassVar[MyUserManager] = MyUserManager()
diff --git a/tests/typecheck/models/test_meta_options.yml b/tests/typecheck/models/test_meta_options.yml
--- a/tests/typecheck/models/test_meta_options.yml
+++ b/tests/typecheck/models/test_meta_options.yml
@@ -77,10 +77,11 @@
         # Should not raise:
         MyModel(field=1)
         MyModel.objects.create(field=2)
+        AbstractModel._default_manager.create()
 
         # Errors:
         AbstractModel()  # E: Cannot instantiate abstract class "AbstractModel" with abstract attributes "DoesNotExist" and "MultipleObjectsReturned"
-        AbstractModel.objects.create()
+        AbstractModel.objects.create()  # E: "Type[AbstractModel]" has no attribute "objects"
     installed_apps:
         - myapp
     files: