Repository: pydantic/pydantic

Issue:
mypy error on default_factory=list in pydantic 1.10
### Initial Checks

- [X] I have searched GitHub for a duplicate issue and I'm sure this is something new
- [X] I have searched Google & StackOverflow for a solution and couldn't find anything
- [X] I have read and followed [the docs](https://pydantic-docs.helpmanual.io) and still think this is a bug
- [X] I am confident that the issue is with pydantic  (not my code, or another library in the ecosystem like [FastAPI](https://fastapi.tiangolo.com) or [mypy](https://mypy.readthedocs.io/en/stable))


### Description

mypy complains about default_factory = list in pydantic 1.10

But using [] works fine. I think this inconsistency should be fixed
```
$ mypy bug.py
bug.py:6: error: Incompatible types in assignment (expression has type "List[_T]", variable has type "List[str]")
Found 1 error in 1 file (checked 1 source file)
```

Also, I don't think the documentation indicates that pydantic deep-copies mutable default values. I found that out at https://stackoverflow.com/questions/63793662/how-to-give-a-pydantic-list-field-a-default-value

### Example Code

```Python
from typing import List
from pydantic import BaseModel, Field


class Model(BaseModel):
    l1: List[str] = Field(default_factory=list)
    l2: List[str] = Field([])
    l3: List[str] = []
```


### Python, Pydantic & OS Version

```Text
pydantic version: 1.10.1
            pydantic compiled: True
                 install path: /home/mfulmer/miniconda3/envs/rk310/lib/python3.10/site-packages/pydantic
               python version: 3.10.4 (main, Mar 31 2022, 08:41:55) [GCC 7.5.0]
                     platform: Linux-5.4.0-124-generic-x86_64-with-glibc2.31
     optional deps. installed: ['dotenv', 'email-validator', 'typing-extensions']
```


### Affected Components

- [ ] [Compatibility between releases](https://pydantic-docs.helpmanual.io/changelog/)
- [ ] [Data validation/parsing](https://pydantic-docs.helpmanual.io/usage/models/#basic-model-usage)
- [ ] [Data serialization](https://pydantic-docs.helpmanual.io/usage/exporting_models/) - `.dict()` and `.json()`
- [ ] [JSON Schema](https://pydantic-docs.helpmanual.io/usage/schema/)
- [ ] [Dataclasses](https://pydantic-docs.helpmanual.io/usage/dataclasses/)
- [ ] [Model Config](https://pydantic-docs.helpmanual.io/usage/model_config/)
- [ ] [Field Types](https://pydantic-docs.helpmanual.io/usage/types/) - adding or changing a particular data type
- [ ] [Function validation decorator](https://pydantic-docs.helpmanual.io/usage/validation_decorator/)
- [ ] [Generic Models](https://pydantic-docs.helpmanual.io/usage/models/#generic-models)
- [ ] [Other Model behaviour](https://pydantic-docs.helpmanual.io/usage/models/) - `construct()`, pickling, private attributes, ORM mode
- [ ] [Settings Management](https://pydantic-docs.helpmanual.io/usage/settings/)
- [X] [Plugins](https://pydantic-docs.helpmanual.io/) and integration with other tools - mypy, FastAPI, python-devtools, Hypothesis, VS Code, PyCharm, etc.


Comments:
Same with dict.
```python
parameters: Dict[str, str] = Field(default_factory=dict)
```

error: Incompatible types in assignment (expression has type "Dict[_KT, _VT]", variable has type "Dict[str, str]")
Could be related to #4086, @richardxia any idea how this can be fixed?

---

By the way, @tapple @Pentusha it would be much easier for you and me if problems like this got caught before releasing v1.10.

In future, please consider testing pydantic pre-releases.

We had three pre-releases over 2 weeks and i talked about them a lot on twitter.
> Could be related to #4086, @richardxia any idea how this can be fixed?

I don't think it's related to #4086; that PR should have only affected whether the mypy plugin thinks a given field is required or not. I also tested reverting that commit locally and testing the example code in this ticket, but I still see the same error message show up.

I did a git bisect, and it looks like https://github.com/pydantic/pydantic/commit/460f858cc5bc9de792b2c4f6373913a1456488aa (https://github.com/pydantic/pydantic/pull/3430) is the first commit that fails to type check the example code in this ticket. Maybe @klaa97 could take a look?
@richardxia Thank you for the tag and sorry for this 🤦  I am pretty sure this comes from my commit, here in particular: https://github.com/klaa97/pydantic/blob/13e0a25cc8c3c8ec5a264f6c9635350ea6b71f3e/pydantic/mypy.py#L165 .

In particular, it comes from the `default_factory` mypy check. Mypygives back a CallableType which return is `builtins.list[_T]`, and unfortunately I am not finding an easy way to "block" this behaviour. It works by specifying the type in the factory (for example a function returning `list[int]`, but in this case it fails.

I wish we'd find it sooner :(, but I'm afraid I cannot find much more documentation on how to tackle this  error therefore I'd say reverting the change might be the easiest fix.
Thanks so much for looking into this.

Maybe we could comment out that check for now and ask on the mypy repo if there's a work around?
I completely agree, it looks like a plan! I could open a PR by end of the day to remove the check on `default_factory`, and proceed with the investigation on the `mypy` repo.
Please tag me in the mypy discussion, I'd be interested to see what they say.

Patch:
diff --git a/pydantic/mypy.py b/pydantic/mypy.py
--- a/pydantic/mypy.py
+++ b/pydantic/mypy.py
@@ -82,7 +82,8 @@ def parse_mypy_version(version: str) -> Tuple[int, ...]:
     return tuple(int(part) for part in version.split('+', 1)[0].split('.'))
 
 
-BUILTINS_NAME = 'builtins' if parse_mypy_version(mypy_version) >= (0, 930) else '__builtins__'
+MYPY_VERSION_TUPLE = parse_mypy_version(mypy_version)
+BUILTINS_NAME = 'builtins' if MYPY_VERSION_TUPLE >= (0, 930) else '__builtins__'
 
 
 def plugin(version: str) -> 'TypingType[Plugin]':
@@ -162,14 +163,22 @@ def _pydantic_field_callback(self, ctx: FunctionContext) -> 'Type':
             # Functions which use `ParamSpec` can be overloaded, exposing the callable's types as a parameter
             # Pydantic calls the default factory without any argument, so we retrieve the first item
             if isinstance(default_factory_type, Overloaded):
-                if float(mypy_version) > 0.910:
+                if MYPY_VERSION_TUPLE > (0, 910):
                     default_factory_type = default_factory_type.items[0]
                 else:
                     # Mypy0.910 exposes the items of overloaded types in a function
                     default_factory_type = default_factory_type.items()[0]  # type: ignore[operator]
 
             if isinstance(default_factory_type, CallableType):
-                return default_factory_type.ret_type
+                ret_type = default_factory_type.ret_type
+                # mypy doesn't think `ret_type` has `args`, you'd think mypy should know,
+                # add this check in case it varies by version
+                args = getattr(ret_type, 'args', None)
+                if args:
+                    if all(isinstance(arg, TypeVarType) for arg in args):
+                        # Looks like the default factory is a type like `list` or `dict`, replace all args with `Any`
+                        ret_type.args = tuple(default_any_type for _ in args)  # type: ignore[attr-defined]
+                return ret_type
 
         return default_any_type
 


Test patch:
diff --git a/tests/mypy/modules/plugin_default_factory.py b/tests/mypy/modules/plugin_default_factory.py
new file mode 100644
--- /dev/null
+++ b/tests/mypy/modules/plugin_default_factory.py
@@ -0,0 +1,21 @@
+"""
+See https://github.com/pydantic/pydantic/issues/4457
+"""
+
+from typing import Dict, List
+
+from pydantic import BaseModel, Field
+
+
+def new_list() -> List[int]:
+    return []
+
+
+class Model(BaseModel):
+    l1: List[str] = Field(default_factory=list)
+    l2: List[int] = Field(default_factory=new_list)
+    l3: List[str] = Field(default_factory=lambda: list())
+    l4: Dict[str, str] = Field(default_factory=dict)
+    l5: int = Field(default_factory=lambda: 123)
+    l6_error: List[str] = Field(default_factory=new_list)
+    l7_error: int = Field(default_factory=list)
diff --git a/tests/mypy/outputs/plugin-fail-strict.txt b/tests/mypy/outputs/plugin-fail-strict.txt
--- a/tests/mypy/outputs/plugin-fail-strict.txt
+++ b/tests/mypy/outputs/plugin-fail-strict.txt
@@ -36,9 +36,8 @@
 219: error: Property "y" defined in "FrozenModel" is read-only  [misc]
 240: error: Incompatible types in assignment (expression has type "None", variable has type "int")  [assignment]
 241: error: Incompatible types in assignment (expression has type "None", variable has type "int")  [assignment]
-244: error: Incompatible types in assignment (expression has type "Set[_T]", variable has type "str")  [assignment]
+244: error: Incompatible types in assignment (expression has type "Set[Any]", variable has type "str")  [assignment]
 245: error: Incompatible types in assignment (expression has type "str", variable has type "int")  [assignment]
-246: error: Incompatible types in assignment (expression has type "List[_T]", variable has type "List[int]")  [assignment]
 247: error: Argument "default_factory" to "Field" has incompatible type "int"; expected "Optional[Callable[[], Any]]"  [arg-type]
 250: error: Field default and default_factory cannot be specified together  [pydantic-field]
 260: error: Missing positional argument "self" in call to "instance_method" of "ModelWithAnnotatedValidator"  [call-arg]
diff --git a/tests/mypy/outputs/plugin-fail.txt b/tests/mypy/outputs/plugin-fail.txt
--- a/tests/mypy/outputs/plugin-fail.txt
+++ b/tests/mypy/outputs/plugin-fail.txt
@@ -25,9 +25,8 @@
 219: error: Property "y" defined in "FrozenModel" is read-only  [misc]
 240: error: Incompatible types in assignment (expression has type "None", variable has type "int")  [assignment]
 241: error: Incompatible types in assignment (expression has type "None", variable has type "int")  [assignment]
-244: error: Incompatible types in assignment (expression has type "Set[_T]", variable has type "str")  [assignment]
+244: error: Incompatible types in assignment (expression has type "Set[Any]", variable has type "str")  [assignment]
 245: error: Incompatible types in assignment (expression has type "str", variable has type "int")  [assignment]
-246: error: Incompatible types in assignment (expression has type "List[_T]", variable has type "List[int]")  [assignment]
 247: error: Argument "default_factory" to "Field" has incompatible type "int"; expected "Optional[Callable[[], Any]]"  [arg-type]
 250: error: Field default and default_factory cannot be specified together  [pydantic-field]
 260: error: Missing positional argument "self" in call to "instance_method" of "ModelWithAnnotatedValidator"  [call-arg]
diff --git a/tests/mypy/outputs/plugin_default_factory.txt b/tests/mypy/outputs/plugin_default_factory.txt
new file mode 100644
--- /dev/null
+++ b/tests/mypy/outputs/plugin_default_factory.txt
@@ -0,0 +1,2 @@
+20: error: Incompatible types in assignment (expression has type "List[int]", variable has type "List[str]")  [assignment]
+21: error: Incompatible types in assignment (expression has type "List[Any]", variable has type "int")  [assignment]
diff --git a/tests/mypy/test_mypy.py b/tests/mypy/test_mypy.py
--- a/tests/mypy/test_mypy.py
+++ b/tests/mypy/test_mypy.py
@@ -49,6 +49,7 @@
     ('pyproject-plugin-strict.toml', 'plugin_fail.py', 'plugin-fail-strict.txt'),
     ('pyproject-plugin-strict.toml', 'fail_defaults.py', 'fail_defaults.txt'),
     ('mypy-plugin-strict.ini', 'settings_config.py', None),
+    ('mypy-plugin-strict.ini', 'plugin_default_factory.py', 'plugin_default_factory.txt'),
 ]
 executable_modules = list({fname[:-3] for _, fname, out_fname in cases if out_fname is None})