Repository: samuelcolvin/pydantic (python)

Issue:
underscore_attrs_are_private causes TypeError
### Checks

- [x] I added a descriptive title to this issue
- [x] I have searched (google, github) for similar issues and couldn't find anything
- [x] I have read and followed [the docs](https://pydantic-docs.helpmanual.io/) and still think this is a bug

# Bug

Output of `python -c "import pydantic.utils; print(pydantic.utils.version_info())"`:

```
        pydantic version: 1.7
       pydantic compiled: True
            install path: /Users/ahedges/.pyenv/versions/3.7.8/envs/sdf/lib/python3.7/site-packages/pydantic
          python version: 3.7.8 (default, Sep 16 2020, 18:33:23)  [Clang 11.0.3 (clang-1103.0.32.59)]
                platform: Darwin-19.6.0-x86_64-i386-64bit
optional deps. installed: ['typing-extensions']
```

I spent a decent amount of time this weekend trying to make a private field using code posted in #655. I was happy to see Pydantic 1.7 came out today and had support for private fields built in. I upgraded and tried to convert my code, but I encountered some unusual problems. (Even though it doesn't work perfectly, I still appreciate the feature.) Most are type errors from mypy (might report later), but this one is more serious. I simplified the problem below.

The issue is that `underscore_attrs_are_private` causes an exception where `PrivateAttr` does not.

When using `underscore_attrs_are_private` with the following code:

```python
from typing import Any
from pydantic import BaseModel

class TestObject(BaseModel):
    public_field: str
    _private_field: str

    class Config:
        underscore_attrs_are_private = True

    def __init__(self, **data: Any) -> None:
        super().__init__(**data)
        self._private_field = "bar"

print(TestObject(public_field="foo"))
```

I get the following output:

```
test.py:4: DeprecationWarning: __class__ not set defining 'TestObject' as <class '__main__.TestObject'>. Was __classcell__ propagated to type.__new__?
  class TestObject(BaseModel):
Traceback (most recent call last):
  File "test.py", line 15, in <module>
    print(TestObject(public_field="foo"))
  File "test.py", line 12, in __init__
    super().__init__(**data)
  File "pydantic/main.py", line 365, in pydantic.main.BaseModel.__init__
  File "pydantic/main.py", line 424, in pydantic.main.BaseModel._init_private_attributes
  File "pydantic/fields.py", line 821, in pydantic.fields.PrivateAttr.get_default
  File "pydantic/utils.py", line 624, in pydantic.utils.smart_deepcopy
  File "/Users/ahedges/.pyenv/versions/3.7.8/lib/python3.7/copy.py", line 169, in deepcopy
    rv = reductor(4)
TypeError: can't pickle cell objects
```

However, when using `PrivateAttr` with the following code:

```python
from typing import Any
from pydantic import BaseModel, PrivateAttr

class TestObject(BaseModel):
    public_field: str
    _private_field: str = PrivateAttr()

    def __init__(self, **data: Any) -> None:
        super().__init__(**data)
        self._private_field = "bar"

print(TestObject(public_field="foo"))
```

I get the following, desired output:

```
public_field='foo'
```

I also noticed that removing `__init__()` from the first example also prevents the crash. However, it is needed to set the private field.

Another thing to note is that in my full code, the exception caused by `underscore_attrs_are_private` appears but the `DeprecationWarning` does not. If you think this matters, I can try to reproduce my code without the warning.


Pull request:
underscore_attrs_are_private causing TypeError
## Change Summary

fix `underscore_attrs_are_private` causing `TypeError` when overriding `__init__`

## Related issue number

fix #2047

## Checklist

* [x] Unit tests for the changes exist
* [x] Tests pass on CI and coverage remains at 100%
* [x] `changes/<pull request or issue id>-<github username>.md` file added describing change


Patch:
diff --git a/Makefile b/Makefile
index 8aa686f4b..cc5d66bcb 100644
--- a/Makefile
+++ b/Makefile
@@ -134,5 +134,5 @@ docs-serve:
 .PHONY: publish-docs
 publish-docs:
 	zip -r site.zip site
-	@curl -f -H "Content-Type: application/zip" -H "Authorization: Bearer ${NETLIFY}" \
+	@curl -H "Content-Type: application/zip" -H "Authorization: Bearer ${NETLIFY}" \
 	      --data-binary "@site.zip" https://api.netlify.com/api/v1/sites/pydantic-docs.netlify.com/deploys
diff --git a/changes/2047-samuelcolvin.md b/changes/2047-samuelcolvin.md
new file mode 100644
index 000000000..6c4c31cf3
--- /dev/null
+++ b/changes/2047-samuelcolvin.md
@@ -0,0 +1,1 @@
+fix `underscore_attrs_are_private` causing `TypeError` when overriding `__init__`
diff --git a/changes/2048-aphedges.md b/changes/2048-aphedges.md
new file mode 100644
index 000000000..0407f2367
--- /dev/null
+++ b/changes/2048-aphedges.md
@@ -0,0 +1,1 @@
+Fix mypy assignment error when using `PrivateAttr`
diff --git a/docs/datamodel_code_generator.md b/docs/datamodel_code_generator.md
index d96a11c28..dec9cbcb5 100644
--- a/docs/datamodel_code_generator.md
+++ b/docs/datamodel_code_generator.md
@@ -1,18 +1,18 @@
-[datamodel-code-generator](https://github.com/koxudaxi/datamodel-code-generator/) is a command to generate pydantic models from other data types.
+The [datamodel-code-generator](https://github.com/koxudaxi/datamodel-code-generator/) project is a library and command-line utility to generate pydantic models from just about any data source, including:
 
+* OpenAPI 3 (YAML/JSON)
+* JSON Schema
+* JSON/YAML Data (which will converted to JSON Schema)
 
-* Supported source types
-    * OpenAPI 3 (YAML/JSON)
-    * JSON Schema
-    * JSON/YAML Data (It will be converted to JSON Schema)
+Whenever you find yourself with any data convertible JSON but without pydantic models, this tool will allow you to generate type-safe model hierarchies on demand.
 
-## Install
+## Installation
 ```bash
 pip install datamodel-code-generator
 ```
 
 ## Example
-In this case, The datamodel-code-generator creates pydantic models from JSON Schema.
+In this case, datamodel-code-generator creates pydantic models from a JSON Schema file.
 ```bash
 datamodel-codegen  --input person.json --input-file-type jsonschema --output model.py
 ```
diff --git a/pydantic/fields.py b/pydantic/fields.py
index c451770c0..36ea690a2 100644
--- a/pydantic/fields.py
+++ b/pydantic/fields.py
@@ -799,21 +799,10 @@ class ModelField(Representation):
         return args
 
 
-class PrivateAttr(Representation):
-    """
-    Indicates that attribute is only used internally and never mixed with regular fields.
-
-    Types or values of private attrs are not checked by pydantic and it's up to you to keep them relevant.
-
-    Private attrs are stored in model __slots__.
-    """
-
+class ModelPrivateAttr(Representation):
     __slots__ = ('default', 'default_factory')
 
     def __init__(self, default: Any = Undefined, *, default_factory: Optional[NoArgAnyCallable] = None) -> None:
-        if default is not Undefined and default_factory is not None:
-            raise TypeError('default and default_factory args can not be used together')
-
         self.default = default
         self.default_factory = default_factory
 
@@ -825,3 +814,28 @@ class PrivateAttr(Representation):
             other.default,
             other.default_factory,
         )
+
+
+def PrivateAttr(
+    default: Any = Undefined,
+    *,
+    default_factory: Optional[NoArgAnyCallable] = None,
+) -> Any:
+    """
+    Indicates that attribute is only used internally and never mixed with regular fields.
+
+    Types or values of private attrs are not checked by pydantic and it's up to you to keep them relevant.
+
+    Private attrs are stored in model __slots__.
+
+    :param default: the attribute’s default value
+    :param default_factory: callable that will be called when a default value is needed for this attribute
+      If both `default` and `default_factory` are set, an error is raised.
+    """
+    if default is not Undefined and default_factory is not None:
+        raise ValueError('cannot specify both default and default_factory')
+
+    return ModelPrivateAttr(
+        default,
+        default_factory=default_factory,
+    )
diff --git a/pydantic/main.py b/pydantic/main.py
index bc5444a70..bc0b5be5d 100644
--- a/pydantic/main.py
+++ b/pydantic/main.py
@@ -29,7 +29,7 @@ from typing import (
 from .class_validators import ValidatorGroup, extract_root_validators, extract_validators, inherit_validators
 from .error_wrappers import ErrorWrapper, ValidationError
 from .errors import ConfigError, DictError, ExtraError, MissingError
-from .fields import SHAPE_MAPPING, ModelField, PrivateAttr, Undefined
+from .fields import SHAPE_MAPPING, ModelField, ModelPrivateAttr, PrivateAttr, Undefined
 from .json import custom_pydantic_encoder, pydantic_encoder
 from .parse import Protocol, load_file, load_str_bytes
 from .schema import default_ref_template, model_schema
@@ -215,7 +215,7 @@ class ModelMetaclass(ABCMeta):
         validators: 'ValidatorListDict' = {}
 
         pre_root_validators, post_root_validators = [], []
-        private_attributes: Dict[str, PrivateAttr] = {}
+        private_attributes: Dict[str, ModelPrivateAttr] = {}
         slots: Set[str] = namespace.get('__slots__', ())
         slots = {slots} if isinstance(slots, str) else set(slots)
 
@@ -271,7 +271,7 @@ class ModelMetaclass(ABCMeta):
 
             for var_name, value in namespace.items():
                 can_be_changed = var_name not in class_vars and not isinstance(value, untouched_types)
-                if isinstance(value, PrivateAttr):
+                if isinstance(value, ModelPrivateAttr):
                     if not is_valid_private_name(var_name):
                         raise NameError(
                             f'Private attributes "{var_name}" must not be a valid field name; '
diff --git a/pydantic/utils.py b/pydantic/utils.py
index 96b792fbe..4592223d1 100644
--- a/pydantic/utils.py
+++ b/pydantic/utils.py
@@ -631,4 +631,4 @@ def is_valid_field(name: str) -> bool:
 
 
 def is_valid_private_name(name: str) -> bool:
-    return not is_valid_field(name) and name not in {'__annotations__', '__module__', '__annotations__', '__qualname__'}
+    return not is_valid_field(name) and name not in {'__annotations__', '__classcell__', '__module__', '__qualname__'}


Test patch:
diff --git a/tests/mypy/modules/success.py b/tests/mypy/modules/success.py
index 202184c93..3d48a33e1 100644
--- a/tests/mypy/modules/success.py
+++ b/tests/mypy/modules/success.py
@@ -9,7 +9,7 @@ from datetime import date, datetime
 from typing import Any, Dict, Generic, List, Optional, TypeVar
 
 from pydantic import BaseModel, NoneStr, PyObject, StrictBool, root_validator, validate_arguments, validator
-from pydantic.fields import Field
+from pydantic.fields import Field, PrivateAttr
 from pydantic.generics import GenericModel
 from pydantic.typing import ForwardRef
 
@@ -145,3 +145,7 @@ class MyConf(BaseModel):
 conf = MyConf()
 var1: date = conf.str_pyobject(2020, 12, 20)
 var2: date = conf.callable_pyobject(2111, 1, 1)
+
+
+class MyPrivateAttr(BaseModel):
+    _private_field: str = PrivateAttr()
diff --git a/tests/test_private_attributes.py b/tests/test_private_attributes.py
index 07f702c7f..48be872e2 100644
--- a/tests/test_private_attributes.py
+++ b/tests/test_private_attributes.py
@@ -158,5 +158,22 @@ def test_slots_are_ignored():
 
 
 def test_default_and_default_factory_used_error():
-    with pytest.raises(TypeError, match='default and default_factory args can not be used together'):
+    with pytest.raises(ValueError, match='cannot specify both default and default_factory'):
         PrivateAttr(default=123, default_factory=lambda: 321)
+
+
+def test_config_override_init():
+    class MyModel(BaseModel):
+        x: str
+        _private_attr: int
+
+        def __init__(self, **data) -> None:
+            super().__init__(**data)
+            self._private_attr = 123
+
+        class Config:
+            underscore_attrs_are_private = True
+
+    m = MyModel(x='hello')
+    assert m.dict() == {'x': 'hello'}
+    assert m._private_attr == 123