Repository: pydantic/pydantic

Issue:
Depreciate `.copy()`, instead implement `__copy__`
As suggested by @PrettyWood we should remove `.copy()` and all it's "powerful" arguments as they're somewhat broken, and instead implement `__copy__`.

I guess we should leave `copy()` in place for now and add a deprecation warning.

@tiangolo @hramezani @adriangb @dmontagu anyone else, what do you think about this - is it going to cause a revolution?

I guess if we really want to keep `.copy()`, we could just use `.model_dump()` (née `.dict()`) and then run validation, to make sure the arguments all behave correctly.

Either way we should definitely implement `__copy__` and `__deepcopy__`. @ybressler you were asking about PRs to work on, maybe you could work on this bit while we discuss removing `.copy()`?


Discussion:
In my own code, I have used the `copy` method plenty of times (with and without `deep=True`), but I don't think I have _ever_ made use of the `update`, `include`, or `exclude` arguments to the `copy` method; the `include` and `exclude` arguments in particular seem like they would be fairly niche.

I do like having a model method that achieves copy (just because I like autocomplete / not needing to add imports), but I also wouldn't mind using `copy.copy`/`copy.deepcopy` if necessary; either way I agree we should implement `__copy__` and `__deepcopy__`.

I think there is one important use case for which it is worth retaining something like the `copy` method with the `update` argument: making (shallow) copies of frozen models with some field values modified. I personally think that use case is worth a `model_replace` method or similar (though a free-standing function could work, I just like having methods), and even if the `model_replace` was functionally very similar to the current `copy` method I think it might still be worth the renaming just to convey the intent (I think `copy` with the `update` argument is a bit less "discoverable" for this use case).

If we kept a method, I think it would be safe to drop the `deep` argument (as I think it's more reasonable to require a call to `copy.deepcopy` when requesting a deep copy), but don't have a strong opinion.
Referencing #4723, the stdlib `dataclasses` includes a `replace` function to duplicate objects (designed for frozen classes) with changes to the fields. A `model_replace` makes sense as well, but I'd opt for replicating with `dataclasses` does tbh.
Conclusions from recent conversation:
* All calls to the existing `copy` method produce DeprecationWarnings
* We'll add a `model_copy` method that only accepts `deep: bool`, which calls `__copy__` or `__deepcopy__` as appropriate.
* Shallow copies will be done with `__copy__`
    * (Drop support for support include/exclude on shallow copy — replaced with model_dump w/ include/exclude and re-parse the model)
* (Deep) copy with include or exclude becomes:
    ```
    data = model.model_dump(include=include, exclude=exclude, round_trip=True)
    return self.model_validate(data)
    ```


Resolving patch:
diff --git a/pydantic/decorator.py b/pydantic/decorator.py
--- a/pydantic/decorator.py
+++ b/pydantic/decorator.py
@@ -179,7 +179,7 @@ def build_values(self, args: Tuple[Any, ...], kwargs: Dict[str, Any]) -> Dict[st
         return values
 
     def execute(self, m: BaseModel) -> Any:
-        d = {k: v for k, v in m._iter() if k in m.__fields_set__ or m.model_fields[k].default_factory}
+        d = {k: v for k, v in m.__dict__.items() if k in m.__fields_set__ or m.model_fields[k].default_factory}
         var_kwargs = d.pop(self.v_kwargs_name, {})
 
         if self.v_args_name in d:
diff --git a/pydantic/main.py b/pydantic/main.py
--- a/pydantic/main.py
+++ b/pydantic/main.py
@@ -7,7 +7,7 @@
 import typing
 import warnings
 from abc import ABCMeta
-from copy import deepcopy
+from copy import copy, deepcopy
 from enum import Enum
 from functools import partial
 from inspect import getdoc
@@ -363,6 +363,7 @@ def _copy_and_set_values(self: Model, values: dict[str, Any], fields_set: set[st
 
         return m
 
+    # @typing_extensions.deprecated('This method is now deprecated; use `model_copy` instead')
     def copy(
         self: Model,
         *,
@@ -372,6 +373,12 @@ def copy(
         deep: bool = False,
     ) -> Model:
         """
+        This method is now deprecated; use `model_copy` instead. If you need include / exclude, use:
+
+            data = self.model_dump(include=include, exclude=exclude, round_trip=True)
+            data = {**data, **(update or {})}
+            copied = self.model_validate(data)
+
         Duplicate a model, optionally choose which fields to include, exclude and change.
 
         :param include: fields to include in new model
@@ -381,6 +388,11 @@ def copy(
         :param deep: set to `True` to make a deep copy of the model
         :return: new model instance
         """
+        warnings.warn(
+            'The `copy` method is deprecated; use `model_copy` instead. '
+            'See the docstring of `BaseModel.copy` for details about how to handle include / exclude / update.',
+            DeprecationWarning,
+        )
 
         values = dict(
             self._iter(to_dict=False, by_alias=False, include=include, exclude=exclude, exclude_unset=False),
@@ -449,6 +461,7 @@ def schema_json(
         )
 
     @classmethod
+    # @typing_extensions.deprecated('This method is only used for _iter, which is deprecated')
     @typing.no_type_check
     def _get_value(
         cls,
@@ -555,6 +568,7 @@ def __iter__(self) -> TupleGenerator:
         """
         yield from self.__dict__.items()
 
+    # @typing_extensions.deprecated('This private method is only used for `BaseModel.copy`, which is deprecated')
     def _iter(
         self,
         to_dict: bool = False,
@@ -567,11 +581,15 @@ def _iter(
     ) -> TupleGenerator:
         # Merge field set excludes with explicit exclude parameter with explicit overriding field set options.
         # The extra "is not None" guards are not logically necessary but optimizes performance for the simple case.
-        # if exclude is not None or self.__exclude_fields__ is not None:
-        #     exclude = _utils.ValueItems.merge(self.__exclude_fields__, exclude)
-        #
-        # if include is not None or self.__include_fields__ is not None:
-        #     include = _utils.ValueItems.merge(self.__include_fields__, include, intersect=True)
+        if exclude is not None:
+            exclude = _utils.ValueItems.merge(
+                {k: v.exclude for k, v in self.model_fields.items() if v.exclude is not None}, exclude
+            )
+
+        if include is not None:
+            include = _utils.ValueItems.merge(
+                {k: v.include for k, v in self.model_fields.items()}, include, intersect=True
+            )
 
         allowed_keys = self._calculate_keys(
             include=include, exclude=exclude, exclude_unset=exclude_unset  # type: ignore
@@ -648,6 +666,51 @@ def __eq__(self, other: Any) -> bool:
         else:
             return self.model_dump() == other
 
+    def model_copy(self: Model, *, update: dict[str, Any] | None = None, deep: bool = False) -> Model:
+        """
+        Returns a copy of the model.
+
+        :param update: values to change/add in the new model. Note: the data is not validated before creating
+            the new model: you should trust this data
+        :param deep: set to `True` to make a deep copy of the model
+        :return: new model instance
+        """
+        copied = self.__deepcopy__() if deep else self.__copy__()
+        if update:
+            copied.__dict__.update(update)
+            copied.__fields_set__.update(update.keys())
+        return copied
+
+    def __copy__(self: Model) -> Model:
+        """
+        Returns a shallow copy of the model
+        """
+        cls = type(self)
+        m = cls.__new__(cls)
+        _object_setattr(m, '__dict__', copy(self.__dict__))
+        _object_setattr(m, '__fields_set__', copy(self.__fields_set__))
+        for name in self.__private_attributes__:
+            value = getattr(self, name, Undefined)
+            if value is not Undefined:
+                _object_setattr(m, name, value)
+        return m
+
+    def __deepcopy__(self: Model, memo: dict[int, Any] | None = None) -> Model:
+        """
+        Returns a deep copy of the model
+        """
+        cls = type(self)
+        m = cls.__new__(cls)
+        _object_setattr(m, '__dict__', deepcopy(self.__dict__, memo=memo))
+        # This next line doesn't need a deepcopy because __fields_set__ is a set[str],
+        # and attempting a deepcopy would be marginally slower.
+        _object_setattr(m, '__fields_set__', copy(self.__fields_set__))
+        for name in self.__private_attributes__:
+            value = getattr(self, name, Undefined)
+            if value is not Undefined:
+                _object_setattr(m, name, deepcopy(value, memo=memo))
+        return m
+
     def __repr_args__(self) -> _repr.ReprArgs:
         return [
             (k, v)


Test patch:
diff --git a/tests/mypy/modules/success.py b/tests/mypy/modules/success.py
--- a/tests/mypy/modules/success.py
+++ b/tests/mypy/modules/success.py
@@ -122,7 +122,7 @@ def day_of_week(dt: datetime) -> int:
 assert m_from_obj.last_name == data['last_name']
 assert m_from_obj.list_of_ints == data['list_of_ints']
 
-m_copy = m_from_obj.copy()
+m_copy = m_from_obj.model_copy()
 
 assert isinstance(m_copy, Model)
 assert m_copy.age == m_from_obj.age
diff --git a/tests/test_config.py b/tests/test_config.py
--- a/tests/test_config.py
+++ b/tests/test_config.py
@@ -334,7 +334,7 @@ class Config:
                 frozen = True
 
         m = Model(a=40, b=10)
-        assert m == m.copy()
+        assert m == m.model_copy()
 
     def test_config_class_is_deprecated(self):
         with pytest.warns(
diff --git a/tests/test_construction.py b/tests/test_construction.py
--- a/tests/test_construction.py
+++ b/tests/test_construction.py
@@ -71,9 +71,39 @@ class Model(BaseModel):
     assert m.b == content_str
 
 
-def test_simple_copy():
+def deprecated_copy(m: BaseModel, *, include=None, exclude=None, update=None, deep=False):
+    """
+    This should only be used to make calls to the deprecated `copy` method with arguments
+    that have been removed from `model_copy`. Otherwise, use the `copy_method` fixture below
+    """
+    with pytest.warns(
+        DeprecationWarning,
+        match=(
+            'The `copy` method is deprecated; use `model_copy` instead. '
+            'See the docstring of `BaseModel.copy` for details about how to handle include / exclude / update.'
+        ),
+    ):
+        return m.copy(include=include, exclude=exclude, update=update, deep=deep)
+
+
+@pytest.fixture(params=['copy', 'model_copy'])
+def copy_method(request):
+    """
+    Fixture to test both the old/deprecated `copy` and new `model_copy` methods.
+    """
+    if request.param == 'copy':
+        return deprecated_copy
+    else:
+
+        def new_copy_method(m, *, update=None, deep=False):
+            return m.model_copy(update=update, deep=deep)
+
+        return new_copy_method
+
+
+def test_simple_copy(copy_method):
     m = Model(a=24)
-    m2 = m.copy()
+    m2 = copy_method(m)
 
     assert m.a == m2.a == 24
     assert m.b == m2.b == 10
@@ -94,10 +124,10 @@ class ModelTwo(BaseModel):
     return ModelTwo
 
 
-def test_deep_copy(ModelTwo):
+def test_deep_copy(ModelTwo, copy_method):
     m = ModelTwo(a=24, d=Model(a='12'))
     m._foo_ = {'new value'}
-    m2 = m.copy(deep=True)
+    m2 = copy_method(m, deep=True)
 
     assert m.a == m2.a == 24
     assert m.b == m2.b == 10
@@ -109,10 +139,9 @@ def test_deep_copy(ModelTwo):
     assert m._foo_ is not m2._foo_
 
 
-@pytest.mark.xfail(reason='working on V2')
 def test_copy_exclude(ModelTwo):
     m = ModelTwo(a=24, d=Model(a='12'))
-    m2 = m.copy(exclude={'b'})
+    m2 = deprecated_copy(m, exclude={'b'})
 
     assert m.a == m2.a == 24
     assert isinstance(m2.d, Model)
@@ -126,10 +155,9 @@ def test_copy_exclude(ModelTwo):
     assert m != m2
 
 
-@pytest.mark.xfail(reason='working on V2')
 def test_copy_include(ModelTwo):
     m = ModelTwo(a=24, d=Model(a='12'))
-    m2 = m.copy(include={'a'})
+    m2 = deprecated_copy(m, include={'a'})
 
     assert m.a == m2.a == 24
     assert set(m.model_dump().keys()) == {'a', 'b', 'c', 'd'}
@@ -138,16 +166,14 @@ def test_copy_include(ModelTwo):
     assert m != m2
 
 
-@pytest.mark.xfail(reason='working on V2')
 def test_copy_include_exclude(ModelTwo):
     m = ModelTwo(a=24, d=Model(a='12'))
-    m2 = m.copy(include={'a', 'b', 'c'}, exclude={'c'})
+    m2 = deprecated_copy(m, include={'a', 'b', 'c'}, exclude={'c'})
 
     assert set(m.model_dump().keys()) == {'a', 'b', 'c', 'd'}
     assert set(m2.model_dump().keys()) == {'a', 'b'}
 
 
-@pytest.mark.xfail(reason='working on V2')
 def test_copy_advanced_exclude():
     class SubSubModel(BaseModel):
         a: str
@@ -162,16 +188,15 @@ class Model(BaseModel):
         f: SubModel
 
     m = Model(e='e', f=SubModel(c='foo', d=[SubSubModel(a='a', b='b'), SubSubModel(a='c', b='e')]))
-    m2 = m.copy(exclude={'f': {'c': ..., 'd': {-1: {'a'}}}})
+    m2 = deprecated_copy(m, exclude={'f': {'c': ..., 'd': {-1: {'a'}}}})
     assert hasattr(m.f, 'c')
     assert not hasattr(m2.f, 'c')
 
     assert m2.model_dump() == {'e': 'e', 'f': {'d': [{'a': 'a', 'b': 'b'}, {'b': 'e'}]}}
-    m2 = m.copy(exclude={'e': ..., 'f': {'d'}})
+    m2 = deprecated_copy(m, exclude={'e': ..., 'f': {'d'}})
     assert m2.model_dump() == {'f': {'c': 'foo'}}
 
 
-@pytest.mark.xfail(reason='working on V2')
 def test_copy_advanced_include():
     class SubSubModel(BaseModel):
         a: str
@@ -186,16 +211,15 @@ class Model(BaseModel):
         f: SubModel
 
     m = Model(e='e', f=SubModel(c='foo', d=[SubSubModel(a='a', b='b'), SubSubModel(a='c', b='e')]))
-    m2 = m.copy(include={'f': {'c'}})
+    m2 = deprecated_copy(m, include={'f': {'c'}})
     assert hasattr(m.f, 'c')
     assert hasattr(m2.f, 'c')
     assert m2.model_dump() == {'f': {'c': 'foo'}}
 
-    m2 = m.copy(include={'e': ..., 'f': {'d': {-1}}})
+    m2 = deprecated_copy(m, include={'e': ..., 'f': {'d': {-1}}})
     assert m2.model_dump() == {'e': 'e', 'f': {'d': [{'a': 'c', 'b': 'e'}]}}
 
 
-@pytest.mark.xfail(reason='working on V2')
 def test_copy_advanced_include_exclude():
     class SubSubModel(BaseModel):
         a: str
@@ -210,37 +234,38 @@ class Model(BaseModel):
         f: SubModel
 
     m = Model(e='e', f=SubModel(c='foo', d=[SubSubModel(a='a', b='b'), SubSubModel(a='c', b='e')]))
-    m2 = m.copy(include={'e': ..., 'f': {'d'}}, exclude={'e': ..., 'f': {'d': {0}}})
+    m2 = deprecated_copy(m, include={'e': ..., 'f': {'d'}}, exclude={'e': ..., 'f': {'d': {0}}})
     assert m2.model_dump() == {'f': {'d': [{'a': 'c', 'b': 'e'}]}}
 
 
-@pytest.mark.xfail(reason='working on V2')
-def test_copy_update(ModelTwo):
+def test_copy_update(ModelTwo, copy_method):
     m = ModelTwo(a=24, d=Model(a='12'))
-    m2 = m.copy(update={'a': 'different'})
+    m2 = copy_method(m, update={'a': 'different'})
 
     assert m.a == 24
     assert m2.a == 'different'
-    assert set(m.model_dump().keys()) == set(m2.model_dump().keys()) == {'a', 'b', 'c', 'd'}
-
-    assert m != m2
+    m_keys = m.model_dump().keys()
+    with pytest.warns(UserWarning, match='Expected `float` but got `str`'):
+        m2_keys = m2.model_dump().keys()
+    assert set(m_keys) == set(m2_keys) == {'a', 'b', 'c', 'd'}
+    with pytest.warns(UserWarning, match='Expected `float` but got `str`'):
+        assert m != m2
 
 
-@pytest.mark.xfail(reason='working on V2')
-def test_copy_update_unset():
+def test_copy_update_unset(copy_method):
     class Foo(BaseModel):
-        foo: Optional[str]
-        bar: Optional[str]
+        foo: Optional[str] = None
+        bar: Optional[str] = None
 
     assert (
-        Foo(foo='hello').copy(update={'bar': 'world'}).model_dump_json(exclude_unset=True)
-        == '{"foo": "hello", "bar": "world"}'
+        copy_method(Foo(foo='hello'), update={'bar': 'world'}).model_dump_json(exclude_unset=True)
+        == b'{"foo":"hello","bar":"world"}'
     )
 
 
-def test_copy_set_fields(ModelTwo):
+def test_copy_set_fields(ModelTwo, copy_method):
     m = ModelTwo(a=24, d=Model(a='12'))
-    m2 = m.copy()
+    m2 = copy_method(m)
 
     assert m.model_dump(exclude_unset=True) == {'a': 24.0, 'd': {'a': 12}}
     assert m.model_dump(exclude_unset=True) == m2.model_dump(exclude_unset=True)
@@ -312,26 +337,26 @@ class PickleModelTwo(BaseModel):
     assert not hasattr(m3, '_foo_')
 
 
-def test_copy_undefined(ModelTwo):
+def test_copy_undefined(ModelTwo, copy_method):
     m = ModelTwo(a=24, d=Model(a='123.45'))
-    m2 = m.copy()
+    m2 = copy_method(m)
     assert m2._foo_ == {'private'}
 
     m._foo_ = Undefined
-    m3 = m.copy()
+    m3 = copy_method(m)
     assert not hasattr(m3, '_foo_')
 
 
-def test_immutable_copy_with_frozen():
+def test_immutable_copy_with_frozen(copy_method):
     class Model(BaseModel):
         model_config = ConfigDict(frozen=True)
         a: int
         b: int
 
     m = Model(a=40, b=10)
-    assert m == m.copy()
+    assert m == copy_method(m)
 
-    m2 = m.copy(update={'b': 12})
+    m2 = copy_method(m, update={'b': 12})
     assert repr(m2) == 'Model(a=40, b=12)'
     with pytest.raises(TypeError):
         m2.b = 13
@@ -344,7 +369,6 @@ def test_pickle_fields_set():
     assert m2.model_dump(exclude_unset=True) == {'a': 24}
 
 
-@pytest.mark.xfail(reason='working on V2')
 def test_copy_update_exclude():
     class SubModel(BaseModel):
         a: str
@@ -356,21 +380,25 @@ class Model(BaseModel):
 
     m = Model(c='ex', d=dict(a='ax', b='bx'))
     assert m.model_dump() == {'c': 'ex', 'd': {'a': 'ax', 'b': 'bx'}}
-    assert m.copy(exclude={'c'}).model_dump() == {'d': {'a': 'ax', 'b': 'bx'}}
-    assert m.copy(exclude={'c'}, update={'c': 42}).model_dump() == {'c': 42, 'd': {'a': 'ax', 'b': 'bx'}}
+    assert deprecated_copy(m, exclude={'c'}).model_dump() == {'d': {'a': 'ax', 'b': 'bx'}}
+    with pytest.warns(UserWarning, match='Expected `str` but got `int`'):
+        assert deprecated_copy(m, exclude={'c'}, update={'c': 42}).model_dump() == {
+            'c': 42,
+            'd': {'a': 'ax', 'b': 'bx'},
+        }
 
     assert m._calculate_keys(exclude={'x': ...}, include=None, exclude_unset=False) == {'c', 'd'}
     assert m._calculate_keys(exclude={'x': ...}, include=None, exclude_unset=False, update={'c': 42}) == {'d'}
 
 
-def test_shallow_copy_modify():
+def test_shallow_copy_modify(copy_method):
     class X(BaseModel):
         val: int
         deep: Any
 
     x = X(val=1, deep={'deep_thing': [1, 2]})
 
-    y = x.copy()
+    y = copy_method(x)
     y.val = 2
     y.deep['deep_thing'].append(3)
 
@@ -387,6 +415,9 @@ class Model(BaseModel):
         foo: List[int] = Field(default_factory=list)
         bar: str = 'Baz'
 
+    # TODO: As part of v1 -> v2, the logic for pydantic.fields.FieldInfo.get_default changed to not
+    #  call `default_factory`. Is this really preferable? Naively, it seems to me that it is a user
+    #  mistake if calling default_factory has side effects.. But maybe there's a good reason?
     m = Model.model_construct()
     assert m.foo == []
     assert m.bar == 'Baz'
@@ -399,7 +430,49 @@ class User(BaseModel):
         dob: str
 
     user = User(name='test_user', age=23, dob='01/01/2000')
-    user_copy = user.copy(exclude={'dob': ...})
+    user_copy = deprecated_copy(user, exclude={'dob': ...})
 
     assert 'dob' in user.__fields_set__
     assert 'dob' not in user_copy.__fields_set__
+
+
+def test_dunder_copy(ModelTwo):
+    m = ModelTwo(a=24, d=Model(a='12'))
+    m2 = m.__copy__()
+    assert m is not m2
+
+    assert m.a == m2.a == 24
+    assert isinstance(m2.d, Model)
+    assert m.d is m2.d
+    assert m.d.a == m2.d.a == 12
+
+    m.a = 12
+    assert m.a != m2.a
+
+
+def test_dunder_deepcopy(ModelTwo):
+    m = ModelTwo(a=24, d=Model(a='12'))
+    m2 = m.__copy__()
+    assert m is not m2
+
+    assert m.a == m2.a == 24
+    assert isinstance(m2.d, Model)
+    assert m.d is m2.d
+    assert m.d.a == m2.d.a == 12
+
+    m.a = 12
+    assert m.a != m2.a
+
+
+def test_model_copy(ModelTwo):
+    m = ModelTwo(a=24, d=Model(a='12'))
+    m2 = m.__copy__()
+    assert m is not m2
+
+    assert m.a == m2.a == 24
+    assert isinstance(m2.d, Model)
+    assert m.d is m2.d
+    assert m.d.a == m2.d.a == 12
+
+    m.a = 12
+    assert m.a != m2.a
diff --git a/tests/test_edge_cases.py b/tests/test_edge_cases.py
--- a/tests/test_edge_cases.py
+++ b/tests/test_edge_cases.py
@@ -2136,6 +2136,9 @@ class MyModel(BaseModel):
         x: int = 1
         y: str = 'a'
 
+    # with pytest.warns(
+    #     DeprecationWarning, match='This private method is only used for `BaseModel.copy`, which is deprecated'
+    # ):
     assert list(MyModel()._iter(by_alias=True)) == [('x', 1), ('y', 'a')]
 
 
diff --git a/tests/test_types.py b/tests/test_types.py
--- a/tests/test_types.py
+++ b/tests/test_types.py
@@ -3059,8 +3059,10 @@ class Foobar(BaseModel):
     assert f.empty_password.get_secret_value() == b''
 
     # Assert that SecretBytes is equal to SecretBytes if the secret is the same.
-    assert f == f.copy()
-    assert f != f.copy(update=dict(password=b'4321'))
+    assert f == f.model_copy()
+    copied_with_changes = f.model_copy()
+    copied_with_changes.password = SecretBytes(b'4321')
+    assert f != copied_with_changes
 
 
 def test_secretbytes_is_secret_field():