Repository: pydantic/pydantic
Pull request: Improve `__setattr__` performance of Pydantic models by caching setter functions
## Change Summary

Attribute setting has been pretty slow for `BaseModel` due to the extensive checks it has been doing for every `__setattr__` call. PR improves performance of `__setattr__` by memoizing the attribute specific handlers to the model class. This makes the attribute assigning some 7x faster. Also add missing benchmarks for attribute usage.

```python
from timeit import timeit
from pydantic import BaseModel

class Model(BaseModel):
    field: int

model = Model(field=1)

def run():
    model.field = 2

# Before 1.048
# After 0.147
print(timeit(run, number=1000000))
```

## Related issue number

fix https://github.com/pydantic/pydantic/issues/10853

## Checklist

* [x] The pull request title is a good summary of the changes - it will be used in the changelog
* [x] Unit tests for the changes exist
* [x] Tests pass on CI
* [x] Documentation reflects the changes where applicable
* [x] My PR is ready to review, **please add a comment including the phrase "please review" to assign reviewers**


Selected Reviewer: @sydney-runkle

Problem statement:
Slow setting an value of a basic Model
### Initial Checks

- [X] I confirm that I'm using Pydantic V2

### Description

Comparing times between instantiating a complete yet simple model and just setting one value of that same model is weird.
Instantiating is 2 times faster than just setting the value.

Here are the results (python 3.12.7, pydantic 2.9.2):
settings the value:           4.338046199991368s
instantiating the model:  2.0315376999787986s

In comparison to pydantic V1 (python 3.10.15, pydantic 1.10.17):
settings the value:           1.136024700012058
instantiating the model:  5.844023300043773


### Example Code

```Python
import timeit
from pydantic import BaseModel


class MyObj(BaseModel):
    name: str
    value: int | None


if __name__ == "__main__":
    my_obj = MyObj(name="a", value=1)

    def set_obj(obj):
        obj.value = 10
        return obj

    def instantiation():
        return MyObj(name="a", value=1)

    print(timeit.timeit("set_obj(my_obj)", globals=globals(), number=1000000))
    print(timeit.timeit("instantiation()", globals=globals(), number=1000000))
```


### Python, Pydantic & OS Version

```Text
pydantic version: 2.9.2
        pydantic-core version: 2.23.4
          pydantic-core build: profile=release pgo=false
                 install path: C:\Users\tweber\.conda\envs\optimizer_be_dev\Lib\site-packages\pydantic
               python version: 3.12.7 | packaged by conda-forge | (main, Oct  4 2024, 15:47:54) [MSC v.1941 64 bit (AMD64)]
                     platform: Windows-10-10.0.19045-SP0
             related packages: fastapi-0.112.2 typing_extensions-4.12.2
                       commit: unknown
```

Hints from issue discussion:
Yes, I have also seen this. It is because `__setattr__` does large amount of checks every time for normal attribute setting before actually setting the value [here](https://github.com/pydantic/pydantic/blob/0157e343bac2313064492744ef07493b5c42bd5b/pydantic/main.py#L938). The data validation works a bit differently hence it's faster (it doesn't need to similar checks every time afaik).

I would wish the `__setattr__` would have a fast path for the usual case of setting normal model attributes. So models would have info about the attribute names that just need to set the value to the `__dict__` and `__pydantic_fields_set__`. Instead of having to do all the checks every single time.

Commit message: fast setattr

Patch to review:
diff --git a/pydantic/_internal/_model_construction.py b/pydantic/_internal/_model_construction.py
index a0c2ecf9cf6..35bd6089cb1 100644
--- a/pydantic/_internal/_model_construction.py
+++ b/pydantic/_internal/_model_construction.py
@@ -152,6 +152,8 @@ def wrapped_model_post_init(self: BaseModel, context: Any, /) -> None:
                 None if cls.model_post_init is BaseModel_.model_post_init else 'model_post_init'
             )
 
+            cls.__pydantic_setattr_handlers__ = {}
+
             cls.__pydantic_decorators__ = DecoratorInfos.build(cls)
 
             # Use the getattr below to grab the __parameters__ from the `typing.Generic` parent class
diff --git a/pydantic/main.py b/pydantic/main.py
index 3468926fe29..4763fe07290 100644
--- a/pydantic/main.py
+++ b/pydantic/main.py
@@ -169,6 +169,9 @@ class BaseModel(metaclass=_model_construction.ModelMetaclass):
     This replaces `Model.__fields__` from Pydantic V1.
     """
 
+    __pydantic_setattr_handlers__: ClassVar[Dict[str, Callable[[BaseModel, Any], None]]]  # noqa: UP006
+    """__setattr__ handlers. Used to speed up __setattr__."""
+
     __pydantic_computed_fields__: ClassVar[Dict[str, ComputedFieldInfo]]  # noqa: UP006
     """A dictionary of computed field names and their corresponding [`ComputedFieldInfo`][pydantic.fields.ComputedFieldInfo] objects."""
 
@@ -890,53 +893,64 @@ def __getattr__(self, item: str) -> Any:
                         raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}')
 
         def __setattr__(self, name: str, value: Any) -> None:
-            if name in self.__class_vars__:
+            if (fast_memo_handler := self.__pydantic_setattr_handlers__.get(name)) is not None:
+                fast_memo_handler(self, value)
+                return
+
+            if (fast_memo_handler := self._setattr_handler(name, value)) is not None:
+                fast_memo_handler(self, value)  # call here to not memo on possibly unknown fields
+                self.__pydantic_setattr_handlers__[name] = fast_memo_handler
+
+        def _setattr_handler(self, name: str, value: Any) -> Callable[[BaseModel, Any], None] | None:
+            """Returns a handler for setting an attribute on the model instance. This handler can be memoized to
+            the class. Gives None when memoization is not safe, then the attribute is set directly.
+            """
+            cls = self.__class__
+            if name in cls.__class_vars__:
                 raise AttributeError(
-                    f'{name!r} is a ClassVar of `{self.__class__.__name__}` and cannot be set on an instance. '
-                    f'If you want to set a value on the class, use `{self.__class__.__name__}.{name} = value`.'
+                    f'{name!r} is a ClassVar of `{cls.__name__}` and cannot be set on an instance. '
+                    f'If you want to set a value on the class, use `{cls.__name__}.{name} = value`.'
                 )
             elif not _fields.is_valid_field_name(name):
-                if self.__pydantic_private__ is None or name not in self.__private_attributes__:
-                    _object_setattr(self, name, value)
-                else:
-                    attribute = self.__private_attributes__[name]
+                if (attribute := cls.__private_attributes__.get(name)) is not None:
                     if hasattr(attribute, '__set__'):
-                        attribute.__set__(self, value)  # type: ignore
+                        return lambda m, val: attribute.__set__(m, val)
                     else:
-                        self.__pydantic_private__[name] = value
-                return
+                        return lambda m, val: m.__pydantic_private__.__setitem__(name, val)
+                else:
+                    return lambda m, val: _object_setattr(m, name, val)
 
-            self._check_frozen(name, value)
+            cls._check_frozen(name, value)
 
-            attr = getattr(self.__class__, name, None)
+            attr = getattr(cls, name, None)
             # NOTE: We currently special case properties and `cached_property`, but we might need
             # to generalize this to all data/non-data descriptors at some point. For non-data descriptors
             # (such as `cached_property`), it isn't obvious though. `cached_property` caches the value
             # to the instance's `__dict__`, but other non-data descriptors might do things differently.
             if isinstance(attr, property):
-                attr.__set__(self, value)
+                return lambda m, val: attr.__set__(m, val)
             elif isinstance(attr, cached_property):
-                self.__dict__[name] = value
-            elif self.model_config.get('validate_assignment', None):
-                self.__pydantic_validator__.validate_assignment(self, name, value)
-            elif self.model_config.get('extra') != 'allow' and name not in self.__pydantic_fields__:
+                return lambda m, val: m.__dict__.__setitem__(name, val)
+            elif cls.model_config.get('validate_assignment'):
+                return lambda m, val: m.__pydantic_validator__.validate_assignment(m, name, val)
+            elif cls.model_config.get('extra') != 'allow' and name not in cls.__pydantic_fields__:
                 # TODO - matching error
-                raise ValueError(f'"{self.__class__.__name__}" object has no field "{name}"')
-            elif self.model_config.get('extra') == 'allow' and name not in self.__pydantic_fields__:
-                if self.model_extra and name in self.model_extra:
-                    self.__pydantic_extra__[name] = value  # type: ignore
+                raise ValueError(f'"{cls.__name__}" object has no field "{name}"')
+            elif cls.model_config.get('extra') == 'allow' and name not in cls.__pydantic_fields__:
+                if attr is None:
+                    # attribute does not exist, so put it in extra
+                    self.__pydantic_extra__[name] = value
+                    return None  # Can not return memoized handler with possibly freeform attr names
                 else:
-                    try:
-                        getattr(self, name)
-                    except AttributeError:
-                        # attribute does not already exist on instance, so put it in extra
-                        self.__pydantic_extra__[name] = value  # type: ignore
-                    else:
-                        # attribute _does_ already exist on instance, and was not in extra, so update it
-                        _object_setattr(self, name, value)
+                    # attribute _does_ exist, and was not in extra, so update it
+                    return lambda m, val: _object_setattr(m, name, val)
             else:
-                self.__dict__[name] = value
-                self.__pydantic_fields_set__.add(name)
+                # Normal model field
+                def handler(m: BaseModel, val: Any) -> None:
+                    m.__dict__[name] = val
+                    m.__pydantic_fields_set__.add(name)
+
+                return handler
 
         def __delattr__(self, item: str) -> Any:
             if item in self.__private_attributes__:
@@ -964,10 +978,11 @@ def __delattr__(self, item: str) -> Any:
                 except AttributeError:
                     raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}')
 
-    def _check_frozen(self, name: str, value: Any) -> None:
-        if self.model_config.get('frozen', None):
+    @classmethod
+    def _check_frozen(cls, name: str, value: Any) -> None:
+        if cls.model_config.get('frozen', None):
             typ = 'frozen_instance'
-        elif getattr(self.__pydantic_fields__.get(name), 'frozen', False):
+        elif getattr(cls.__pydantic_fields__.get(name), 'frozen', False):
             typ = 'frozen_field'
         else:
             return
@@ -976,7 +991,7 @@ def _check_frozen(self, name: str, value: Any) -> None:
             'loc': (name,),
             'input': value,
         }
-        raise pydantic_core.ValidationError.from_exception_data(self.__class__.__name__, [error])
+        raise pydantic_core.ValidationError.from_exception_data(cls.__name__, [error])
 
     def __getstate__(self) -> dict[Any, Any]:
         private = self.__pydantic_private__
diff --git a/tests/benchmarks/test_attribute_access.py b/tests/benchmarks/test_attribute_access.py
new file mode 100644
index 00000000000..8b8bf05fce7
--- /dev/null
+++ b/tests/benchmarks/test_attribute_access.py
@@ -0,0 +1,84 @@
+from functools import cached_property
+
+import pytest
+
+from pydantic import BaseModel, ConfigDict, ValidationError
+
+
+class InnerValidateAssignment(BaseModel):
+    model_config = ConfigDict(validate_assignment=True)
+    inner_field1: str
+    inner_field2: int
+
+
+class Model(BaseModel):
+    field1: str
+    field2: int
+    field3: float
+    inner1: InnerValidateAssignment
+    inner2: InnerValidateAssignment
+
+    _private_field1: str
+    _private_field2: int
+    _private_field3: float
+
+    @cached_property
+    def prop_cached1(self) -> str:
+        return self.field1 + self._private_field1
+
+    @cached_property
+    def prop_cached2(self) -> int:
+        return self.field2 + self._private_field2
+
+    @cached_property
+    def prop_cached3(self) -> float:
+        return self.field3 + self._private_field3
+
+
+def test_setattr(benchmark):
+    def set_attrs(m):
+        m.field1 = 'test1'
+        m.field2 = 43
+        m.field3 = 4.0
+        m.inner1.inner_field1 = 'test inner1'
+        m.inner1.inner_field2 = 421
+        m.inner2.inner_field1 = 'test inner2'
+        m.inner2.inner_field2 = 422
+        m._private_field1 = 'test2'
+        m._private_field2 = 44
+        m._private_field3 = 5.1
+        m.prop_cached1 = 'cache override'
+        m.prop_cached2 = 10
+        m.prop_cached3 = 10.1
+
+    inner = {'inner_field1': 'test inner', 'inner_field2': 420}
+    model = Model(field1='test', field2=42, field3=3.14, inner1=inner, inner2=inner)
+    benchmark(set_attrs, model)
+
+    model.field2 = 'bad'  # check benchmark setup
+    with pytest.raises(ValidationError):
+        model.inner1.field2 = 'bad'
+
+
+def test_getattr(benchmark):
+    def get_attrs(m):
+        _ = m.field1
+        _ = m.field2
+        _ = m.field3
+        _ = m.inner1.inner_field1
+        _ = m.inner1.inner_field2
+        _ = m.inner2.inner_field1
+        _ = m.inner2.inner_field2
+        _ = m._private_field1
+        _ = m._private_field2
+        _ = m._private_field3
+        _ = m.prop_cached1
+        _ = m.prop_cached2
+        _ = m.prop_cached3
+
+    inner = {'inner_field1': 'test inner', 'inner_field2': 420}
+    model = Model(field1='test1', field2=42, field3=3.14, inner1=inner, inner2=inner)
+    model._private_field1 = 'test2'
+    model._private_field2 = 43
+    model._private_field3 = 4.14
+    benchmark(get_attrs, model)
diff --git a/tests/test_edge_cases.py b/tests/test_edge_cases.py
index 5dba203f7bc..53ad018f0e4 100644
--- a/tests/test_edge_cases.py
+++ b/tests/test_edge_cases.py
@@ -2900,3 +2900,30 @@ def default_int_factory() -> int: ...
         int_factory: Callable[[], int] = Field(default=default_int_factory)
 
     assert SomeModel.model_fields['int_factory'].default is SomeModel.default_int_factory
+
+
+def test_setattr_handler_memo_does_not_inherit() -> None:
+    class Model1(BaseModel):
+        a: int
+
+    class Model2(Model1):
+        a: int
+
+    assert not Model1.__pydantic_setattr_handlers__
+    assert not Model2.__pydantic_setattr_handlers__
+
+    m1 = Model1(a=1)
+    m2 = Model2(a=10)
+
+    assert not Model1.__pydantic_setattr_handlers__
+    assert not Model2.__pydantic_setattr_handlers__
+
+    m2.a = 11
+    assert not Model1.__pydantic_setattr_handlers__
+    assert 'a' in Model2.__pydantic_setattr_handlers__
+    handler2 = Model2.__pydantic_setattr_handlers__['a']
+
+    m1.a = 2
+    assert 'a' in Model1.__pydantic_setattr_handlers__
+    assert Model1.__pydantic_setattr_handlers__['a'] is not handler2
+    assert Model2.__pydantic_setattr_handlers__['a'] is handler2
diff --git a/tests/test_main.py b/tests/test_main.py
index cfcece87bb1..ba9e59a43ab 100644
--- a/tests/test_main.py
+++ b/tests/test_main.py
@@ -735,8 +735,12 @@ def test_validating_assignment_pass(ValidateAssignmentModel):
     assert p.model_dump() == {'a': 2, 'b': 'hi'}
 
 
-def test_validating_assignment_fail(ValidateAssignmentModel):
+@pytest.mark.parametrize('init_valid', [False, True])
+def test_validating_assignment_fail(ValidateAssignmentModel, init_valid: bool):
     p = ValidateAssignmentModel(a=5, b='hello')
+    if init_valid:
+        p.a = 5
+        p.b = 'hello'
 
     with pytest.raises(ValidationError) as exc_info:
         p.a = 'b'


Code review comments:
diff_hunk: @@ -890,53 +893,64 @@ def __getattr__(self, item: str) -> Any:
                         raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}')
 
         def __setattr__(self, name: str, value: Any) -> None:
-            if name in self.__class_vars__:
+            if (fast_memo_handler := self.__pydantic_setattr_handlers__.get(name)) is not None:
+                fast_memo_handler(self, value)
+                return
+
+            if (fast_memo_handler := self._setattr_handler(name, value)) is not None:
+                fast_memo_handler(self, value)  # call here to not memo on possibly unknown fields
+                self.__pydantic_setattr_handlers__[name] = fast_memo_handler
+
+        def _setattr_handler(self, name: str, value: Any) -> Callable[[BaseModel, Any], None] | None:
+            """Returns a handler for setting an attribute on the model instance. This handler can be memoized to
+            the class. Gives None when memoization is not safe, then the attribute is set directly.
path: pydantic/main.py
text: @user1:
Can we use the classic one line summary, then a `Returns:` block below that?

@user1:
https://google.github.io/styleguide/pyguide.html#383-functions-and-methods

@author:
Done

diff_hunk: @@ -169,6 +169,9 @@ class BaseModel(metaclass=_model_construction.ModelMetaclass):
     This replaces `Model.__fields__` from Pydantic V1.
     """
 
+    __pydantic_setattr_handlers__: ClassVar[Dict[str, Callable[[BaseModel, Any], None]]]  # noqa: UP006
+    """__setattr__ handlers. Memoizing the setattr handlers leads to a dramatic performance improvement in `__setattr__`"""
path: pydantic/main.py
text: @user1:
```suggestion
    """`__setattr__` handlers. Memoizing the handlers leads to a dramatic performance improvement in `__setattr__`"""
```

diff_hunk: @@ -890,53 +893,67 @@ def __getattr__(self, item: str) -> Any:
                         raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}')
 
         def __setattr__(self, name: str, value: Any) -> None:
-            if name in self.__class_vars__:
+            if (setattr_handler := self.__pydantic_setattr_handlers__.get(name)) is not None:
+                setattr_handler(self, value)
+                return
+
+            if (setattr_handler := self._setattr_handler(name, value)) is not None:
path: pydantic/main.py
text: @user1:
```suggestion
            elif (setattr_handler := self._setattr_handler(name, value)) is not None:
```

diff_hunk: @@ -890,53 +893,64 @@ def __getattr__(self, item: str) -> Any:
                         raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}')
 
         def __setattr__(self, name: str, value: Any) -> None:
-            if name in self.__class_vars__:
+            if (fast_memo_handler := self.__pydantic_setattr_handlers__.get(name)) is not None:
+                fast_memo_handler(self, value)
path: pydantic/main.py
text: @user1:
I wouldn't expect to have a `slow_memo_handler`, given that the point of memoization is to make things fast. Maybe we could use `setattr_handler` instead?

@user2:
I think the word `fast` in the name of this variable is carrying some weight in terms of documenting what is going on here — it doesn't seem to be indicated explicitly that the reason for memoizing the attribute handler is purely for performance and isn't otherwise logically required. I mean, I know we have the docstring of `__pydantic_setattr_handlers__`, but that is far away from the implementing code and still doesn't make it clear that the current logic _only_ makes use of the memoization for performance reasons (and isn't otherwise dependent on the memoization), which may be useful context if we refactor this in the future.

I don't really mind the word fast in the name, but I think either way we should add a comment here reiterating that the memoization is purely for performance reasons. And if we do, the word fast probably becomes less useful.

@author:
Done, simplified the name, it wasnt so pretty yet :)