Repository: pydantic/pydantic

Issue:
smart_deepcopy causes sqlalchemy boolean value error on import
### Checks

* [X] I added a descriptive title to this issue
* [X] I have searched (google, github) for similar issues and couldn't find anything
* [X] I have read and followed [the docs](https://pydantic-docs.helpmanual.io/) and still think this is a bug

<!-- Sorry to sound so draconian, but every second saved replying to issues is time spend improving pydantic :-) -->

# Bug

Output of `python -c "import pydantic.utils; print(pydantic.utils.version_info())"`:
```
root@20116b25f897:/[REDACTED] # python -c "import pydantic.utils; print(pydantic.utils.version_info())"
             pydantic version: 1.8.2
            pydantic compiled: False
                 install path: /usr/local/lib/python3.10/site-packages/pydantic
               python version: 3.10.5 (main, Jun  7 2022, 18:49:47) [GCC 10.2.1 20210110]
                     platform: Linux-5.10.104-linuxkit-x86_64-with-glibc2.31
     optional deps. installed: ['dotenv', 'email-validator', 'typing-extensions']
```
<!-- or if you're using pydantic prior to v1.3, manually include: OS, python version and pydantic version -->

<!-- Please read the [docs](https://pydantic-docs.helpmanual.io/) and search through issues to
confirm your bug hasn't already been reported. -->

<!-- Where possible please include a self-contained code snippet describing your bug: -->

```py
"""Base class for all database models."""

from typing import Any, Dict, Optional
from uuid import UUID, uuid4

import sqlalchemy as sa
import sqlalchemy.dialects.postgresql as pg
from inflection import underscore
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import declared_attr
from sqlalchemy_utils import Timestamp

from pydantic import BaseModel as _BaseModel

class CommonBase(_BaseModel, Timestamp):
    """SQLAlchemy Base common base class for all database models."""

    id: UUID = sa.Column(pg.UUID(as_uuid=True), primary_key=True, default=uuid4)

    @declared_attr
    def __tablename__(cls) -> Optional[str]:  # pylint: disable=no-self-argument
        """Define table name for all models as the snake case of the model's name."""
        return underscore(cls.__name__)

    def as_dict(self) -> Dict[str, Any]:
        """Return python dictionary of SQLAlchemy model."""
        return self.__dict__

BaseModel = declarative_base(cls=CommonBase)
```

On import of this BaseModel, we are getting a hard to track down error that seems to be related to sqlalchemy boolean expressions, but stemming from the smart_deepcopy line in pydantic and I am not quite sure what the real issue is, and whether it is caused by pydantic, sqlalchemy, or my setup (most likely).

The stack trace received on import is:

```
app/models/base_model.py:15: in <module>
    class CommonBase(_BaseModel, Timestamp):
/usr/local/lib/python3.10/site-packages/pydantic/main.py:299: in __new__
    fields[ann_name] = ModelField.infer(
/usr/local/lib/python3.10/site-packages/pydantic/fields.py:411: in infer
    return cls(
/usr/local/lib/python3.10/site-packages/pydantic/fields.py:342: in __init__
    self.prepare()
/usr/local/lib/python3.10/site-packages/pydantic/fields.py:445: in prepare
    self._set_default_and_type()
/usr/local/lib/python3.10/site-packages/pydantic/fields.py:473: in _set_default_and_type
    default_value = self.get_default()
/usr/local/lib/python3.10/site-packages/pydantic/fields.py:345: in get_default
    return smart_deepcopy(self.default) if self.default_factory is None else self.default_factory()
/usr/local/lib/python3.10/site-packages/pydantic/utils.py:627: in smart_deepcopy
    elif not obj and obj_type in BUILTIN_COLLECTIONS:
/usr/local/lib/python3.10/site-packages/sqlalchemy/sql/elements.py:582: in __bool__
    raise TypeError("Boolean value of this clause is not defined")
E   TypeError: Boolean value of this clause is not defined
```

I placed a debugger just before the raise and found that:

```
(Pdb) self
Column(None, UUID(as_uuid=True), table=None, primary_key=True, nullable=False, default=ColumnDefault(<function uuid4 at 0x7f99d085e200>))
(Pdb) self.__dict__
{'key': None, 'name': None, 'table': None, 'type': UUID(as_uuid=True), 'is_literal': False, 'primary_key': True, '_user_defined_nullable': symbol('NULL_UNSPECIFIED'), 'nullable': False, 'default': ColumnDefault(<function uuid4 at 0x7f99d085e200>), 'server_default': None, 'server_onupdate': None, 'index': None, 'unique': None, 'system': False, 'doc': None, 'onupdate': None, 'autoincrement': 'auto', 'constraints': set(), 'foreign_keys': set(), 'comment': None, 'computed': None, 'identity': None, '_creation_order': 88, 'comparator': <sqlalchemy.sql.type_api.TypeEngine.Comparator object at 0x7f99d07f0a00>, '_memoized_keys': frozenset({'comparator'})}
```

This is not new code in pydantic, and it also seems this check in sqlalchemy was [added almost a decade ago](https://docs.sqlalchemy.org/en/14/changelog/migration_06.html) so I am sure that is must be something wrong with either my expectatins or setup here. While investigating, it seems that this indeed is the correct way to setup a postgres UUID pk column within a pydantic model backed by sqlalchemy, but I am clearly missing something.

Thanks for the incredible work you do on this library! 




Discussion:
Off the top of my head, I don't know.

That's a config setting called `copy_on_model_validation` try setting it to false and seeing if that avoids a copy that causes the problem.
`copy_on_model_validation` didn't change things. I whittled the setup down to the minimum reproducible case:

```py
from uuid import UUID, uuid4
import sqlalchemy as sa
import sqlalchemy.dialects.postgresql as pg
from pydantic import BaseModel as _BaseModel
from sqlalchemy.ext.declarative import declarative_base

class CommonBase(_BaseModel):

    id: UUID = sa.Column(pg.UUID(as_uuid=True), primary_key=True, default=uuid4)

    class Config:
        copy_on_model_validation = False

BaseModel = declarative_base(cls=CommonBase)
```

I also updated to pydantic 1.9.1 without change 🤔 
1.9.1 hasn't changed `smart_deepcopy` so that wouldn't make a difference.

Really I don't think it's correct for the sqlalchemy type to be raising an error on `__bool__`, but I get why they want to.

I think the best solution would be to catch errors like this in `smart_deepcopy` and just fall back to using `deepcopy`, but I wonder whether even deep copy can cope with this type?

Something like 

```py
def smart_deepcopy(obj: Obj) -> Obj:
    """
    Return type as is for immutable built-in types
    Use obj.copy() for built-in empty collections
    Use copy.deepcopy() for non-empty collections and unknown objects
    """

    obj_type = obj.__class__
    if obj_type in IMMUTABLE_NON_COLLECTIONS_TYPES:
        return obj  # fastest case: obj is immutable and not collection therefore will not be copied anyway
    try:
        if not obj and obj_type in BUILTIN_COLLECTIONS:
            # faster way for empty collections, no need to copy its members
            return obj if obj_type is tuple else obj.copy()  # type: ignore  # tuple doesn't have copy method
    except (TypeError, ValueError, RuntimeError):
        # do we really dare to catch ALL errors? Seems a bit risky
        pass
    return deepcopy(obj)  # slowest way when we actually might need a deepcopy
```

What do you think?

PR welcome, but I'm not sure when I'll get around to it - I'm currently flat out on pydantic-core getting ready for pydantic v2.

But at least your issue has made me think about how we deal with deepcopy in v2 😄 .
per their [docs](https://docs.sqlalchemy.org/en/14/changelog/migration_06.html)

> Code that wants to check for the presence of a ClauseElement expression should instead say:
>  `if expression is not None:`
>  `     print("the expression is:", expression)`

So could this perhaps be fixed by changing the smart deep copy line to

```py
if obj is None and obj_type in BUILTIN_COLLECTIONS:
```

Does this method expect other `obj` references that are falsy rather than `None`? if so, perhaps this:

```py
if (obj is None or not obj) and obj_type in BUILTIN_COLLECTIONS:
```
That doesn't work, `None` is already covered by the `IMMUTABLE_NON_COLLECTIONS_TYPES` check above, and this line is using falsy as a proxy for empty on iterable types, so a `None` check is different.
Isn't the immutable collection type check covering `None` on the `obj_type`, not `obj`? Isn't the raise happening on the `not obj` statement where sqlalchemy added the` __bool__` override?
I'm not sure what you mean, but neither of your code suggestions will work. Best to try it and see what happens.
You were right, my train of thought would have required a quite ugly double negative to work. I was only trying to avoid catching errors, but I suspect your proposed approach is the cleanest and safest. Will submit a pr

Resolving patch:
diff --git a/pydantic/utils.py b/pydantic/utils.py
--- a/pydantic/utils.py
+++ b/pydantic/utils.py
@@ -651,9 +651,14 @@ def smart_deepcopy(obj: Obj) -> Obj:
     obj_type = obj.__class__
     if obj_type in IMMUTABLE_NON_COLLECTIONS_TYPES:
         return obj  # fastest case: obj is immutable and not collection therefore will not be copied anyway
-    elif not obj and obj_type in BUILTIN_COLLECTIONS:
-        # faster way for empty collections, no need to copy its members
-        return obj if obj_type is tuple else obj.copy()  # type: ignore  # tuple doesn't have copy method
+    try:
+        if not obj and obj_type in BUILTIN_COLLECTIONS:
+            # faster way for empty collections, no need to copy its members
+            return obj if obj_type is tuple else obj.copy()  # type: ignore  # tuple doesn't have copy method
+    except (TypeError, ValueError, RuntimeError):
+        # do we really dare to catch ALL errors? Seems a bit risky
+        pass
+
     return deepcopy(obj)  # slowest way when we actually might need a deepcopy
 
 


Test patch:
diff --git a/tests/test_utils.py b/tests/test_utils.py
--- a/tests/test_utils.py
+++ b/tests/test_utils.py
@@ -461,6 +461,17 @@ def test_smart_deepcopy_collection(collection, mocker):
     assert smart_deepcopy(collection) is expected_value
 
 
+@pytest.mark.parametrize('error', [TypeError, ValueError, RuntimeError])
+def test_smart_deepcopy_error(error, mocker):
+    class RaiseOnBooleanOperation(str):
+        def __bool__(self):
+            raise error('raised error')
+
+    obj = RaiseOnBooleanOperation()
+    expected_value = deepcopy(obj)
+    assert smart_deepcopy(obj) == expected_value
+
+
 T = TypeVar('T')