Repository: PrefectHQ/prefect

Issue:
`AttributeError` when flow copies a Pydantic object allowing extra fields
### First check

- [X] I added a descriptive title to this issue.
- [X] I used the GitHub search to find a similar issue and didn't find it.
- [X] I searched the Prefect documentation for this issue.
- [X] I checked that this issue is related to Prefect and not one of its dependencies.

### Bug summary

Run the example flow below. The `main` flow will fail and report `AttributeError: 'Something' object has no attribute 'b'`.

If, instead of copying the Pydantic object, I create a new one with the fields that I need, then the flow will run correctly.

### Reproduction

```python3
from prefect import flow
from pydantic import BaseModel, Extra


class Something(BaseModel, extra=Extra.allow):
    a: int


@flow(name="main")
def main_flow() -> Something:
    o1 = Something(a=1, b=2)
    o2 = o1.copy(exclude={"b"})
    # Works if line above changed to
    # o2 = Something(**o1.dict(exclude={"b"}))
    return o2


if __name__ == "__main__":
    main_flow()
```


### Error

```python3
15:43:02.359 | INFO    | prefect.engine - Created flow run 'spotted-rottweiler' for flow 'main'
15:43:03.251 | ERROR   | Flow run 'spotted-rottweiler' - Crash detected! Execution was interrupted by an unexpected exception: AttributeError: 'Something' object has no attribute 'b'

Traceback (most recent call last):
  File "/home/tcoleman/source/stack-model/notebooks/test_pydantic.py", line 19, in <module>
    main_flow()
  File "/home/tcoleman/miniconda3/envs/stack-model/lib/python3.9/site-packages/prefect/flows.py", line 448, in __call__
    return enter_flow_run_engine_from_flow_call(
  File "/home/tcoleman/miniconda3/envs/stack-model/lib/python3.9/site-packages/prefect/engine.py", line 164, in enter_flow_run_engine_from_flow_call
    return anyio.run(begin_run)
  File "/home/tcoleman/miniconda3/envs/stack-model/lib/python3.9/site-packages/anyio/_core/_eventloop.py", line 70, in run
    return asynclib.run(func, *args, **backend_options)
  File "/home/tcoleman/miniconda3/envs/stack-model/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 292, in run
    return native_run(wrapper(), debug=debug)
  File "/home/tcoleman/miniconda3/envs/stack-model/lib/python3.9/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/home/tcoleman/miniconda3/envs/stack-model/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete
    return future.result()
  File "/home/tcoleman/miniconda3/envs/stack-model/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 287, in wrapper
    return await func(*args)
  File "/home/tcoleman/miniconda3/envs/stack-model/lib/python3.9/site-packages/prefect/client/utilities.py", line 47, in with_injected_client
    return await fn(*args, **kwargs)
  File "/home/tcoleman/miniconda3/envs/stack-model/lib/python3.9/site-packages/prefect/engine.py", line 237, in create_then_begin_flow_run
    state = await begin_flow_run(
  File "/home/tcoleman/miniconda3/envs/stack-model/lib/python3.9/site-packages/prefect/engine.py", line 372, in begin_flow_run
    terminal_or_paused_state = await orchestrate_flow_run(
  File "/home/tcoleman/miniconda3/envs/stack-model/lib/python3.9/site-packages/prefect/engine.py", line 679, in orchestrate_flow_run
    await resolve_futures_to_states(result),
  File "/home/tcoleman/miniconda3/envs/stack-model/lib/python3.9/site-packages/prefect/futures.py", line 344, in resolve_futures_to_states
    return await run_sync_in_worker_thread(
  File "/home/tcoleman/miniconda3/envs/stack-model/lib/python3.9/site-packages/prefect/utilities/asyncutils.py", line 91, in run_sync_in_worker_thread
    return await anyio.to_thread.run_sync(
  File "/home/tcoleman/miniconda3/envs/stack-model/lib/python3.9/site-packages/anyio/to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/home/tcoleman/miniconda3/envs/stack-model/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "/home/tcoleman/miniconda3/envs/stack-model/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "/home/tcoleman/miniconda3/envs/stack-model/lib/python3.9/site-packages/prefect/utilities/collections.py", line 298, in visit_collection
    items = [visit_nested(getattr(expr, key)) for key in model_fields]
  File "/home/tcoleman/miniconda3/envs/stack-model/lib/python3.9/site-packages/prefect/utilities/collections.py", line 298, in <listcomp>
    items = [visit_nested(getattr(expr, key)) for key in model_fields]
AttributeError: 'Something' object has no attribute 'b'
```


### Versions

```Text
Version:             2.7.6
API version:         0.8.4
Python version:      3.9.13
Git commit:          528c8300
Built:               Wed, Jan 4, 2023 5:27 PM
OS/Arch:             linux/x86_64
Profile:             cloud
Server type:         cloud
```


### Additional context

_No response_


Discussion:
Looking at the Pydantic object here, I found that:
```python
print(o2.__fields_set__)
# {'a', 'b'}
```
According to the [Pydantic docs](https://docs.pydantic.dev/usage/models/#model-properties), `__fields_set__` is the set of field names when the model instance was instantiated. This is true here, i.e. `o1` was instantiated with `b`, but is unhelpful when inspecting `o2` where the `b` field has been removed.
Thanks for the clear issue! In my opinion, this is a Pydantic bug as this should not be in `__fields_set__` on the copied object. We can guard against this though, I guess we'll have to add an extra `hasattr` check to the `visit_nested` list comprehension.

Contributions are welcome :)

Resolving patch:
diff --git a/src/prefect/utilities/collections.py b/src/prefect/utilities/collections.py
--- a/src/prefect/utilities/collections.py
+++ b/src/prefect/utilities/collections.py
@@ -327,7 +327,10 @@ def visit_expression(expr):
         # NOTE: This implementation *does not* traverse private attributes
         # Pydantic does not expose extras in `__fields__` so we use `__fields_set__`
         # as well to get all of the relevant attributes
-        model_fields = expr.__fields_set__.union(expr.__fields__)
+        # Check for presence of attrs even if they're in the field set due to pydantic#4916
+        model_fields = {
+            f for f in expr.__fields_set__.union(expr.__fields__) if hasattr(expr, f)
+        }
         items = [visit_nested(getattr(expr, key)) for key in model_fields]
 
         if return_data:


Test patch:
diff --git a/tests/utilities/test_collections.py b/tests/utilities/test_collections.py
--- a/tests/utilities/test_collections.py
+++ b/tests/utilities/test_collections.py
@@ -240,6 +240,7 @@ def test_visit_collection_and_transform_data(self, inp, expected):
             (SimpleDataclass(x=1, y=2), {2}),
             (SimplePydantic(x=1, y=2), {2}),
             (ExtraPydantic(x=1, y=2, z=4), {2, 4}),
+            (ExtraPydantic(x=1, y=2, z=4).copy(exclude={"z"}), {2}),
             (ExampleAnnotation(4), {4}),
         ],
     )