Repository: deepset-ai/haystack
Pull request #9356: perf: Don't deepcopy Components, Tools, or Toolsets
Author: login: sjrl
Labels: name: type:documentation
description: Improvements on the docs
name: topic:tests
name: topic:core
Created: 2025-05-08T08:13:56Z | Merged: 2025-05-08T12:48:08Z

### Related Issues

- partially address https://github.com/deepset-ai/haystack/issues/9349

### Proposed Changes:

 <!--- In case of a bug: Describe what caused the issue and how you solved it -->
 <!--- In case of a feature: Describe what did you add and how it works -->
In our pipeline run methods we deepcopy the inputs and outputs of components to ensure that we don't cause unintended behavior such as altering the Pipeline run logic unepxectedly when the same input reference is passed to multiple components. 

This has worked well under the assumption that all run time inputs and outputs are data-like objects (e.g. strings, dataclasses, numbers, etc.) but has recently caused issues with the introduction of being able to pass Tools as runtime parameters to ChatGenerators. Since Tools can and often contain Components (e.g. ComponentTool) or even full pipelines (e.g. SuperComponent wrapped by ComponentTool) we find that our Pipeline run logic tries to deepcopy said tool leading to deepcopying of Components, Document Stores, etc. 

This has unsurprisingly caused issues since many of our Components contain attributes that aren't deepcopyable such as Jinja2 templates or some clients (e.g. ProxyClient). Or they contain objects that'd we'd prefer not to deepcopy such as local ML models.  

So I think it makes sense to update the new utility method `deepcopy_with_fallback` to automatically skip over Component, Tool or Toolset if they are passed into the deepcopy. 

### How did you test it?

<!-- unit tests, integration tests, manual verification, instructions for manual tests -->
- Updated unit tests.
- Also manually triggered end to end tests https://github.com/deepset-ai/haystack/actions/runs/14905702825

### Notes for the reviewer

<!-- E.g. point out section where the reviewer  -->

Some performance checks on the deepcopying
```python
import time
import tracemalloc

from haystack.core.pipeline.utils import deepcopy_with_fallback
from haystack.components.rankers.transformers_similarity import TransformersSimilarityRanker
from haystack.tools import ComponentTool
from haystack.utils import ComponentDevice


st = TransformersSimilarityRanker(device=ComponentDevice.from_str("cpu"))
st.warm_up()

tracemalloc.start()
start_time = time.time()

comp_tool = ComponentTool(
    component=st,
    name="text_embedder",
    description="Use this tool to embed text.",
)

_ = deepcopy_with_fallback(
    {
        "tools": [
            comp_tool, comp_tool, comp_tool, comp_tool, comp_tool, comp_tool, comp_tool, comp_tool, comp_tool, comp_tool
        ]
    }
)

end_time = time.time()
current, peak = tracemalloc.get_traced_memory()
tracemalloc.stop()

print(f"Time elapsed: {end_time - start_time:.4f} seconds")
print(f"Current memory usage: {current / 1024**2:.2f} MB; Peak: {peak / 1024**2:.2f} MB")

# When skipping copying of Tool 
# Ten tools
# Time elapsed: 0.0273 seconds
# Current memory usage: 0.14 MB; Peak: 0.16 MB

# When deepcopying Tool
# Ten tools
# Time elapsed: 0.4356 seconds
# Current memory usage: 3.81 MB; Peak: 4.54 MB
```

Another example with a custom component that contains an attribute that causes a circular deepcopy
```python
import time
import tracemalloc

import logging
logging.basicConfig(level=logging.INFO)

from haystack import component
from haystack.core.pipeline.utils import deepcopy_with_fallback
from haystack.tools import ComponentTool
from ddtrace.settings._config import Config


@component
class BadComponent:
    def __init__(self):
        # Object that can't be deepcopied
        self.config = Config()

    @component.output_types(result=str)
    def run(self, inp: str):
        return inp


bad_component = BadComponent()

tracemalloc.start()
start_time = time.time()

comp_tool = ComponentTool(
    component=bad_component,
    name="bad_component",
    description="Use this tool to return text.",
)

# _ = deepcopy_with_fallback({"tools": [comp_tool]})
_ = deepcopy_with_fallback(
    {
        "tools": [
            comp_tool, comp_tool, comp_tool, comp_tool, comp_tool, comp_tool, comp_tool, comp_tool, comp_tool, comp_tool
        ]
    }
)

end_time = time.time()
current, peak = tracemalloc.get_traced_memory()
tracemalloc.stop()

print(f"Time elapsed: {end_time - start_time:.4f} seconds")
print(f"Current memory usage: {current / 1024**2:.2f} MB; Peak: {peak / 1024**2:.2f} MB")

# When using new version
# One tool
# Time elapsed: 0.0033 seconds
# Current memory usage: 0.06 MB; Peak: 0.06 MB
# Ten tools
# Time elapsed: 0.0036 seconds
# Current memory usage: 0.06 MB; Peak: 0.06 MB

# When using old version
# One tool
# Time elapsed: 0.0611 seconds
# Current memory usage: 0.15 MB; Peak: 0.56 MB
# Ten tools
# Time elapsed: 0.6274 seconds
# Current memory usage: 0.15 MB; Peak: 0.57 MB
```

### Checklist

- I have read the [contributors guidelines](https://github.com/deepset-ai/haystack/blob/main/CONTRIBUTING.md) and the [code of conduct](https://github.com/deepset-ai/haystack/blob/main/code_of_conduct.txt)
- I have updated the related issue with new insights and changes
- I added unit tests and updated the docstrings
- I've used one of the [conventional commit types](https://www.conventionalcommits.org/en/v1.0.0/) for my PR title: `fix:`, `feat:`, `build:`, `chore:`, `ci:`, `docs:`, `style:`, `refactor:`, `perf:`, `test:` and added `!` in case the PR includes breaking changes.
- I documented my code
- I ran [pre-commit hooks](https://github.com/deepset-ai/haystack/blob/main/CONTRIBUTING.md#installation) and fixed any issue



Diff:
diff --git a/haystack/components/agents/agent.py b/haystack/components/agents/agent.py
index 7d978ef5a8..b65f046527 100644
--- a/haystack/components/agents/agent.py
+++ b/haystack/components/agents/agent.py
@@ -3,7 +3,6 @@
 # SPDX-License-Identifier: Apache-2.0
 
 import inspect
-from copy import deepcopy
 from typing import Any, Dict, List, Optional, Union
 
 from haystack import component, default_from_dict, default_to_dict, logging, tracing
@@ -11,6 +10,7 @@
 from haystack.components.tools import ToolInvoker
 from haystack.core.pipeline.async_pipeline import AsyncPipeline
 from haystack.core.pipeline.pipeline import Pipeline
+from haystack.core.pipeline.utils import _deepcopy_with_exceptions
 from haystack.core.serialization import component_to_dict
 from haystack.dataclasses import ChatMessage
 from haystack.dataclasses.state import State, _schema_from_dict, _schema_to_dict, _validate_schema
@@ -111,7 +111,7 @@ def __init__(
         self._state_schema = state_schema or {}
 
         # Initialize state schema
-        resolved_state_schema = deepcopy(self._state_schema)
+        resolved_state_schema = _deepcopy_with_exceptions(self._state_schema)
         if resolved_state_schema.get("messages") is None:
             resolved_state_schema["messages"] = {"type": List[ChatMessage], "handler": merge_lists}
         self.state_schema = resolved_state_schema
@@ -244,8 +244,6 @@ def run(
             init_callback=self.streaming_callback, runtime_callback=streaming_callback, requires_async=False
         )
 
-        input_data = deepcopy({"messages": messages, "streaming_callback": streaming_callback, **kwargs})
-
         state = State(schema=self.state_schema, data=kwargs)
         state.set("messages", messages)
 
@@ -253,7 +251,10 @@ def run(
 
         component_visits = dict.fromkeys(["chat_generator", "tool_invoker"], 0)
         with self._create_agent_span() as span:
-            span.set_content_tag("haystack.agent.input", input_data)
+            span.set_content_tag(
+                "haystack.agent.input",
+                _deepcopy_with_exceptions({"messages": messages, "streaming_callback": streaming_callback, **kwargs}),
+            )
             counter = 0
             while counter < self.max_agent_steps:
                 # 1. Call the ChatGenerator
@@ -332,8 +333,6 @@ async def run_async(
             init_callback=self.streaming_callback, runtime_callback=streaming_callback, requires_async=True
         )
 
-        input_data = deepcopy({"messages": messages, "streaming_callback": streaming_callback, **kwargs})
-
         state = State(schema=self.state_schema, data=kwargs)
         state.set("messages", messages)
 
@@ -341,7 +340,10 @@ async def run_async(
 
         component_visits = dict.fromkeys(["chat_generator", "tool_invoker"], 0)
         with self._create_agent_span() as span:
-            span.set_content_tag("haystack.agent.input", input_data)
+            span.set_content_tag(
+                "haystack.agent.input",
+                _deepcopy_with_exceptions({"messages": messages, "streaming_callback": streaming_callback, **kwargs}),
+            )
             counter = 0
             while counter < self.max_agent_steps:
                 # 1. Call the ChatGenerator
diff --git a/haystack/core/pipeline/async_pipeline.py b/haystack/core/pipeline/async_pipeline.py
index 9d7b2d9414..2f4dc4e216 100644
--- a/haystack/core/pipeline/async_pipeline.py
+++ b/haystack/core/pipeline/async_pipeline.py
@@ -15,7 +15,7 @@
     ComponentPriority,
     PipelineBase,
 )
-from haystack.core.pipeline.utils import deepcopy_with_fallback
+from haystack.core.pipeline.utils import _deepcopy_with_exceptions
 from haystack.telemetry import pipeline_running
 
 logger = logging.getLogger(__name__)
@@ -58,7 +58,7 @@ async def _run_component_async(  # pylint: disable=too-many-positional-arguments
         with PipelineBase._create_component_span(
             component_name=component_name, instance=instance, inputs=component_inputs, parent_span=parent_span
         ) as span:
-            span.set_content_tag(_COMPONENT_INPUT, deepcopy_with_fallback(component_inputs))
+            span.set_content_tag(_COMPONENT_INPUT, _deepcopy_with_exceptions(component_inputs))
             logger.info("Running component {component_name}", component_name=component_name)
 
             if getattr(instance, "__haystack_supports_async__", False):
@@ -76,7 +76,7 @@ async def _run_component_async(  # pylint: disable=too-many-positional-arguments
                 raise PipelineRuntimeError.from_invalid_output(component_name, instance.__class__, outputs)
 
             span.set_tag(_COMPONENT_VISITS, component_visits[component_name])
-            span.set_content_tag(_COMPONENT_OUTPUT, deepcopy_with_fallback(outputs))
+            span.set_content_tag(_COMPONENT_OUTPUT, _deepcopy_with_exceptions(outputs))
 
             return outputs
 
@@ -238,7 +238,7 @@ async def _run_highest_in_isolation(component_name: str) -> AsyncIterator[Dict[s
                         partial_result = finished.result()
                         scheduled_components.discard(finished_component_name)
                         if partial_result:
-                            yield_dict = {finished_component_name: deepcopy_with_fallback(partial_result)}
+                            yield_dict = {finished_component_name: _deepcopy_with_exceptions(partial_result)}
                             yield yield_dict  # partial outputs
 
                 if component_name in scheduled_components:
@@ -274,7 +274,7 @@ async def _run_highest_in_isolation(component_name: str) -> AsyncIterator[Dict[s
 
                 scheduled_components.remove(component_name)
                 if pruned:
-                    yield {component_name: deepcopy_with_fallback(pruned)}
+                    yield {component_name: _deepcopy_with_exceptions(pruned)}
 
             async def _schedule_task(component_name: str) -> None:
                 """
@@ -337,7 +337,7 @@ async def _wait_for_one_task_to_complete() -> AsyncIterator[Dict[str, Any]]:
                         partial_result = finished.result()
                         scheduled_components.discard(finished_component_name)
                         if partial_result:
-                            yield {finished_component_name: deepcopy_with_fallback(partial_result)}
+                            yield {finished_component_name: _deepcopy_with_exceptions(partial_result)}
 
             async def _wait_for_all_tasks_to_complete() -> AsyncIterator[Dict[str, Any]]:
                 """
@@ -350,7 +350,7 @@ async def _wait_for_all_tasks_to_complete() -> AsyncIterator[Dict[str, Any]]:
                         partial_result = finished.result()
                         scheduled_components.discard(finished_component_name)
                         if partial_result:
-                            yield {finished_component_name: deepcopy_with_fallback(partial_result)}
+                            yield {finished_component_name: _deepcopy_with_exceptions(partial_result)}
 
             # -------------------------------------------------
             # MAIN SCHEDULING LOOP
@@ -428,7 +428,7 @@ async def _wait_for_all_tasks_to_complete() -> AsyncIterator[Dict[str, Any]]:
                 yield partial_res
 
             # 4) Yield final pipeline outputs
-            yield deepcopy_with_fallback(pipeline_outputs)
+            yield _deepcopy_with_exceptions(pipeline_outputs)
 
     async def run_async(
         self, data: Dict[str, Any], include_outputs_from: Optional[Set[str]] = None, concurrency_limit: int = 4
diff --git a/haystack/core/pipeline/base.py b/haystack/core/pipeline/base.py
index e008d7ce79..c641ccdb0e 100644
--- a/haystack/core/pipeline/base.py
+++ b/haystack/core/pipeline/base.py
@@ -32,7 +32,7 @@
     is_any_greedy_socket_ready,
     is_socket_lazy_variadic,
 )
-from haystack.core.pipeline.utils import FIFOPriorityQueue, deepcopy_with_fallback, parse_connect_string
+from haystack.core.pipeline.utils import FIFOPriorityQueue, _deepcopy_with_exceptions, parse_connect_string
 from haystack.core.serialization import DeserializationCallbacks, component_from_dict, component_to_dict
 from haystack.core.type_utils import _type_name, _types_are_compatible
 from haystack.marshal import Marshaller, YamlMarshaller
@@ -175,7 +175,7 @@ def from_dict(
         :returns:
             Deserialized component.
         """
-        data_copy = deepcopy_with_fallback(data)  # to prevent modification of original data
+        data_copy = _deepcopy_with_exceptions(data)  # to prevent modification of original data
         metadata = data_copy.get("metadata", {})
         max_runs_per_component = data_copy.get("max_runs_per_component", 100)
         connection_type_validation = data_copy.get("connection_type_validation", True)
@@ -894,7 +894,7 @@ def _prepare_component_input_data(self, data: Dict[str, Any]) -> Dict[str, Dict[
         # deepcopying the inputs prevents the Pipeline run logic from being altered unexpectedly
         # when the same input reference is passed to multiple components.
         for component_name, component_inputs in data.items():
-            data[component_name] = {k: deepcopy_with_fallback(v) for k, v in component_inputs.items()}
+            data[component_name] = {k: _deepcopy_with_exceptions(v) for k, v in component_inputs.items()}
 
         return data
 
diff --git a/haystack/core/pipeline/pipeline.py b/haystack/core/pipeline/pipeline.py
index b3f90e7a69..2b4cc29b44 100644
--- a/haystack/core/pipeline/pipeline.py
+++ b/haystack/core/pipeline/pipeline.py
@@ -14,7 +14,7 @@
     ComponentPriority,
     PipelineBase,
 )
-from haystack.core.pipeline.utils import deepcopy_with_fallback
+from haystack.core.pipeline.utils import _deepcopy_with_exceptions
 from haystack.telemetry import pipeline_running
 
 logger = logging.getLogger(__name__)
@@ -54,7 +54,7 @@ def _run_component(
         ) as span:
             # We deepcopy the inputs otherwise we might lose that information
             # when we delete them in case they're sent to other Components
-            span.set_content_tag(_COMPONENT_INPUT, deepcopy_with_fallback(inputs))
+            span.set_content_tag(_COMPONENT_INPUT, _deepcopy_with_exceptions(inputs))
             logger.info("Running component {component_name}", component_name=component_name)
             try:
                 component_output = instance.run(**inputs)
@@ -252,7 +252,7 @@ def run(  # noqa: PLR0915, PLR0912
                 )
 
                 if component_pipeline_outputs:
-                    pipeline_outputs[component_name] = deepcopy_with_fallback(component_pipeline_outputs)
+                    pipeline_outputs[component_name] = _deepcopy_with_exceptions(component_pipeline_outputs)
                 if self._is_queue_stale(priority_queue):
                     priority_queue = self._fill_queue(ordered_component_names, inputs, component_visits)
 
diff --git a/haystack/core/pipeline/utils.py b/haystack/core/pipeline/utils.py
index cc05714af1..c724ad676b 100644
--- a/haystack/core/pipeline/utils.py
+++ b/haystack/core/pipeline/utils.py
@@ -8,45 +8,40 @@
 from typing import Any, List, Optional, Tuple
 
 from haystack import logging
+from haystack.core.component import Component
+from haystack.tools import Tool, Toolset
 
 logger = logging.getLogger(__name__)
 
 
-def deepcopy_with_fallback(obj: Any, max_depth: Optional[int] = 100) -> Any:
+def _deepcopy_with_exceptions(obj: Any) -> Any:
     """
-    Attempts to create a deep copy of the given object with a safe fallback mechanism.
+    Attempts to perform a deep copy of the given object.
 
-    If the object is a dictionary, each value is copied individually using this function recursively.
-    If an individual item fails to be copied, the original value is used as a fallback.
-    A maximum recursion depth is enforced to avoid deep or cyclic structures from causing issues.
+    This function recursively handles common container types (lists, tuples, sets, and dicts) to ensure deep copies
+    of nested structures. For specific object types that are known to be problematic for deepcopying-such as
+    instances of `Component`, `Tool`, or `Toolset` - the original object is returned as-is.
+    If `deepcopy` fails for any other reason, the original object is returned and a log message is recorded.
 
-    :param obj: The object to attempt to deep copy.
-    :param max_depth: The maximum depth to recurse during deep copying. If 0, returns the object as-is.
-    :return: A deep copy of the object if successful; otherwise, the original object.
+    :param obj: The object to be deep-copied.
+
+    :returns:
+        A deep-copied version of the object, or the original object if deepcopying fails.
     """
-    if max_depth is not None and max_depth <= 0:
+    if isinstance(obj, (list, tuple, set)):
+        return type(obj)(_deepcopy_with_exceptions(v) for v in obj)
+
+    if isinstance(obj, dict):
+        return {k: _deepcopy_with_exceptions(v) for k, v in obj.items()}
+
+    # Components and Tools often contain objects that we do not want to deepcopy or are not deepcopyable
+    # (e.g. models, clients, etc.). In this case we return the object as-is.
+    if isinstance(obj, (Component, Tool, Toolset)):
         return obj
 
     try:
         return deepcopy(obj)
     except Exception as e:
-        # If the deepcopy fails we try to deepcopy each individual value if the object is a dictionary
-        next_depth = None if max_depth is None else max_depth - 1
-        if isinstance(obj, dict):
-            logger.info(
-                "Deepcopy failed for object of type '{obj_type}'. Error: {error}. Attempting item-wise copy.",
-                obj_type=type(obj).__name__,
-                error=e,
-            )
-            return {key: deepcopy_with_fallback(value, next_depth) for key, value in obj.items()}
-        elif isinstance(obj, (list, tuple, set)):
-            logger.info(
-                "Deepcopy failed for object of type '{obj_type}'. Error: {error}. Attempting item-wise copy.",
-                obj_type=type(obj).__name__,
-                error=e,
-            )
-            return type(obj)(deepcopy_with_fallback(item, next_depth) for item in obj)
-
         logger.info(
             "Deepcopy failed for object of type '{obj_type}'. Error: {error}. Returning original object instead.",
             obj_type=type(obj).__name__,
diff --git a/releasenotes/notes/dont-deepcopy-components-tools-b1be7ef7b4365116.yaml b/releasenotes/notes/dont-deepcopy-components-tools-b1be7ef7b4365116.yaml
new file mode 100644
index 0000000000..2cda6b5842
--- /dev/null
+++ b/releasenotes/notes/dont-deepcopy-components-tools-b1be7ef7b4365116.yaml
@@ -0,0 +1,7 @@
+---
+enhancements:
+  - |
+    Updated pipeline execution logic to use a new utility method `deepcopy_with_fallback`,
+    which skips deep-copying of `Component`, `Tool`, and `Toolset` instances when used as
+    runtime parameters. This prevents errors and unintended behavior caused by trying to
+    deepcopy objects that contain non-copyable attributes (e.g. Jinja2 templates, clients).
diff --git a/test/core/pipeline/test_utils.py b/test/core/pipeline/test_utils.py
index 976fd2eec0..155ec3a982 100644
--- a/test/core/pipeline/test_utils.py
+++ b/test/core/pipeline/test_utils.py
@@ -6,7 +6,8 @@
 import pytest
 
 from haystack.components.builders.prompt_builder import PromptBuilder
-from haystack.core.pipeline.utils import parse_connect_string, FIFOPriorityQueue, deepcopy_with_fallback
+from haystack.components.generators.chat.openai import OpenAIChatGenerator
+from haystack.core.pipeline.utils import parse_connect_string, FIFOPriorityQueue, _deepcopy_with_exceptions
 from haystack.tools import ComponentTool, Tool
 
 
@@ -184,78 +185,65 @@ def test_queue_ordering_parametrized(empty_queue, items):
         assert empty_queue.pop() == (priority, item)
 
 
+class Copyable:
+    def __init__(self, name="copyable"):
+        self.name = name
+
+
+class NotCopyable:
+    def __init__(self, name="not_copyable"):
+        self.name = name
+
+    def __deepcopy__(self, memo):
+        raise TypeError("This object cannot be deepcopied.")
+
+
 class TestDeepcopyWithFallback:
     def test_deepcopy_with_fallback_copyable(self, caplog):
-        tool = Tool(
-            name="weather",
-            description="Get weather report",
-            parameters={"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]},
-            function=get_weather_report,
-        )
-        original = {"tools": tool}
+        original = {"class": Copyable()}
         with caplog.at_level(logging.INFO):
-            copy = deepcopy_with_fallback(original)
+            copy = _deepcopy_with_exceptions(original)
             assert "Deepcopy failed for object of type" not in caplog.text
-        assert copy["tools"] == original["tools"]
-        # This should be a true copy so changing the name in the copy should not affect the original
-        copy["tools"].name = "copied_tool"
-        assert copy["tools"] != original["tools"]
+        assert copy["class"] is not original["class"]
 
-    def test_deepcopy_with_fallback_not_copyable(self, caplog):
-        problematic_tool = ComponentTool(
-            name="problematic_tool", description="This is a problematic tool.", component=PromptBuilder("{{query}}")
-        )
-        original = {"tools": problematic_tool}
+    def test_deepcopy_with_fallback_not_copyable_error(self, caplog):
+        original = {"class": NotCopyable()}
         with caplog.at_level(logging.INFO):
-            copy = deepcopy_with_fallback(original)
-            assert "Deepcopy failed for object of type 'ComponentTool'" in caplog.text
-        assert copy["tools"] == original["tools"]
-        # Not a true copy but a shallow copy so changing the name in the copy should also affect the original
-        copy["tools"].name = "copied_tool"
-        assert copy["tools"] == original["tools"]
+            copy = _deepcopy_with_exceptions(original)
+            assert "Deepcopy failed for object of type 'NotCopyable'" in caplog.text
+        assert copy["class"] is original["class"]
 
     def test_deepcopy_with_fallback_mixed_copyable_list(self, caplog):
-        tool1 = Tool(
-            name="tool1",
-            description="Get weather report",
-            parameters={"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]},
-            function=get_weather_report,
-        )
-        tool2 = ComponentTool(
-            name="problematic_tool", description="This is a problematic tool.", component=PromptBuilder("{{query}}")
-        )
-        original = {"tools": [tool1, tool2]}
+        obj1 = Copyable()
+        obj2 = NotCopyable()
+        original = {"objects": [obj1, obj2]}
         with caplog.at_level(logging.INFO):
-            copy = deepcopy_with_fallback(original)
-            assert "Deepcopy failed for object of type 'ComponentTool'" in caplog.text
-        assert copy["tools"][0] == original["tools"][0]
-        # First tool should be a true copy so changing the name in the copy should not affect the original
-        copy["tools"][0].name = "copied_tool1"
-        assert copy["tools"][0] != original["tools"][0]
-        assert copy["tools"][1] == original["tools"][1]
-        # second should be a shallow copy so changing the name in the copy should also affect the original
-        copy["tools"][1].name = "copied_tool2"
-        assert copy["tools"][1] == original["tools"][1]
+            copy = _deepcopy_with_exceptions(original)
+            assert "Deepcopy failed for object of type 'NotCopyable'" in caplog.text
+        assert copy["objects"][0] is not original["objects"][0]
+        assert copy["objects"][1] is original["objects"][1]
 
     def test_deepcopy_with_fallback_mixed_copyable_tuple(self, caplog):
-        tool1 = Tool(
-            name="tool1",
-            description="Get weather report",
-            parameters={"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]},
-            function=get_weather_report,
-        )
-        tool2 = ComponentTool(
+        obj1 = Copyable()
+        obj2 = NotCopyable()
+        original = {"objects": (obj1, obj2)}
+        with caplog.at_level(logging.INFO):
+            copy = _deepcopy_with_exceptions(original)
+            assert "Deepcopy failed for object of type 'NotCopyable'" in caplog.text
+        assert copy["objects"][0] is not original["objects"][0]
+        assert copy["objects"][1] is original["objects"][1]
+
+    def test_deepcopy_with_fallback_tool(self):
+        tool = ComponentTool(
             name="problematic_tool", description="This is a problematic tool.", component=PromptBuilder("{{query}}")
         )
-        original = {"tools": (tool1, tool2)}
-        with caplog.at_level(logging.INFO):
-            copy = deepcopy_with_fallback(original)
-            assert "Deepcopy failed for object of type 'ComponentTool'" in caplog.text
-        assert copy["tools"][0] == original["tools"][0]
-        # First tool should be a true copy so changing the name in the copy should not affect the original
-        copy["tools"][0].name = "copied_tool1"
-        assert copy["tools"][0] != original["tools"][0]
-        assert copy["tools"][1] == original["tools"][1]
-        # second should be a shallow copy so changing the name in the copy should also affect the original
-        copy["tools"][1].name = "copied_tool2"
-        assert copy["tools"][1] == original["tools"][1]
+        original = {"tools": tool}
+        copy = _deepcopy_with_exceptions(original)
+        assert copy["tools"] is original["tools"]
+
+    def test_deepcopy_with_fallback_component(self, monkeypatch):
+        monkeypatch.setenv("OPENAI_API_KEY", "test")
+        comp = OpenAIChatGenerator()
+        original = {"component": comp}
+        res = _deepcopy_with_exceptions(original)
+        assert res["component"] is original["component"]
diff --git a/test/tools/test_component_tool.py b/test/tools/test_component_tool.py
index c80bb5f6ae..e2f292fa04 100644
--- a/test/tools/test_component_tool.py
+++ b/test/tools/test_component_tool.py
@@ -19,7 +19,7 @@
 from haystack.components.generators.chat import OpenAIChatGenerator
 from haystack.components.tools import ToolInvoker
 from haystack.components.websearch.serper_dev import SerperDevWebSearch
-from haystack.core.pipeline.utils import deepcopy_with_fallback
+from haystack.core.pipeline.utils import _deepcopy_with_exceptions
 from haystack.dataclasses import ChatMessage, ChatRole, Document
 from haystack.tools import ComponentTool
 from haystack.utils.auth import Secret
@@ -644,14 +644,10 @@ def test_pipeline_component_fails(self):
             ComponentTool(component=component)
 
     def test_deepcopy_with_jinja_based_component(self):
-        # Jinja2 templates throw an Exception when we deepcopy them (see https://github.com/pallets/jinja/issues/758)
-        # When we use a ComponentTool in a pipeline at runtime, we deepcopy the tool
-        # We overwrite ComponentTool.__deepcopy__ to fix this in experimental until a more comprehensive fix is merged.
-        # We track the issue here: https://github.com/deepset-ai/haystack/issues/9011
         builder = PromptBuilder("{{query}}")
         tool = ComponentTool(component=builder)
         result = tool.function(query="Hello")
-        tool_copy = deepcopy_with_fallback(tool)
+        tool_copy = _deepcopy_with_exceptions(tool)
         result_from_copy = tool_copy.function(query="Hello")
         assert "prompt" in result_from_copy
         assert result_from_copy["prompt"] == result["prompt"]


Reviews:
user: login: sjrl
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-08T08:29:17Z
user: login: sjrl
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-08T08:32:49Z
user: login: anakin87
body: The benchmarks look convincing.

I've left some minor comments.
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-08T12:05:48Z
user: login: sjrl
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-08T12:10:38Z
user: login: sjrl
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-08T12:16:13Z
user: login: anakin87
body: Sorry I thought it was easier to introduce types.
At this point, I propose removing typing altogether and going back to `Any` (using `cast` is practically the same).

Other than that, looks good.
state: APPROVED
author_association: MEMBER
submitted_at: 2025-05-08T12:36:01Z

Review comments:
diff_hunk: @@ -24,6 +26,11 @@ def deepcopy_with_fallback(obj: Any, max_depth: Optional[int] = 100) -> Any:
     :param max_depth: The maximum depth to recurse during deep copying. If 0, returns the object as-is.
     :return: A deep copy of the object if successful; otherwise, the original object.
     """
+    # Components and Tools often contain objects that we do not want to deepcopy or are not deepcopyable
+    # (e.g. pytorch models, clients, etc.). In this case we return the object as-is.
+    if isinstance(obj, (Component, Tool, Toolset)):
+        return obj
path: haystack/core/pipeline/utils.py
user: login: sjrl
body: I'm wondering actually now that I think of it more if we should instead create custom `__deepcopy__` functions for each of these classes to just return itself so 
```python
def __deepcopy__(self):
    return self
```
created_at: 2025-05-08T08:29:17Z
author_association: CONTRIBUTOR
diff_hunk: @@ -24,6 +26,11 @@ def deepcopy_with_fallback(obj: Any, max_depth: Optional[int] = 100) -> Any:
     :param max_depth: The maximum depth to recurse during deep copying. If 0, returns the object as-is.
     :return: A deep copy of the object if successful; otherwise, the original object.
     """
+    # Components and Tools often contain objects that we do not want to deepcopy or are not deepcopyable
+    # (e.g. pytorch models, clients, etc.). In this case we return the object as-is.
+    if isinstance(obj, (Component, Tool, Toolset)):
+        return obj
path: haystack/core/pipeline/utils.py
user: login: sjrl
body: However, this could potentially be undesirable in cases users create custom Component, Tool or Toolsets that they would in fact like to deepcopy. 
created_at: 2025-05-08T08:32:49Z
author_association: CONTRIBUTOR
diff_hunk: @@ -8,45 +8,40 @@
 from typing import Any, List, Optional, Tuple
 
 from haystack import logging
+from haystack.core.component import Component
+from haystack.tools import Tool, Toolset
 
 logger = logging.getLogger(__name__)
 
 
-def deepcopy_with_fallback(obj: Any, max_depth: Optional[int] = 100) -> Any:
+def deepcopy_with_fallback(obj: Any) -> Any:
path: haystack/core/pipeline/utils.py
user: login: anakin87
body: Nitpicks:
1. Do you think that `deepcopy_with_fallback` is still an explanatory name? Or can you think of better options?
2. Types: `Any` does not give any guarantees in terms of static type checking.
    What about something like the following?
    ```python
         from typing import TypeVar

         T = TypeVar('T')
         def deepcopy_with_fallback(obj: T) -> T:
    ``` 
    [Relevant StackOverflow thread](https://stackoverflow.com/questions/39817081/typing-any-vs-object)
created_at: 2025-05-08T12:00:04Z
author_association: MEMBER
diff_hunk: @@ -184,78 +185,89 @@ def test_queue_ordering_parametrized(empty_queue, items):
         assert empty_queue.pop() == (priority, item)
 
 
+class Copyable:
+    def __init__(self, name="copyable"):
+        self.name = name
+
+
+class NotCopyable:
+    def __init__(self, name="not_copyable"):
+        self.name = name
+
+    def __deepcopy__(self, memo):
+        raise TypeError("This object cannot be deepcopied.")
+
+
 class TestDeepcopyWithFallback:
     def test_deepcopy_with_fallback_copyable(self, caplog):
-        tool = Tool(
-            name="weather",
-            description="Get weather report",
-            parameters={"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]},
-            function=get_weather_report,
-        )
-        original = {"tools": tool}
+        original = {"class": Copyable()}
         with caplog.at_level(logging.INFO):
             copy = deepcopy_with_fallback(original)
             assert "Deepcopy failed for object of type" not in caplog.text
-        assert copy["tools"] == original["tools"]
+        assert copy["class"].name == original["class"].name
path: test/core/pipeline/test_utils.py
user: login: anakin87
body: probably this and other tests could be simplified using the `is` keyword

`copy is original` should be `True` if the two variables refer to the same object
created_at: 2025-05-08T12:04:02Z
author_association: MEMBER
diff_hunk: @@ -8,45 +8,40 @@
 from typing import Any, List, Optional, Tuple
 
 from haystack import logging
+from haystack.core.component import Component
+from haystack.tools import Tool, Toolset
 
 logger = logging.getLogger(__name__)
 
 
-def deepcopy_with_fallback(obj: Any, max_depth: Optional[int] = 100) -> Any:
+def deepcopy_with_fallback(obj: Any) -> Any:
path: haystack/core/pipeline/utils.py
user: login: sjrl
body: I agree on the types!

As for name I updated to `_deepcopy_with_exceptions`. Is that more clear? I also made it private so we can easily rename/change in the future
created_at: 2025-05-08T12:10:37Z
author_association: CONTRIBUTOR
diff_hunk: @@ -184,78 +185,89 @@ def test_queue_ordering_parametrized(empty_queue, items):
         assert empty_queue.pop() == (priority, item)
 
 
+class Copyable:
+    def __init__(self, name="copyable"):
+        self.name = name
+
+
+class NotCopyable:
+    def __init__(self, name="not_copyable"):
+        self.name = name
+
+    def __deepcopy__(self, memo):
+        raise TypeError("This object cannot be deepcopied.")
+
+
 class TestDeepcopyWithFallback:
     def test_deepcopy_with_fallback_copyable(self, caplog):
-        tool = Tool(
-            name="weather",
-            description="Get weather report",
-            parameters={"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]},
-            function=get_weather_report,
-        )
-        original = {"tools": tool}
+        original = {"class": Copyable()}
         with caplog.at_level(logging.INFO):
             copy = deepcopy_with_fallback(original)
             assert "Deepcopy failed for object of type" not in caplog.text
-        assert copy["tools"] == original["tools"]
+        assert copy["class"].name == original["class"].name
path: test/core/pipeline/test_utils.py
user: login: sjrl
body: thanks! simplified the tests
created_at: 2025-05-08T12:16:13Z
author_association: CONTRIBUTOR

PR comments:
user: login: coveralls
created_at: 2025-05-08T08:17:48Z
body: ## Pull Request Test Coverage Report for [Build 14906629231](https://coveralls.io/builds/73575026)


### Details

* **0** of **0**   changed or added relevant lines in **0** files are covered.
* **11** unchanged lines in **1** file lost coverage.
* Overall coverage increased (+**0.005%**) to **90.417%**

---


|  Files with Coverage Reduction | New Missed Lines | % |
| :-----|--------------|--: |
| [components/agents/agent.py](https://coveralls.io/builds/73575026/source?filename=components%2Fagents%2Fagent.py#L327) | 11 | 90.57% |
<!-- | **Total:** | **11** |  | -->

|  Totals | [![Coverage Status](https://coveralls.io/builds/73575026/badge)](https://coveralls.io/builds/73575026) |
| :-- | --: |
| Change from base [Build 14888597428](https://coveralls.io/builds/73560690): |  0.005% |
| Covered Lines: | 10917 |
| Relevant Lines: | 12074 |

---
##### 💛  - [Coveralls](https://coveralls.io)

author_association: COLLABORATOR
user: login: anakin87
created_at: 2025-05-08T08:37:42Z
body: I have nothing against this idea, if it does not break stuff.

In general, since we are talking about performance, it would be nice to see some (even simple) benchmarks: maybe running one/a few Pipelines with `ComponentTool` before and after this PR.

> ... the way the function is currently set up it will first try to deepcopy inputs and then only if it fails with an exception it will iterate through each key-value pair in this case and try to deepcopy the values. Only then would it skip over each individual tool.

Is it difficult to change this aspect?
author_association: MEMBER
user: login: sjrl
created_at: 2025-05-08T08:44:00Z
body: > Is it difficult to change this aspect?

Hmm it might not be if we decide to only check for a few base types to iterate through like dict, list, tuple, and set. 
author_association: CONTRIBUTOR
user: login: sjrl
created_at: 2025-05-08T09:06:17Z
body: > I have nothing against this idea, if it does not break stuff.
> 
> In general, since we are talking about performance, it would be nice to see some (even simple) benchmarks: maybe running one/a few Pipelines with `ComponentTool` before and after this PR.
> 
> > ... the way the function is currently set up it will first try to deepcopy inputs and then only if it fails with an exception it will iterate through each key-value pair in this case and try to deepcopy the values. Only then would it skip over each individual tool.
> 
> Is it difficult to change this aspect?

@anakin87 I was able to refactor this to solve this issue
author_association: CONTRIBUTOR
user: login: sjrl
created_at: 2025-05-08T09:15:18Z
body: And I agree on adding performance benchmarks. This isn't urgent since the original version of `deepcopy_with_fallback` should already unblock errors we've been seeing, just wanted to get this out while it was fresh in my head. 
author_association: CONTRIBUTOR
user: login: sjrl
created_at: 2025-05-08T11:45:03Z
body: @anakin87 I've added some performance examples.

My interpretation of the results is that the performance impact linearly scales with the number of problematic components provided through tools. My examples are relatively simple but show how going from one problematic tool to ten can cause a noticeable delay. In this case the delay would happen every time the pipeline is run. 

And this problem only gets worse and worse if users provide SuperComponents as Tools or Agents as Tools and we get a continuing nested nature of components getting copied. 
author_association: CONTRIBUTOR