Repository: run-llama/llama_index

Issue:
[Bug]: Tool Schema: Annotated pydantic fileds can no longer provide a description
### Bug Description

When using Pydantic fields for tool specification, [it is recommended](https://docs.pydantic.dev/latest/concepts/validation_decorator/#using-the-field-function-to-describe-function-parameters) to use the Field in combination with [Annotated](https://docs.python.org/3/library/typing.html#typing.Annotated). However, the current scheme generation following #17411 can only process strings and no Pydantic Fields for the description extraction. Prior to version 0.12.10, the description of annotated fields of tools was included in the tool function shema, because the whole type including the annotation metadata was passed to the model creation. Now only the annotated type is passed and optional the annotated metadata if it is a string.

Side note: The [llama index documentation](https://docs.llamaindex.ai/en/stable/module_guides/deploying/agents/tools/#:~:text=For%20a%20better%20function%20definition%2C%20you%20can%20also%20leverage%20pydantic%20for%20the%20function%20arguments.) doesn't follow the [Pydantic recommendation](https://docs.pydantic.dev/latest/concepts/validation_decorator/#using-the-field-function-to-describe-function-parameters).

### Version

0.12.14

### Steps to Reproduce

```
from llama_index.core.tools import FunctionTool
def tool_get_weather(
    self,
    location: Annotated[
        str,
        Field(
            description="Location to get weather for.",
        ),
    ],
) -> str:
    """Get the current weather conditions for a specific location."""
    return "Sunny"
tool = FunctionTool.from_defaults(tool_get_weather)
print(tool.metedata.fn_schema.model_fields["location"].description) # Is None, should be "Location to get weather for."
```

### Relevant Logs/Tracbacks

```shell

```

Hints from issue comments:
@dosubot: <!-- Greeting -->
@Dranero Hello! I'm here to help you with any bugs, questions, or contributions you have. Let's tackle this issue together!

<!-- Answer -->
The issue you're encountering is related to a change in how the schema generation process handles descriptions from `Annotated` Pydantic fields. As of version 0.12.10, the schema generation only processes string metadata for descriptions, and not Pydantic `Field` descriptions. This change was introduced with the update #17411, which aimed to support `typing.Annotated` for adding parameter descriptions but seems to have limited the handling of Pydantic `Field` descriptions <sup>[[1]](https://github.com/run-llama/llama_index/blob/main/CHANGELOG.md)</sup>.

In your example, the description from the `Field` is not being extracted because the current implementation only considers string metadata directly from `Annotated`. This is why the description is `None` instead of "Location to get weather for."


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*

---
<sup>Help Dosu learn! Give it feedback: [Great Response](https://app.dosu.dev/response-feedback/8e0810cb-d586-4c0d-ae90-036c11bd1682?feedback_type=great_response) | [Irrelevant Answer](https://app.dosu.dev/response-feedback/8e0810cb-d586-4c0d-ae90-036c11bd1682?feedback_type=irrelevant_answer) | [Incorrect Sources](https://app.dosu.dev/response-feedback/8e0810cb-d586-4c0d-ae90-036c11bd1682?feedback_type=incorrect_sources) | [Too Verbose](https://app.dosu.dev/response-feedback/8e0810cb-d586-4c0d-ae90-036c11bd1682?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/8e0810cb-d586-4c0d-ae90-036c11bd1682?feedback_type=hallucination) | [Other](https://app.dosu.dev/response-feedback/8e0810cb-d586-4c0d-ae90-036c11bd1682?feedback_type=other)</sup>
@alexander-azizi-martin: Created a pr to fix this https://github.com/run-llama/llama_index/pull/17679

Solution patch:
diff --git a/docs/docs/examples/retrievers/bm25_retriever.ipynb b/docs/docs/examples/retrievers/bm25_retriever.ipynb
index 27443b4ec..7c8cbb47d 100644
--- a/docs/docs/examples/retrievers/bm25_retriever.ipynb
+++ b/docs/docs/examples/retrievers/bm25_retriever.ipynb
@@ -621,7 +621,7 @@
    "source": [
     "storage_context.docstore.persist(\"./docstore.json\")\n",
     "\n",
-    "# or, we could ignore the docstore and just persist the bm25 retriever as shown above\n",
+    "# or, we could ignore the docstore and just persist the bm25 retriever as shown below\n",
     "# bm25_retriever.persist(\"./bm25_retriever\")"
    ]
   },
diff --git a/llama-index-core/llama_index/core/tools/utils.py b/llama-index-core/llama_index/core/tools/utils.py
index 73349fe6e..22e7e8adb 100644
--- a/llama-index-core/llama_index/core/tools/utils.py
+++ b/llama-index-core/llama_index/core/tools/utils.py
@@ -42,6 +42,8 @@ def create_schema_from_function(
             param_type = args[0]
             if isinstance(args[1], str):
                 description = args[1]
+            elif isinstance(args[1], FieldInfo):
+                description = args[1].description
 
         if param_type is params[param_name].empty:
             param_type = Any
diff --git a/llama-index-core/llama_index/core/workflow/context.py b/llama-index-core/llama_index/core/workflow/context.py
index 10ad0eca8..f46f05b8a 100644
--- a/llama-index-core/llama_index/core/workflow/context.py
+++ b/llama-index-core/llama_index/core/workflow/context.py
@@ -1,14 +1,14 @@
 import asyncio
 import json
-import warnings
 import uuid
+import warnings
 from collections import defaultdict
-from typing import Dict, Any, Optional, List, Type, TYPE_CHECKING, Set, Tuple, TypeVar
+from typing import TYPE_CHECKING, Any, Dict, List, Optional, Set, Tuple, Type, TypeVar
 
 from .context_serializers import BaseSerializer, JsonSerializer
 from .decorators import StepConfig
-from .events import Event
 from .errors import WorkflowRuntimeError
+from .events import Event
 
 if TYPE_CHECKING:  # pragma: no cover
     from .workflow import Workflow
@@ -60,7 +60,7 @@ class Context:
         self._lock = asyncio.Lock()
         self._globals: Dict[str, Any] = {}
         # Step-specific instance
-        self._events_buffer: Dict[Type[Event], List[Event]] = defaultdict(list)
+        self._events_buffer: Dict[str, List[Event]] = defaultdict(list)
 
     def _serialize_queue(self, queue: asyncio.Queue, serializer: BaseSerializer) -> str:
         queue_items = list(queue._queue)  # type: ignore
@@ -240,14 +240,17 @@ class Context:
         warnings.warn(msg, DeprecationWarning)
         return self
 
+    def _get_full_path(self, ev_type: Type[Event]) -> str:
+        return f"{ev_type.__module__}.{ev_type.__name__}"
+
     def collect_events(
         self, ev: Event, expected: List[Type[Event]]
     ) -> Optional[List[Event]]:
-        self._events_buffer[type(ev)].append(ev)
+        self._events_buffer[self._get_full_path(type(ev))].append(ev)
 
         retval: List[Event] = []
         for e_type in expected:
-            e_instance_list = self._events_buffer.get(e_type)
+            e_instance_list = self._events_buffer.get(self._get_full_path(e_type))
             if e_instance_list:
                 retval.append(e_instance_list.pop(0))
 
@@ -256,7 +259,7 @@ class Context:
 
         # put back the events if unable to collect all
         for ev in retval:
-            self._events_buffer[type(ev)].append(ev)
+            self._events_buffer[self._get_full_path(type(ev))].append(ev)
 
         return None
 


Test patch:
diff --git a/llama-index-core/tests/tools/test_utils.py b/llama-index-core/tests/tools/test_utils.py
index 6ca980ba7..8b13708e3 100644
--- a/llama-index-core/tests/tools/test_utils.py
+++ b/llama-index-core/tests/tools/test_utils.py
@@ -73,3 +73,25 @@ def test_create_schema_from_function_with_typing_annotated() -> None:
 
     instance = schema(x=5)
     assert instance.x == 5  # type: ignore
+
+
+def test_create_schema_from_function_with_field_annotated() -> None:
+    """Test create_schema_from_function with Annotated[pydantic.Field]."""
+
+    def tmp_function(x: Annotated[int, Field(description="An integer")] = 3) -> str:
+        return str(x)
+
+    schema = create_schema_from_function("TestSchema", tmp_function)
+    actual_schema = schema.model_json_schema()
+
+    assert "x" in actual_schema["properties"]
+    assert actual_schema["properties"]["x"]["type"] == "integer"
+    assert actual_schema["properties"]["x"]["default"] == 3
+    assert actual_schema["properties"]["x"]["description"] == "An integer"
+
+    # Test the created schema
+    instance = schema()
+    assert instance.x == 3  # type: ignore
+
+    instance = schema(x=5)
+    assert instance.x == 5  # type: ignore
diff --git a/llama-index-core/tests/workflow/test_context.py b/llama-index-core/tests/workflow/test_context.py
index 1cb35c237..41afa9476 100644
--- a/llama-index-core/tests/workflow/test_context.py
+++ b/llama-index-core/tests/workflow/test_context.py
@@ -1,18 +1,18 @@
 import asyncio
+import json
+from typing import Optional, Union
 from unittest import mock
-from typing import Union, Optional
 
 import pytest
+from llama_index.core.workflow.decorators import step
+from llama_index.core.workflow.errors import WorkflowRuntimeError
+from llama_index.core.workflow.events import Event, StartEvent, StopEvent
 from llama_index.core.workflow.workflow import (
-    Workflow,
     Context,
+    Workflow,
 )
-from llama_index.core.workflow.decorators import step
-from llama_index.core.workflow.errors import WorkflowRuntimeError
-from llama_index.core.workflow.events import StartEvent, StopEvent, Event
-from llama_index.core.workflow.workflow import Workflow
 
-from .conftest import OneTestEvent, AnotherTestEvent
+from .conftest import AnotherTestEvent, OneTestEvent
 
 
 @pytest.mark.asyncio()
@@ -115,6 +115,11 @@ def test_get_result(ctx):
     assert ctx.get_result() == 42
 
 
+def test_to_dict_with_events_buffer(ctx):
+    ctx.collect_events(OneTestEvent(), [OneTestEvent, AnotherTestEvent])
+    assert json.dumps(ctx.to_dict())
+
+
 @pytest.mark.asyncio()
 async def test_deprecated_params(ctx):
     with pytest.warns(
diff --git a/llama-index-core/tests/workflow/test_workflow.py b/llama-index-core/tests/workflow/test_workflow.py
index 32cde7168..91d6375c8 100644
--- a/llama-index-core/tests/workflow/test_workflow.py
+++ b/llama-index-core/tests/workflow/test_workflow.py
@@ -220,6 +220,9 @@ async def test_workflow_num_workers():
             ctx.session.send_event(OneTestEvent(test_param="test2"))
             ctx.session.send_event(OneTestEvent(test_param="test3"))
 
+            # send one extra event
+            ctx.session.send_event(AnotherTestEvent(another_test_param="test4"))
+
             return LastEvent()
 
         @step(num_workers=3)
@@ -241,11 +244,21 @@ async def test_workflow_num_workers():
     workflow = NumWorkersWorkflow()
 
     start_time = time.time()
-    result = await workflow.run()
+    handler = workflow.run()
+    result = await handler
     end_time = time.time()
 
     assert workflow.is_done()
-    assert set(result) == {"test1", "test2", "test3"}
+    assert set(result) == {"test1", "test2", "test4"}
+
+    # ctx should have 1 extra event
+    assert (
+        len(handler.ctx._events_buffer["tests.workflow.conftest.AnotherTestEvent"]) == 1
+    )
+
+    # ensure ctx is serializable
+    ctx = handler.ctx
+    ctx.to_dict()
 
     # Check if the execution time is close to 1 second (with some tolerance)
     execution_time = end_time - start_time