generative-ai-toolkit 0.23.0
Toolkit to support developers in building and operating high quality LLM-based applications
License: Apache-2.0
Requires Python: >=3.13
Keywords: generative-ai-toolkit, generative-ai, gen-ai, agent, llm, testing

# Generative AI Toolkit

The **Generative AI Toolkit** is a lightweight library for building, testing and evaluating AI agents in Python, using any of the LLMs supported by the Amazon Bedrock Converse API.

Compared to other libraries out there, the Generative AI Toolkit indexes heavily on production observability, tracing, testing and evaluation, and simplicity of deployment on AWS. A typical production-grade deployment uses just AWS Lambda (or ECS, EKS), Amazon DynamoDB and Amazon CloudWatch.

**Sample usage**

```python
from generative_ai_toolkit.agent import BedrockConverseAgent
from generative_ai_toolkit.context import AgentContext
from generative_ai_toolkit.test import Case, Expect

agent = BedrockConverseAgent(
    model_id="eu.amazon.nova-micro-v1:0",
    system_prompt="You are a helpful assistant. Use your tools to help the user.",
)


def weather_report(city_name: str) -> str:
    """
    Gets the current weather report for a given city

    Parameters
    ------
    city_name: string
      The name of the city
    """

    # Example of how to add tracing to your tool implementations
    tracer = AgentContext.current().tracer

    with tracer.trace("inside-weather-report") as span:
        span.add_attribute(
            "attribute_name", {"Attribute values": ["can be any Python object"]}
        )

        # Tool response
        return "Sunny"


agent.register_tool(weather_report)

# Send a message to the agent and have it stream its response back:
for chunk in agent.converse_stream("What's the weather like right now in Amsterdam?"):
    print(chunk, end="")

# Assert that the weather_report tool was used (raises an error if not):
Expect(agent.traces).tool_invocations.to_include("weather_report")

# Similar, but using test case with 2 turns:
test_case = Case(user_inputs=["What's the weather like right now?", "In Amsterdam"])
traces = test_case.run(agent)
Expect(traces).tool_invocations.to_include("weather_report").with_output("Sunny")

# Start a new conversation, with empty history:
agent.reset()

# Stream traces instead of text chunks:
# (Note: this also yields relevant snapshots of traces that are still underway)
for trace in agent.converse_stream(
    "What's the weather like right now in Amsterdam?", stream="traces"
):
    print(trace.as_human_readable())

# Trace attributes are OpenTelemetry compatible but preserve fidelity:
tool_trace = next(
    trace for trace in agent.traces if trace.span_name == "inside-weather-report"
)
assert tool_trace.attributes["attribute_name"] == {
    "Attribute values": ["can be any Python object"]
}

# You can also collect metrics, run tests in parallel, compare different model ids
# and other agent parameters against each other, view metrics and traces in a UI,
# add your own tracers, use an LLM mock, expose the agent over HTTP, and much more.
...
```

**Major features**

- **Traces** are front and center in the Generative AI Toolkit––everything your agent does can be traced. Traces for the current conversation can be accessed with `agent.traces`, which returns traces from subagents too (recursively). This simplifies testing and visualization of your single or multi-agent architectures. Out-of-the-box tracers for DynamoDB and AWS X-Ray are included, and it's easy to add custom tracers. In automated tests you can use the `Expect()` class to express your test assertions against the traces.
- For **evaluation**, use the out-of-the-box metrics such as cost, latency, cosine similarity, conciseness, or add your own custom metrics. The metrics you use while developing your agent, can continuously be run against your deployed agent in production too (asynchronously). If your agent's performance degrades, you will know! You have the full power of Amazon CloudWatch Metrics available to you to define alarms and thresholds, and catch anomalies.
- Helpers for **mocking** the Amazon Bedrock Converse API, to create unit tests and partially mocked integration tests that are deterministic and fast. The mock supports multi-turn conversation, and dynamic response generation. With the mock you can develop and run your agent locally without needing access to an actual LLM
- Integrates with **[Amazon Bedrock AgentCore](https://aws.amazon.com/bedrock/agentcore/)**
- The testing and evaluation capabilities can be used for agents and LLM-based applications created with **other libraries** (for example [Strands](https://strandsagents.com/latest/documentation/docs/)).

Interested? Please read on. And check out our research paper: [GENERATIVE AI TOOLKIT- INCREASING THE QUALITY OF LLM-BASED APPLICATIONS OVER THEIR WHOLE LIFE CYCLE](https://arxiv.org/abs/2412.14215).

## Screenshots

You can view the traces, as well as the collected metrics, in various developer friendly ways, e.g. with the web UI:

<img src="./assets/images/ui-conversation.png" alt="UI Conversation Display Screenshot" title="UI Conversation Display" width="1200"/>

Traces are also visible in AWS X-Ray (or any other OpenTelemetry compatible tool of your choosing):

<img src="./assets/images/x-ray-trace-map.png" alt="AWS X-Ray Trace Map Screenshot" title="AWS X-Ray Trace Map" width="1200"/>

Details of the traces, such as the LLM inputs and outputs, are visible in the trace timelines as metadata:

<img src="./assets/images/x-ray-trace-segments-timeline.png" alt="AWS X-Ray Trace Segments Timeline Screenshot" title="AWS X-Ray Trace Segments Timeline" width="1200"/>

Metrics can be emitted to Amazon CloudWatch easily, so you can create dashboards and alarms there, and tap into the full power of Amazon CloudWatch for observability:

<img src="./assets/images/sample-metric.png" alt="Sample Amazon Cloud Metric" width="1200" />

## Reference Architecture

The following is a reference architecture for a setup that uses the Generative AI Toolkit to implement an agent, collect traces, and run automated evaluation. The resulting metrics are fed back to the agent's developers via dashboards and alerts. Metrics are calculated and captured continuously, as real users interact with the agent, thereby giving the agent's developers insight into how the agent is performing at all times, allowing for continuous improvement:

<img src="./assets/images/architecture.drawio.png" alt="Architecture" width="1200" />

> Also see our **sample notebook [deploying_on_aws.ipynb](/examples/deploying_on_aws.ipynb)**.

## Key Terms

To fully utilize the Generative AI Toolkit, it’s essential to understand the following key terms:

- **Traces**: Traces are records of the internal operations of your LLM-based application, e.g. LLM invocations and tool invocations. Traces capture the entire request-response cycle, including input prompts, model outputs, tool calls, and metadata such as latency, token usage, and execution details. Traces form the foundation for evaluating an LLM-based application's behavior and performance.

- **Metrics**: Metrics are measurements derived from traces that evaluate various aspects of an LLM-based application's performance. Examples include latency, token usage, similarity with expected responses, sentiment, and cost. Metrics can be customized to measure specific behaviors or to enforce validation rules.

- **Cases**: Cases are repeatable test inputs that simulate conversations with the agent, e.g. for the purpose of agent evaluation. They consist of a sequence of user inputs and expected agent behaviors or outcomes. Cases are used to validate the agent's responses against defined expectations, ensuring consistent performance across scenarios.

- **Agents**: An agent is an implementation of an LLM-based application that processes user inputs and generates responses. The toolkit provides a simple and extensible agent implementation with built-in support for tracing and tool integration.

- **Tools**: Tools are external functions or APIs that agents can invoke to provide additional capabilities (e.g., fetching weather data or querying a database). Tools are registered with agents and seamlessly integrated into the conversation flow.

- **Conversation History**: This refers to the sequence of messages exchanged between the user and the agent. It can be stored in memory or persisted to external storage, such as DynamoDB, to maintain context across sessions.

- **CloudWatch Custom Metrics**: These are metrics logged to Amazon CloudWatch in Embedded Metric Format (EMF), enabling the creation of dashboards, alarms, and aggregations to monitor agent performance in production environments.

- **Web UI**: A local web-based interface that allows developers to inspect traces, debug conversations, and view evaluation results interactively. This is particularly useful for identifying and resolving issues in the agent's responses.

## Table of Contents

2.1 [Installation](#21-installation)  
2.2 [Agent Implementation](#22-agent-implementation)  
 2.2.1 [Chat with agent](#221-chat-with-agent)  
 2.2.2 [Conversation history](#222-conversation-history)  
 2.2.3 [Reasoning and other Bedrock Converse Arguments](#223-bedrock-converse-arguments)  
 2.2.4 [Tools](#224-tools)  
 2.2.5 [Multi-agent support](#225-multi-agent-support)  
 2.2.6 [Tracing](#226-tracing)  
2.3 [Evaluation Metrics](#23-evaluation-metrics)  
2.4 [Repeatable Cases](#24-repeatable-cases)  
2.5 [Cases with Dynamic Expectations](#25-cases-with-dynamic-expectations)  
2.6 [Generating Traces: Running Cases in Bulk](#26-generating-traces-running-cases-in-bulk)  
2.7 [CloudWatch Custom Metrics](#27-cloudwatch-custom-metrics)  
2.8 [Deploying and Invoking the BedrockConverseAgent](#28-deploying-and-invoking-the-bedrockconverseagent)  
2.9 [Web UI for Conversation Debugging](#29-web-ui-for-conversation-debugging)  
2.10 [Mocking and Testing](#210-mocking-and-testing)  
2.11 [Model Context Protocol (MCP) Client](#211-model-context-protocol-mcp-client)

### 2.1 Installation

Install `generative_ai_toolkit` with support for all features, amongst which interactive evaluation of metrics:

```bash
pip install "generative-ai-toolkit[all]"  # Note the [all] modifier
```

If you don't use the `[all]` installation modifier, only the minimal set of dependencies will be included that you'll need for creating an agent.

Other available modifiers are:

- `[run-agent]`: includes dependencies such as `gunicorn` that allow you to use `generative_ai_toolkit.run.agent.Runner` to expose your agent over HTTP.
- `[evaluate]`: includes dependencies that allow you to run evaluations against traces.

### 2.2 Agent implementation

The heart of the Generative AI Toolkit are the traces it collects, that are the basis for evaluations (explained below). The toolkit includes a simple agent implementation that is backed by the [Amazon Bedrock Converse API](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html) and that is instrumented to collect traces in the right format.

A benefit of using this agent implementation, is that you can run the agent locally––it doesn't require any AWS deployment at all and only needs Amazon Bedrock model access. You can quickly iterate and try different agent settings, such as the backing LLM model id, system prompt, temperature, tools, etc. You can create repeatable test cases and run extensive and rigorous evaluations locally.

> We'll first explain how our agent implementation works. Feel free to directly skip to the explanation of [Tracing](#23-tracing) or [Metrics](#24-metrics) instead.

The Generative AI Toolkit Agent implementation is simple and lightweight, and makes for a no-nonsense developer experience. You can easily instantiate and converse with agents while working in the Python interpreter (REPL) or in a notebook:

```python
from generative_ai_toolkit.agent import BedrockConverseAgent

agent = BedrockConverseAgent(
    model_id="anthropic.claude-3-sonnet-20240229-v1:0",
)
```

Obviously right now this agent doesn't have any tools yet (we'll add some shortly), but you can already chat with it.

#### 2.2.1 Chat with agent

Use `converse()` to chat with the agent. You pass the user's input to this function, and it will return the agent's response as string:

```python
response = agent.converse("What's the capital of France?")
print(response) # "The capital of France is Paris."
```

##### Response streaming

You can also use `converse_stream()` to chat with the agent. You pass the user's input to this function, and it will return an iterator that will progressively return the response fragments. You should concatenate these fragments to collect the full response.

The benefit over using `converse()` is that you can show the user the agent's response tokens as they're being generated, instead of only showing the full response at the very end:

```python
for fragment in agent.converse_stream("What's the capital of France?"):
    print(fragment)
```

That example might now print several lines to the console, for each set of tokens received, e.g.:

```
The
 capital
 of France is
 Paris.
```

#### 2.2.2 Conversation history

The agent maintains the conversation history, so e.g. after the question just asked, this would now work:

```python
response = agent.converse("What are some touristic highlights there?") # This goes back to what was said earlier in the conversation
print(response) # "Here are some of the major tourist highlights and attractions in Paris, France:\n\n- Eiffel Tower - One of the most famous monuments ..."
```

By default conversation history is stored in memory only. If you want to use conversation history across different process instantiations, you need conversation history that is persisted to durable storage.

##### Persisting conversation history

You can use the `DynamoDbConversationHistory` class to persist conversations to DynamoDB. Conversation history is maintained per conversation ID. The agent will create a new conversation ID automatically:

```python
from generative_ai_toolkit.agent import BedrockConverseAgent
from generative_ai_toolkit.conversation_history import DynamoDbConversationHistory

agent = BedrockConverseAgent(
    model_id="anthropic.claude-3-sonnet-20240229-v1:0",
    conversation_history=DynamoDbConversationHistory(table_name="conversations") # This table needs to exist, with string keys "pk" and "sk"
)

print(agent.conversation_id) # e.g.: "01J5D9ZNK5XKZX472HC81ZYR5P"

agent.converse("What's the capital of France?") # This message, and the agent's response, will now be stored in DynamoDB under conversation ID "01J5D9ZNK5XKZX472HC81ZYR5P"
```

Then later, in another process, if you want to continue this conversation, set the conversation ID first:

```python
from generative_ai_toolkit.agent import BedrockConverseAgent
from generative_ai_toolkit.conversation_history import DynamoDbConversationHistory

agent = BedrockConverseAgent(
    model_id="anthropic.claude-3-sonnet-20240229-v1:0",
    conversation_history=DynamoDbConversationHistory(table_name="conversations")
)

agent.set_conversation_id("01J5D9ZNK5XKZX472HC81ZYR5P")

response = agent.converse("What are some touristic highlights there?")
print(response) # "Here are some of the major tourist highlights and attractions in Paris, France:\n\n- Eiffel Tower - One of the most famous monuments ..."
```

There is also the `SqliteConversationHistory` class that will store conversations in a local database (by default `conversations.db`), in the current working directory:

```python
from generative_ai_toolkit.agent import BedrockConverseAgent
from generative_ai_toolkit.conversation_history import SqliteConversationHistory

agent = BedrockConverseAgent(
    model_id="anthropic.claude-3-sonnet-20240229-v1:0",
    conversation_history=SqliteConversationHistory()
)
```

##### Viewing the conversation history

You can manually view the conversation history like so:

```python
print(agent.messages)
# [{'role': 'user', 'content': [{'text': "What's the capital of France?"}]}, {'role': 'assistant', 'content': [{'text': 'The capital of France is Paris.'}]}, {'role': 'user', 'content': [{'text': 'What are some touristic ...
```

Conversation history is included automatically in the prompt to the LLM. That is, you only have to provide new user input when you call `converse()` (or `converse_stream()`), but under the hood the agent will include all past messages as well.

This is generally how conversations with LLMs work––the LLM has no memory of the current conversation, you need to provide all past messages, including those from the LLM (the "assistant"), as part of your prompt to the LLM.

##### Starting a fresh conversation

Calling `agent.reset()` starts a new conversation, with empty conversation history:

```python
print(agent.conversation_id)  # e.g.: "01J5D9ZNK5XKZX472HC81ZYR5P"
agent.converse("Hi!")
print(len(agent.messages)) # 2 (user input + agent response)
agent.reset()
print(len(agent.messages)) # 0
print(agent.conversation_id)  # e.g.: "01J5DQRD864TR3BF314CZK8X5B" (changed)
```

##### Subcontexts

Typically, a conversation (identified by the conversation id) and the "context", i.e. the messages sent back and forth to the LLM, are the same. The conversation id is (/may be) externally known, and after you set it on an agent instance with `agent.set_conversation_id(...)`, accessing `agent.messages` only shows the messages of that conversation. If you send a new message to the agent with `agent.converse()` it is added to the conversation, and then sent, with all prior messages (including assistant responses) to the agent: the "context". The context thus keeps growing as the conversation progresses.

But what if, during a larger conversation, you wanted to ask a side question to your agent? What if that side question would not require the full context to be sent to the LLM? What if you wanted to send a smaller more targeted context to the LLM to increase your chances of a good response?

In such cases you could of course start a new conversation all together, by setting a new conversation id. But the side conversation, the "subcontext", is in a way part of the main conversation. You may want it to have the same conversation id, so that it's easy to query and see all the messages for the entire conversation, including its side conversations. For that purpose, you can use a **subcontext**.

This is precisely what is needed in multi-agent scenarios, where a supervisor agent invokes multiple subagents, potentially in parallel. Each subagent invocation should maintain its own independent conversational memory.

**Note:** Subcontexts are managed automatically when you use agents as tools (see [Multi-agent support](#225-multi-agent-support)). The example below is provided to help you understand how subcontexts work, but you typically would not create and manage subcontexts manually like this:

```python
# No subcontext yet
agent.set_conversation_id("01J5D9ZNK5XKZX472HC81ZYR5Z")
agent.converse("Hi there!") # This send the message to the LLM and collects the agent's response
print(len(agent.messages))  # 2 (user message + agent response)

# Set both conversation ID and subcontext ID
agent.set_conversation_id("01J5D9ZNK5XKZX472HC81ZYR5Z", subcontext_id="subcontext-123")

# Access the current subcontext ID
print(agent.subcontext_id)  # "subcontext-123"
print(len(agent.messages))  # 0 (fresh conversational memory in this subcontext)

# Have a conversation in this subcontext
agent.converse("My favorite color is blue")
print(len(agent.messages))  # 2 (user message + agent response)

# Switch to a different subcontext within the same conversation
agent.set_conversation_id("01J5D9ZNK5XKZX472HC81ZYR5Z", subcontext_id="subcontext-456")
print(len(agent.messages))  # 0 (fresh conversational memory in this subcontext)

# This subcontext has its own separate history
agent.converse("My favorite color is red")
print(len(agent.messages))  # 2

# Switch back to the first subcontext
agent.set_conversation_id("01J5D9ZNK5XKZX472HC81ZYR5Z", subcontext_id="subcontext-123")
print(len(agent.messages))  # 2 (the original messages about blue are still here)
```

Subcontexts allow:

- Parallel subagent invocations with separate memory
- Sequential subagent invocations without carrying over prior messages
- Follow-up messages to the same subagent using the same subcontext ID

##### Multi-modal messages

To send multi-modal messages (image, video, documents) to the agent, use `add_message()` on the agent's conversation history:

```python
image = open("/path/to/image", "rb").read()

agent.conversation_history.add_message(
    {
        "role": "user",
        "content": [
            {"image": {"format": "png", "source": {"bytes": image}}}
        ],
    }
)

# Then, when you chat with the agent, it will include the message you added to the LLM invocation:
agent.converse("Describe the image please")
```

#### 2.2.3 Bedrock Converse Arguments

Upon instantiating the `BedrockConverseAgent` you can pass any arguments that the Bedrock Converse API accepts, and these will be used for all invocations of the Converse API by the agent. You could for example specify usage of [Amazon Bedrock Guardrails](https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails.html):

```python
from generative_ai_toolkit.agent import BedrockConverseAgent

agent = BedrockConverseAgent(
    model_id="anthropic.claude-3-sonnet-20240229-v1:0",
    system_prompt="your system prompt",
    max_tokens=500,
    temperature=0.0,
    top_p=0.8,
    stop_sequences=["stop sequence"],
    guardrail_identifier="guardrail-id",
    guardrail_version="guardrail-version",
    guardrail_trace="enabled_full",
    guardrail_stream_processing_mode="async",
    additional_model_request_fields={"foo": "bar"},
    prompt_variables={"foo": {"text": "bar"}},
    additional_model_response_field_paths=["/path"],
    request_metadata={"foo": "bar"},
    performance_config={"latency": "optimized"},
)
```

##### Reasoning

If you want to use reasoning with a model that supports it (e.g. `anthropic.claude-3-7-sonnet-20250219-v1:0`), specify `additional_model_request_fields`:

```python
from generative_ai_toolkit.agent import BedrockConverseAgent

agent = BedrockConverseAgent(
    model_id="anthropic.claude-3-7-sonnet-20250219-v1:0",
    additional_model_request_fields={
        "reasoning_config": {"type": "enabled", "budget_tokens": 1024}
    },
)
```

Then, when calling `converse` or `converse_stream`, reasoning texts will be included within `<thinking>` tags in the output:

```python
response = agent.converse("How should I make Spaghetti Carbonara?")
print(response)
```

Would print e.g.:

```
<thinking>
The user is asking for a recipe for Spaghetti Carbonara. I have a tool available called `get_recipe` that can provide recipes.

The required parameter for this function is:
- dish: The name of the dish to get a recipe for

In this case, the dish is "Spaghetti Carbonara". This is clearly stated in the user's request, so I can call the function with this parameter.
</thinking>

I can help you with a recipe for Spaghetti Carbonara! Let me get that for you.
Here is the recipe ...
```

If you do not want to include the reasoning texts in the output, you can turn that off like so:

```python
from generative_ai_toolkit.agent import BedrockConverseAgent

agent = BedrockConverseAgent(
    model_id="anthropic.claude-3-7-sonnet-20250219-v1:0",
    additional_model_request_fields={
        "reasoning_config": {"type": "enabled", "budget_tokens": 1024}
    },
    include_reasoning_text_within_thinking_tags=False, # set this to False
)
```

#### 2.2.4 Tools

If you want to give the agent access to tools, you can define them as Python functions, and register them with the agent. Your Python function must have type annotations for input and output, and a docstring like so:

```python
def weather_report(city_name: str) -> str:
    """
    Gets the current weather report for a given city

    Parameters
    ------
    city_name: string
      The name of the city
    """
    return "Sunny" # return a string, number, dict or list --> something that can be turned into JSON

agent.register_tool(weather_report)

response = agent.converse("What's the weather like right now in Amsterdam?")
print(response) # Okay, let me get the current weather report for Amsterdam using the available tool: The weather report for Amsterdam shows that it is currently sunny there.
```

As you can see, tools that you've registered will be invoked automatically by the agent. The output from `converse` is always just a string with the agent's response to the user.

##### Multi-modal responses

Tools can return multi-modal responses (image, video, documents) as well. If you want that, your tool response should match the format expected by the Amazon Bedrock Converse API:

```python
from mypy_boto3_bedrock_runtime.type_defs import ToolResultContentBlockUnionTypeDef  # Optional, to help you with coding

def get_image() -> list[ToolResultContentBlockUnionTypeDef]:
    """
    Read image from disk
    """

    image = open("/path/to/image", "rb").read()

    return [{"image": {"format": "png", "source": {"bytes": image}}}]

agent.register_tool(get_image)
```

See more examples in our test suite [here](/tests/integration/test_tool_multi_modal.py).

##### Other tools

If you don't want to register a Python function as tool, but have a tool with tool spec ready, you can also use it directly, as long as your tool satisfies the `Tool` protocol, i.e. has this shape:

```python
from typing import Any
from mypy_boto3_bedrock_runtime.type_defs import ToolSpecificationTypeDef

class MyTool:

    @property
    def tool_spec(self) -> ToolSpecificationTypeDef:
        return {"name":"my-tool","description":"This tool helps with ...", "inputSchema": {...}}

    def invoke(self, *args, **kwargs) -> Any:
        return "Tool response"

agent.register_tool(MyTool())
```

It's also possible to provide the tool spec explicitly alongside your plain Python function:

```python
agent.register_tool(
    lambda preferred_weather: f"Not {preferred_weather}",
    tool_spec={
        "name": "get_weather",
        "description": "Gets the current weather",
        "inputSchema": {
            "json": {
                "type": "object",
                "properties": {
                    "preferred_weather": {
                        "type": "string",
                        "description": "The preferred weather",
                    },
                },
                "required": [
                    "preferred_weather",
                ],
            }
        },
    },
)
```

##### Tools override

It's possible to set and override the tool selection when calling converse:

```python
def bad_weather_report(city_name: str) -> str:
    """
    Gets the current weather report for a given city

    Parameters
    ------
    city_name: string
      The name of the city
    """
    return "Rainy"

response = agent.converse("What's the weather like right now in Amsterdam?", tools=[bad_weather_report])
print(response) # Okay, let me check the current weather report for Amsterdam using the available tool:\nAccording to the tool, the current weather report for Amsterdam is rainy.
```

Note that this does not force the agent to use the provided tools, it merely makes them available for the agent to use.

##### Tool Development with Pydantic

You can use Pydantic models to define your tool's interface. This approach provides several key benefits:

1. **Clear Interface Documentation**: Input/output schemas are automatically generated from your models. The LLM "reads" both the model's docstring and the `description` attributes of Pydantic Field objects to understand how to use the tool correctly. This natural language documentation helps the LLM make informed decisions about parameter values.
2. **Error Handling with Self-Correction**: Built-in error handling and validation messages are fed back to the LLM, allowing it to understand what went wrong and self-correct its tool usage. For example, if the LLM provides an invalid value for a parameter, Pydantic's detailed error message helps the LLM understand why it was invalid and how to fix it in subsequent attempts.
3. **Strong Type Validation**: Pydantic enforces strict type checking and validation at runtime

You can find a complete example in `examples/pydantic_tools/` that demonstrates this approach. The example implements a weather alerts tool with proper input validation, error handling, and response structuring:

```python
from pydantic import BaseModel, Field

class WeatherAlertRequest(BaseModel):
    """
    Request parameters for the weather alerts tool.
    """
    area: Optional[str] = Field(
        default=None,
        description="State code (e.g., 'CA', 'TX') or zone/county code to filter alerts by area."
    )
    severity: Optional[str] = Field(
        default=None,
        description="Filter by severity level: 'Extreme', 'Severe', 'Moderate', 'Minor', or 'Unknown'.",
        pattern="^(Extreme|Severe|Moderate|Minor|Unknown)$"
    )

class WeatherAlertsTool:
    @property
    def tool_spec(self) -> Dict[str, Any]:
        """Tool specification is automatically generated from the Pydantic model."""
        schema = WeatherAlertRequest.model_json_schema()
        return {
            "name": "get_weather_alerts",
            "description": WeatherAlertRequest.__doc__,
            "inputSchema": {"json": schema}
        }

    def invoke(self, **kwargs) -> Dict[str, Any]:
        """
        Invoke the weather alerts tool with validated parameters.
        """
        try:
            request = WeatherAlertRequest(**kwargs)  # Validation happens here
            return self._get_weather_alerts(request)
        except ValidationError as e:
            return {"error": str(e)}
```

##### Tool Registry

You can organize and discover tools using the `ToolRegistry` and `@tool` decorator. Using the `@tool` decorator can be easier than importing and invoking `agent.register_tool()` for each tool individually.

For example, let's say you have this in `my_tools/weather.py`:

```python
from generative_ai_toolkit.agent import registry

# Use the decorator to register a function with the default tool registry:
@registry.tool
def get_weather(city: str) ->; str:
    """Gets the current weather for a city"""
    return f"Sunny in {city}"

# More tools here, all decorated with @registry.tool
# ...
```

You can then import all modules under `my_tools` and add the tools therein to your agent like so:

```python
from generative_ai_toolkit.agent import BedrockConverseAgent
from generative_ai_toolkit.agent.registry import ToolRegistry, DEFAULT_TOOL_REGISTRY

# You have to import the Python modules with your tools.
# If they are separate .py files in a local folder,
# import the folder and all Python modules in it:
import my_tools
ToolRegistry.recursive_import(my_tools)

# This would have worked too, without needing recursive import,
# but would be inconvenient if there's many such modules:
import my_tools.weather

# Then, use the populated registry upon creating your agent:
agent = BedrockConverseAgent(
    model_id="anthropic.claude-3-sonnet-20240229-v1:0",
    tools=DEFAULT_TOOL_REGISTRY,
)
```

By default the `@tool` decorator adds tools to the `DEFAULT_TOOL_REGISTRY` but you can also add them to a custom registry. This can be convenient in a multi-agent scenario:

```python
from generative_ai_toolkit.agent.registry import ToolRegistry, tool

# Create separate registries for different agents:
weather_registry = ToolRegistry()
finance_registry = ToolRegistry()

# Register tools with specific registries:
@tool(tool_registry=weather_registry)
def get_weather_forecast(city: str) -> str:
    """Gets the weather forecast for a city"""
    return f"Sunny forecast for {city}"

@tool(tool_registry=finance_registry)
def get_stock_price(ticker: str) -> float:
    """Gets the current stock price"""
    return 100.0

# Common tool:
@tool(tool_registry=[weather_registry, finance_registry])
def common_tool(param: str) -> str:
    """A common tool that should be available to both agents"""
    return "common"


# Create specialized agents with their own tool sets:
weather_agent = BedrockConverseAgent(
    model_id="anthropic.claude-3-haiku-20240307-v1:0",
    tools=weather_registry,
)

finance_agent = BedrockConverseAgent(
    model_id="anthropic.claude-3-sonnet-20240229-v1:0",
    tools=finance_registry,
)
```

##### Agent Context in Tools

Tools can access contextual information about the current agent execution through the `AgentContext` class, that has the following shape:

```python
class AgentContext:
    conversation_id: str
    """The conversation ID of the agent"""

    tracer: Tracer
    """The tracer that is used by the agent; tools can use it for adding their own traces"""

    auth_context: AuthContext
    """The auth context; tools can use it for enforcing authentication and authorization"""

    stop_event: Event
    """
    Stop event (threading) that may be set by the user to signal abortion; tools that run for a longer span of time
    should consult the stop event regularly (`stop_event.is_set()`) and abort early if it is set
    """

    @classmethod
    def current(cls) -> "AgentContext":
        """
        Access the current agent context from within a tool invocation
        """
        ...
```

Example usage:

```python
from generative_ai_toolkit.context import AgentContext

def context_aware_tool(some_parameter: str) -> str:
    """
    A tool that demonstrates access to agent context

    Parameters
    ----------
    some_parameter : str
        Some parameter
    """

    # Access the current agent context:
    context = AgentContext.current()

    # Access conversation and authentication information:
    conversation_id = context.conversation_id
    principal_id = context.auth_context["principal_id"]
    other_auth_data = context.auth_context["extra"]["other_auth_data"]

    # Access the tracer to be able to use it from within the tool
    # Add attributes to the current span:
    current_trace = context.tracer.current_trace
    current_trace.add_attribute("foo", "bar")

    # Start a new span:
    with context.tracer.trace("new-span") as trace:
        ...

    # Consult the stop event regularly in long running tasks:
    while True:
        if context.stop_event.is_set():
            raise RuntimeException("Early abort")
        ...

    return "response"

agent.register_tool(context_aware_tool)

# Set context on your agent:
agent.set_conversation_id("01J5D9ZNK5XKZX472HC81ZYR5Z")
agent.set_auth_context(principal_id="john", extra={"other_auth_data":"foo"})

# Now, when the agent invokes the tool during the conversation, the tool can access the context:
agent.converse("Hello!")
```

If you want to use a `stop_event`, create one and pass it to `converse` or `converse_stream`:

```python
stop_event = threading.Event()
for trace in agent.converse_stream("Hello again!", stop_event=stop_event):
    # The stop event that you provided is set onto the agent context
    ...

# At some point in your code, stop the agent and all tool invocations:
stop_event.set()
```

##### Testing Tools that Use AgentContext

When testing tools that depend on `AgentContext.current()`, you can use the `set_test_context()` helper method to set up test fixtures:

```python
import pytest

from generative_ai_toolkit.context import AgentContext

@pytest.fixture
def agent_context():
    return AgentContext.set_test_context()

# Or with custom values:
@pytest.fixture
def custom_agent_context():
    return AgentContext.set_test_context(
        conversation_id="test-conversation",
        AuthContext(principal_id="test", extras={"role": "admin"})
    )

# Example tool that uses context
def example_tool(message: str) -> str:
    """Example tool that accesses agent context"""
    context = AgentContext.current()
    return f"User {context.auth_context['principal_id']} says: {message}"

# Test using the fixture
def test_tool_with_context(agent_context):
    result = example_tool("Hello")
    assert "test" in result
    assert "Hello" in result
```

#### 2.2.5 Multi-Agent Support

Agents can themselves be used as tool too. This allows you to build hierarchical multi-agent systems, where a supervisor agent can use specialized subordinate agents to delegate tasks to.

To use an agent as a tool, the agent must have a `name` and `description`:

```python
# Create a specialized weather agent:
weather_agent = BedrockConverseAgent(
    model_id="anthropic.claude-3-haiku-20240307-v1:0",
    system_prompt="You provide the weather forecast for the specified city.",
    name="transfer_to_weather_agent",  # will be used as the tool name when registered
    description="Get the weather forecast for a city.",  # will be used as the tool description
)

# Add tools to the specialized agent:
def get_weather(city: str):
    """Gets the weather forecast for the provided city"""
    return "Sunny"

weather_agent.register_tool(get_weather)

# Create a supervisor agent that uses the specialized agent:
supervisor = BedrockConverseAgent(
    model_id="anthropic.claude-3-sonnet-20240229-v1:0",
    system_prompt="You provide users with information about cities they want to visit.",
)

# Register the specialized agent as a tool with the supervisor:
supervisor.register_tool(weather_agent)

# The supervisor will delegate to the specialized agent:
response = supervisor.converse("What's the weather like in Amsterdam?")
```

Notes:

- More layers of nesting can be added if desired; a subordinate agent can itself be supervisor to its own set of subordinate agents, etc.
- The above example is obviously contrived; for a more comprehensive example with multiple specialized agents working together, see [multi_agent.ipynb](/examples/multi_agent.ipynb).

##### Input schema

By default, when an agent is used as tool (i.e. as subordinate agent by a supervisor agent), its input schema is:

```json
{
  "type": "object",
  "properties": {
    "user_input": {
      "type": "string",
      "description": "The input to the agent"
    }
  },
  "required": ["user_input"]
}
```

Note: the above schema matches the `converse()` method of the `BedrockConverseAgent`, as that will be used under the hood.

If you want to make sure the agent is called with particular inputs, you can provide an input schema explicitly:

```python
weather_agent = BedrockConverseAgent(
    model_id="anthropic.claude-3-haiku-20240307-v1:0",
    system_prompt="You provide the weather forecast for the specified city.",
    name="transfer_to_weather_agent",  # will be used as the tool name when registered
    description="Get the weather forecast for a city.",  # will be used as the tool description
    input_schema={
        "type": "object",
        "properties": {
            "user_input": {
                "type": "string",
                "description": "The city to get the weather for"
            }
        },
        "required": ["city"]
    }
)
```

Then, when the supervisor invokes the subordinate agent, the supervisor will call the subordinate agent's `converse()` method with `user_input` that includes a (stringified) JSON object, according to the input schema:

```
Your input is:

{"city": "Amsterdam"}
```

So, the `user_input` to the agent will always be a Python `str`, but using an `input_schema` allows you to 'nudge' the LLM (of the supervisor agent) to include the requested fields explicitly. Alternatively, you could express which fields you require in the subordinate agent's description. Both approaches can work––you'll have to see what works best for your case.

#### 2.2.6 Tracing

You can make `BedrockConverseAgent` log traces of the LLM and tool calls it performs, by providing a tracer class.

In the following example, the `InMemoryTracer` is used, which is meant for use during development:

```python
from generative_ai_toolkit.agent import BedrockConverseAgent
from generative