Repository: microsoft/autogen
Pull request: add anthropic native support
<!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. -->

Claude 3.7 just came out. Its a pretty capable model and it would be great to support it in Autogen. 
This will could augment the already excellent support we have for Anthropic via the SKAdapters  in the following ways

- Based on the ChatCompletion API similar to the ollama and openai client 
- Configurable/serializable (can be dumped) .. this means it can be used easily in AGS.

## What is Supported 

(video below shows the client being used in autogen studio)
https://github.com/user-attachments/assets/8fb7c17c-9f9c-4525-aa9c-f256aad0f40b



- streaming 
- tool callign / function calling 
- drop in integration with assistant agent. 
- multimodal support

```python

from dotenv import load_dotenv
import os 

load_dotenv()

from autogen_agentchat.agents import AssistantAgent
from autogen_agentchat.ui import Console
from autogen_ext.models.openai import OpenAIChatCompletionClient
from autogen_ext.models.anthropic import AnthropicChatCompletionClient 
model_client =   AnthropicChatCompletionClient(
        model="claude-3-7-sonnet-20250219" 
    )

async def get_weather(city: str) -> str:
    """Get the weather for a given city."""
    return f"The weather in {city} is 73 degrees and Sunny."

 
agent = AssistantAgent(
    name="weather_agent",
    model_client=model_client,
    tools=[get_weather],
    system_message="You are a helpful assistant.", 
    # model_client_stream=True,   
)

# Run the agent and stream the messages to the console.
async def main() -> None:
    await Console(agent.run_stream(task="What is the weather in New York?"))
await main()
```

result 

```
messages = [
    UserMessage(content="Write a very short story about a dragon.", source="user"),
]

# Create a stream.
stream = model_client.create_stream(messages=messages)

# Iterate over the stream and print the responses.
print("Streamed responses:")
async for response in stream:  # type: ignore
    if isinstance(response, str):
        # A partial response is a string.
        print(response, flush=True, end="")
    else:
        # The last response is a CreateResult object with the complete message.
        print("\n\n------------\n")
        print("The complete response:", flush=True)
        print(response.content, flush=True)
        print("\n\n------------\n")
        print("The token usage was:", flush=True)
        print(response.usage, flush=True)
```

 

<!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. -->

## Why are these changes needed?

<!-- Please give a short summary of the change and the problem this solves. -->

## Related issue number

<!-- For example: "Closes #1234" --> 

Closes #5205 
Closes #5708

## Checks

- [ ] I've included any doc changes needed for <https://microsoft.github.io/autogen/>. See <https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to build and test documentation locally.
- [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR.
- [ ] I've made sure all auto checks have passed. 



cc @rohanthacker 


Problem statement:
Support for anthropic models in v0.4
### What feature would you like to be added?

Currently there is no way to use anthropic models with autogen 0.4 I need a  way to use Claude models just like I was able to do so in 0.2.

Also there is no support for performing RAG as of now

### Why is this needed?

Claude models are better at planning and deliberating in multiagent environments.

RAG support makes the the agent capable of handling a variety of tasks


Support Anthropic Client in AutoGen v0.4 and Studio

Support Anthropic Client in AutoGen v0.4 
Expected behaviour below

```python


from dotenv import load_dotenv
import os 

load_dotenv()

from autogen_agentchat.agents import AssistantAgent
from autogen_agentchat.ui import Console
from autogen_ext.models.openai import OpenAIChatCompletionClient
from autogen_ext.models.anthropic import AnthropicChatCompletionClient 
model_client =   AnthropicChatCompletionClient(
        model="claude-3-7-sonnet-20250219" 
    )

async def get_weather(city: str) -> str:
    """Get the weather for a given city."""
    return f"The weather in {city} is 73 degrees and Sunny."

 
agent = AssistantAgent(
    name="weather_agent",
    model_client=model_client,
    tools=[get_weather],
    system_message="You are a helpful assistant.", 
    # model_client_stream=True,   
)

# Run the agent and stream the messages to the console.
async def main() -> None:
    await Console(agent.run_stream(task="What is the weather in New York?"))
await main()

```

Hints from issue discussion:
Thanks @raagul-qb. I am editing this Issue to remove the RAG part to help making this issue more specific. 

@rohanthacker was that you or someone else posted on Reddit about working on an Anthropic client? 
For RAG, please see rag tools like graphrag: https://microsoft.github.io/autogen/stable/reference/python/autogen_ext.tools.graphrag.html#autogen_ext.tools.graphrag.LocalSearchTool, checkout samples too. https://github.com/microsoft/autogen/tree/main/python/samples/agentchat_graphrag

For RAG agent, use this #4742 to track
Yes that was me. I've built out initial versions of the gemini and anthropic clients.
Thanks! I have been using the Azure AI client now. Thanks for your contribution. 

Would love to review the PR for Gemini and Anthropic clients. 
+1 I want to use it using the OpenAI compatiblity but unable to do so. 
Is there any alternative 

```
model_client = OpenAIChatCompletionClient(
    model="claude-3-5-sonnet-20241022",
    api_key="sxxx",
    base_url="https://api.anthropic.com/v1/messages",
    model_capabilities={
        "vision": False,
        "function_calling": True,
        "json_output": False,
        'tool_calls' : True
    }
)
```
> Thanks! I have been using the Azure AI client now. Thanks for your contribution.
> 
> Would love to review the PR for Gemini and Anthropic clients.

I'm back on this project again this week. I was a bit tied up with work and the end of the year activities. I'll push up a draft soon

> +1 I want to use it using the OpenAI compatiblity but unable to do so. Is there any alternative
> 
> ```
> model_client = OpenAIChatCompletionClient(
>     model="claude-3-5-sonnet-20241022",
>     api_key="sxxx",
>     base_url="https://api.anthropic.com/v1/messages",
>     model_capabilities={
>         "vision": False,
>         "function_calling": True,
>         "json_output": False,
>         'tool_calls' : True
>     }
> )
> ```
@Navanit-git 
I could be wrong, but I think this will work with the `OpenAIChatCompletionClient` as the API's and client libraries for these services are not cross-compatible out of the box. 

For an immediate alternative you could try using the new `SKChatCompletionAdapter` along with the `AnthropicChatCompletion` from the sematic kernel package.

Refer these docs for more info:
[SKChatCompletionAdapter from Autogen][1]
[AnthropicChatCompletion from Semantic Kernel][2]

[1]: https://microsoft.github.io/autogen/stable/reference/python/autogen_ext.models.semantic_kernel.html
[2]: https://learn.microsoft.com/en-us/semantic-kernel/concepts/ai-services/chat-completion/?tabs=csharp-AzureOpenAI%2Cpython-Anthropic%2Cjava-AzureOpenAI&pivots=programming-language-python
Bumping this—just adding my voice to say this is a valuable feature, and it's unfortunate that 0.4 was released with only OpenAI support (unless I missed something). Personally, I need Anthropic support badly enough that I'll have to go with a different framework for now. Autogen 0.4 looks great and fits what I need, but without Anthropic support—and without time to implement it myself—I’ll likely stick with LangGraph and end up reinventing what AgentChat already provides.
@ai-is-here This feature is work in progress. 
For immediate support without having to implement and with minimal code you could try the Sematic Kernel adapter mentioned above.   

@ekzhu Can this task be assigned to me I'm working on this. 
@rohanthacker  thank you! 

@ai-is-here , see the following example code:

```python
import asyncio
import os
from semantic_kernel import Kernel
from semantic_kernel.memory.null_memory import NullMemory
from semantic_kernel.connectors.ai.anthropic import AnthropicChatCompletion
from autogen_ext.models.semantic_kernel import SKChatCompletionAdapter
from autogen_agentchat.agents import AssistantAgent

async def main() -> None:
    sk_anthropic_client = AnthropicChatCompletion(
        ai_model_id="claude-3-5-sonnet-20241022",
        api_key=os.environ["ANTHROPIC_API_KEY"],
        service_id="my-service-id", # Optional; for targeting specific services within Semantic Kernel
    )

    model_client = SKChatCompletionAdapter(sk_anthropic_client, kernel=Kernel(memory=NullMemory()))

    assistant = AssistantAgent("assistant", model_client=model_client)

    result = await assistant.run(task="What is the capital of France?")
    print(result)

asyncio.run(main())
```

It should just work.

Commit message: update tests

Patch to review:
diff --git a/python/packages/autogen-ext/src/autogen_ext/models/anthropic/__init__.py b/python/packages/autogen-ext/src/autogen_ext/models/anthropic/__init__.py
new file mode 100644
index 000000000000..75365d781eb3
--- /dev/null
+++ b/python/packages/autogen-ext/src/autogen_ext/models/anthropic/__init__.py
@@ -0,0 +1,14 @@
+from ._anthropic_client import AnthropicChatCompletionClient, BaseAnthropicChatCompletionClient
+from .config import (
+    AnthropicClientConfiguration,
+    AnthropicClientConfigurationConfigModel,
+    CreateArgumentsConfigModel,
+)
+
+__all__ = [
+    "AnthropicChatCompletionClient",
+    "BaseAnthropicChatCompletionClient",
+    "AnthropicClientConfiguration",
+    "AnthropicClientConfigurationConfigModel",
+    "CreateArgumentsConfigModel",
+]
diff --git a/python/packages/autogen-ext/src/autogen_ext/models/anthropic/_anthropic_client.py b/python/packages/autogen-ext/src/autogen_ext/models/anthropic/_anthropic_client.py
new file mode 100644
index 000000000000..20ac46ab0241
--- /dev/null
+++ b/python/packages/autogen-ext/src/autogen_ext/models/anthropic/_anthropic_client.py
@@ -0,0 +1,987 @@
+import asyncio
+import base64
+import inspect
+import json
+import logging
+import re
+import warnings
+
+# from asyncio import Task
+from typing import (
+    Any,
+    AsyncGenerator,
+    Coroutine,
+    Dict,
+    List,
+    Literal,
+    Mapping,
+    Optional,
+    Sequence,
+    Set,
+    Union,
+    cast,
+)
+
+import tiktoken
+from anthropic import AsyncAnthropic, AsyncStream
+from anthropic.types import (
+    ContentBlock,
+    ImageBlockParam,
+    Message,
+    MessageParam,
+    RawMessageStreamEvent,  # type: ignore
+    TextBlock,
+    TextBlockParam,
+    ToolParam,
+    ToolResultBlockParam,
+    ToolUseBlock,
+)
+from anthropic.types.image_block_param import Source
+from autogen_core import (
+    EVENT_LOGGER_NAME,
+    TRACE_LOGGER_NAME,
+    CancellationToken,
+    Component,
+    FunctionCall,
+    Image,
+    MessageHandlerContext,
+)
+from autogen_core.logging import LLMCallEvent
+from autogen_core.models import (
+    AssistantMessage,
+    ChatCompletionClient,
+    CreateResult,
+    FinishReasons,
+    FunctionExecutionResultMessage,
+    LLMMessage,
+    ModelCapabilities,  # type: ignore
+    ModelFamily,
+    ModelInfo,
+    RequestUsage,
+    SystemMessage,
+    UserMessage,
+)
+from autogen_core.tools import Tool, ToolSchema
+from typing_extensions import Self, Unpack
+
+from . import _model_info
+from .config import AnthropicClientConfiguration, AnthropicClientConfigurationConfigModel
+
+logger = logging.getLogger(EVENT_LOGGER_NAME)
+trace_logger = logging.getLogger(TRACE_LOGGER_NAME)
+
+# Common parameters for message creation
+anthropic_message_params = {
+    "system",
+    "messages",
+    "max_tokens",
+    "temperature",
+    "top_p",
+    "top_k",
+    "stop_sequences",
+    "tools",
+    "tool_choice",
+    "stream",
+    "metadata",
+}
+disallowed_create_args = {"stream", "messages"}
+required_create_args: Set[str] = {"model"}
+
+anthropic_init_kwargs = set(inspect.getfullargspec(AsyncAnthropic.__init__).kwonlyargs)
+
+
+def _anthropic_client_from_config(config: Mapping[str, Any]) -> AsyncAnthropic:
+    # Filter config to only include valid parameters
+    client_config = {k: v for k, v in config.items() if k in anthropic_init_kwargs}
+    return AsyncAnthropic(**client_config)
+
+
+def _create_args_from_config(config: Mapping[str, Any]) -> Dict[str, Any]:
+    create_args = {k: v for k, v in config.items() if k in anthropic_message_params or k == "model"}
+    create_args_keys = set(create_args.keys())
+
+    if not required_create_args.issubset(create_args_keys):
+        raise ValueError(f"Required create args are missing: {required_create_args - create_args_keys}")
+
+    if disallowed_create_args.intersection(create_args_keys):
+        raise ValueError(f"Disallowed create args are present: {disallowed_create_args.intersection(create_args_keys)}")
+
+    return create_args
+
+
+def type_to_role(message: LLMMessage) -> str:
+    if isinstance(message, SystemMessage):
+        return "system"
+    elif isinstance(message, UserMessage):
+        return "user"
+    elif isinstance(message, AssistantMessage):
+        return "assistant"
+    else:
+        return "tool"
+
+
+def get_mime_type_from_image(image: Image) -> Literal["image/jpeg", "image/png", "image/gif", "image/webp"]:
+    """Get a valid Anthropic media type from an Image object."""
+    # Get base64 data first
+    base64_data = image.to_base64()
+
+    # Decode the base64 string
+    image_data = base64.b64decode(base64_data)
+
+    # Check the first few bytes for known signatures
+    if image_data.startswith(b"\xff\xd8\xff"):
+        return "image/jpeg"
+    elif image_data.startswith(b"\x89PNG\r\n\x1a\n"):
+        return "image/png"
+    elif image_data.startswith(b"GIF87a") or image_data.startswith(b"GIF89a"):
+        return "image/gif"
+    elif image_data.startswith(b"RIFF") and image_data[8:12] == b"WEBP":
+        return "image/webp"
+    else:
+        # Default to JPEG as a fallback
+        return "image/jpeg"
+
+
+def user_message_to_anthropic(message: UserMessage) -> MessageParam:
+    assert_valid_name(message.source)
+
+    if isinstance(message.content, str):
+        return {
+            "role": "user",
+            "content": message.content,
+        }
+    else:
+        blocks: List[Union[TextBlockParam, ImageBlockParam]] = []
+
+        for part in message.content:
+            if isinstance(part, str):
+                blocks.append(TextBlockParam(type="text", text=part))
+            elif isinstance(part, Image):
+                blocks.append(
+                    ImageBlockParam(
+                        type="image",
+                        source=Source(
+                            type="base64",
+                            media_type=get_mime_type_from_image(part),
+                            data=part.to_base64(),
+                        ),
+                    )
+                )
+            else:
+                raise ValueError(f"Unknown content type: {part}")
+
+        return {
+            "role": "user",
+            "content": blocks,
+        }
+
+
+def system_message_to_anthropic(message: SystemMessage) -> str:
+    return message.content
+
+
+def assistant_message_to_anthropic(message: AssistantMessage) -> MessageParam:
+    assert_valid_name(message.source)
+
+    if isinstance(message.content, list):
+        # Tool calls
+        tool_use_blocks: List[ToolUseBlock] = []
+
+        for func_call in message.content:
+            # Parse the arguments and convert to dict if it's a JSON string
+            args = func_call.arguments
+            if isinstance(args, str):
+                try:
+                    args_dict = json.loads(args)
+                except json.JSONDecodeError:
+                    args_dict = {"text": args}
+            else:
+                args_dict = args
+
+            tool_use_blocks.append(
+                ToolUseBlock(
+                    type="tool_use",
+                    id=func_call.id,
+                    name=func_call.name,
+                    input=args_dict,
+                )
+            )
+
+        # Include thought if available
+        content_blocks: List[ContentBlock] = []
+        if hasattr(message, "thought") and message.thought is not None:
+            content_blocks.append(TextBlock(type="text", text=message.thought))
+
+        content_blocks.extend(tool_use_blocks)
+
+        return {
+            "role": "assistant",
+            "content": content_blocks,
+        }
+    else:
+        # Simple text content
+        return {
+            "role": "assistant",
+            "content": message.content,
+        }
+
+
+def tool_message_to_anthropic(message: FunctionExecutionResultMessage) -> List[MessageParam]:
+    # Create a single user message containing all tool results
+    content_blocks: List[ToolResultBlockParam] = []
+
+    for result in message.content:
+        content_blocks.append(
+            ToolResultBlockParam(
+                type="tool_result",
+                tool_use_id=result.call_id,
+                content=result.content,
+            )
+        )
+
+    return [
+        {
+            "role": "user",  # Changed from "tool" to "user"
+            "content": content_blocks,
+        }
+    ]
+
+
+def to_anthropic_type(message: LLMMessage) -> Union[str, List[MessageParam], MessageParam]:
+    if isinstance(message, SystemMessage):
+        return system_message_to_anthropic(message)
+    elif isinstance(message, UserMessage):
+        return user_message_to_anthropic(message)
+    elif isinstance(message, AssistantMessage):
+        return assistant_message_to_anthropic(message)
+    else:
+        return tool_message_to_anthropic(message)
+
+
+def convert_tools(tools: Sequence[Tool | ToolSchema]) -> List[ToolParam]:
+    result: List[ToolParam] = []
+
+    for tool in tools:
+        if isinstance(tool, Tool):
+            tool_schema = tool.schema
+        else:
+            assert isinstance(tool, dict)
+            tool_schema = tool
+
+        # Convert parameters to match Anthropic's schema format
+        tool_params: Dict[str, Any] = {}
+        if "parameters" in tool_schema:
+            params = tool_schema["parameters"]
+
+            # Transfer properties
+            if "properties" in params:
+                tool_params["properties"] = params["properties"]
+
+            # Transfer required fields
+            if "required" in params:
+                tool_params["required"] = params["required"]
+
+            # Handle schema type
+            if "type" in params:
+                tool_params["type"] = params["type"]
+            else:
+                tool_params["type"] = "object"
+
+        result.append(
+            ToolParam(
+                name=tool_schema["name"],
+                input_schema=tool_params,
+                description=tool_schema.get("description", ""),
+            )
+        )
+
+        # Check if the tool has a valid name
+        assert_valid_name(tool_schema["name"])
+
+    return result
+
+
+def normalize_name(name: str) -> str:
+    """
+
+    def __init__(self, **kwargs: Unpack[AnthropicClientConfiguration]):
+        if "model" not in kwargs:
+            raise ValueError("model is required for AnthropicChatCompletionClient")
+
+        model_capabilities: Optional[ModelCapabilities] = None  # type: ignore
+        self._raw_config: Dict[str, Any] = dict(kwargs).copy()
+        copied_args = dict(kwargs).copy()
+
+        if "model_capabilities" in kwargs:
+            model_capabilities = kwargs["model_capabilities"]
+            del copied_args["model_capabilities"]
+
+        model_info: Optional[ModelInfo] = None
+        if "model_info" in kwargs:
+            model_info = kwargs["model_info"]
+            del copied_args["model_info"]
+
+        client = _anthropic_client_from_config(copied_args)
+        create_args = _create_args_from_config(copied_args)
+
+        super().__init__(
+            client=client,
+            create_args=create_args,
+            model_capabilities=model_capabilities,
+            model_info=model_info,
+        )
+
+    def __getstate__(self) -> Dict[str, Any]:
+        state = self.__dict__.copy()
+        state["_client"] = None
+        return state
+
+    def __setstate__(self, state: Dict[str, Any]) -> None:
+        self.__dict__.update(state)
+        self._client = _anthropic_client_from_config(state["_raw_config"])
+
+    def _to_config(self) -> AnthropicClientConfigurationConfigModel:
+        copied_config = self._raw_config.copy()
+        return AnthropicClientConfigurationConfigModel(**copied_config)
+
+    @classmethod
+    def _from_config(cls, config: AnthropicClientConfigurationConfigModel) -> Self:
+        copied_config = config.model_copy().model_dump(exclude_none=True)
+        return cls(**copied_config)
+    Normalize names by replacing invalid characters with underscore.
+    """
+    return re.sub(r"[^a-zA-Z0-9_-]", "_", name)[:64]
+
+
+def assert_valid_name(name: str) -> str:
+    """
+    Ensure that configured names are valid, raises ValueError if not.
+    """
+    if not re.match(r"^[a-zA-Z0-9_-]+$", name):
+        raise ValueError(f"Invalid name: {name}. Only letters, numbers, '_' and '-' are allowed.")
+    if len(name) > 64:
+        raise ValueError(f"Invalid name: {name}. Name must be less than 64 characters.")
+    return name
+
+
+def normalize_stop_reason(stop_reason: str | None) -> FinishReasons:
+    if stop_reason is None:
+        return "unknown"
+
+    # Convert to lowercase for comparison
+    stop_reason = stop_reason.lower()
+
+    # Map Anthropic stop reasons to standard reasons
+    KNOWN_STOP_MAPPINGS: Dict[str, FinishReasons] = {
+        "end_turn": "stop",
+        "max_tokens": "length",
+        "stop_sequence": "stop",
+        "tool_use": "function_calls",
+    }
+
+    return KNOWN_STOP_MAPPINGS.get(stop_reason, "unknown")
+
+
+def _add_usage(usage1: RequestUsage, usage2: RequestUsage) -> RequestUsage:
+    return RequestUsage(
+        prompt_tokens=usage1.prompt_tokens + usage2.prompt_tokens,
+        completion_tokens=usage1.completion_tokens + usage2.completion_tokens,
+    )
+
+
+class BaseAnthropicChatCompletionClient(ChatCompletionClient):
+    def __init__(
+        self,
+        client: AsyncAnthropic,
+        *,
+        create_args: Dict[str, Any],
+        model_capabilities: Optional[ModelCapabilities] = None,  # type: ignore
+        model_info: Optional[ModelInfo] = None,
+    ):
+        self._client = client
+
+        # Handle model info and capabilities
+        if model_capabilities is None and model_info is None:
+            try:
+                self._model_info = _model_info.get_info(create_args["model"])
+            except KeyError as err:
+                raise ValueError("model_info is required when model name is not recognized") from err
+        elif model_capabilities is not None and model_info is not None:
+            raise ValueError("model_capabilities and model_info are mutually exclusive")
+        elif model_capabilities is not None and model_info is None:
+            warnings.warn("model_capabilities is deprecated, use model_info instead", DeprecationWarning, stacklevel=2)
+            info = cast(ModelInfo, model_capabilities)
+            info["family"] = ModelFamily.CLAUDE_3_5_SONNET
+            self._model_info = info
+        elif model_capabilities is None and model_info is not None:
+            self._model_info = model_info
+
+        # Validate model_info
+        if not all(key in self._model_info for key in ["vision", "function_calling", "json_output", "family"]):
+            raise ValueError("model_info must contain 'vision', 'function_calling', 'json_output', and 'family' keys")
+
+        self._create_args = create_args
+        self._total_usage = RequestUsage(prompt_tokens=0, completion_tokens=0)
+        self._actual_usage = RequestUsage(prompt_tokens=0, completion_tokens=0)
+
+    async def create(
+        self,
+        messages: Sequence[LLMMessage],
+        *,
+        tools: Sequence[Tool | ToolSchema] = [],
+        json_output: Optional[bool] = None,
+        extra_create_args: Mapping[str, Any] = {},
+        cancellation_token: Optional[CancellationToken] = None,
+    ) -> CreateResult:
+        # Copy create args and update with extra args
+        create_args = self._create_args.copy()
+        create_args.update(extra_create_args)
+
+        # Check for vision capability if images are present
+        if self.model_info["vision"] is False:
+            for message in messages:
+                if isinstance(message, UserMessage):
+                    if isinstance(message.content, list) and any(isinstance(x, Image) for x in message.content):
+                        raise ValueError("Model does not support vision and image was provided")
+
+        # Handle JSON output format
+        if json_output is not None:
+            if self.model_info["json_output"] is False and json_output is True:
+                raise ValueError("Model does not support JSON output")
+
+            if json_output is True:
+                create_args["response_format"] = {"type": "json_object"}
+
+        # Process system message separately
+        system_message = None
+        anthropic_messages: List[MessageParam] = []
+
+        for message in messages:
+            if isinstance(message, SystemMessage):
+                if system_message is not None:
+                    raise ValueError("Multiple system messages are not supported")
+                system_message = to_anthropic_type(message)
+            else:
+                anthropic_message = to_anthropic_type(message)
+                if isinstance(anthropic_message, list):
+                    anthropic_messages.extend(anthropic_message)
+                elif isinstance(anthropic_message, str):
+                    msg = MessageParam(
+                        role="user" if isinstance(message, UserMessage) else "assistant", content=anthropic_message
+                    )
+                    anthropic_messages.append(msg)
+                else:
+                    anthropic_messages.append(anthropic_message)
+
+        # Check for function calling support
+        if self.model_info["function_calling"] is False and len(tools) > 0:
+            raise ValueError("Model does not support function calling")
+
+        # Set up the request
+        request_args: Dict[str, Any] = {
+            "model": create_args["model"],
+            "messages": anthropic_messages,
+            "max_tokens": create_args.get("max_tokens", 4096),
+            "temperature": create_args.get("temperature", 1.0),
+        }
+
+        # Add system message if present
+        if system_message is not None:
+            request_args["system"] = system_message
+
+        has_tool_results = any(isinstance(msg, FunctionExecutionResultMessage) for msg in messages)
+
+        # Store and add tools if present
+        if len(tools) > 0:
+            converted_tools = convert_tools(tools)
+            self._last_used_tools = converted_tools
+            request_args["tools"] = converted_tools
+        elif has_tool_results:
+            # anthropic requires tools to be present even if there is any tool use
+            request_args["tools"] = self._last_used_tools
+
+        # Optional parameters
+        for param in ["top_p", "top_k", "stop_sequences", "metadata"]:
+            if param in create_args:
+                request_args[param] = create_args[param]
+
+        # Execute the request
+        future: asyncio.Task[Message] = asyncio.ensure_future(self._client.messages.create(**request_args))  # type: ignore
+
+        if cancellation_token is not None:
+            cancellation_token.link_future(future)  # type: ignore
+
+        result: Message = cast(Message, await future)  # type: ignore
+
+        # Extract usage statistics
+        usage = RequestUsage(
+            prompt_tokens=result.usage.input_tokens,
+            completion_tokens=result.usage.output_tokens,
+        )
+
+        # Log the event if in a handler context
+        try:
+            agent_id = MessageHandlerContext.agent_id()
+        except RuntimeError:
+            agent_id = None
+
+        logger.info(
+            LLMCallEvent(
+                messages=cast(Dict[str, Any], anthropic_messages),
+                response=result.model_dump(),
+                prompt_tokens=usage.prompt_tokens,
+                completion_tokens=usage.completion_tokens,
+                agent_id=agent_id,
+            )
+        )
+
+        # Process the response
+        content: Union[str, List[FunctionCall]]
+        thought = None
+
+        # Check if the response includes tool uses
+        tool_uses = [block for block in result.content if getattr(block, "type", None) == "tool_use"]
+
+        if tool_uses:
+            # Handle tool use response
+            content = []
+
+            # Check for text content that should be treated as thought
+            text_blocks: List[TextBlock] = [block for block in result.content if isinstance(block, TextBlock)]
+            if text_blocks:
+                thought = "".join([block.text for block in text_blocks])
+
+            # Process tool use blocks
+            for tool_use in tool_uses:
+                if isinstance(tool_use, ToolUseBlock):
+                    tool_input = tool_use.input
+                    if isinstance(tool_input, dict):
+                        tool_input = json.dumps(tool_input)
+                    else:
+                        tool_input = str(tool_input) if tool_input is not None else ""
+
+                    content.append(
+                        FunctionCall(
+                            id=tool_use.id,
+                            name=normalize_name(tool_use.name),
+                            arguments=tool_input,
+                        )
+                    )
+        else:
+            # Handle text response
+            content = "".join([block.text if isinstance(block, TextBlock) else "" for block in result.content])
+
+        # Create the final result
+        response = CreateResult(
+            finish_reason=normalize_stop_reason(result.stop_reason),
+            content=content,
+            usage=usage,
+            cached=False,
+            thought=thought,
+        )
+
+        # Update usage statistics
+        self._total_usage = _add_usage(self._total_usage, usage)
+        self._actual_usage = _add_usage(self._actual_usage, usage)
+
+        return response
+
+    async def create_stream(
+        self,
+        messages: Sequence[LLMMessage],
+        *,
+        tools: Sequence[Tool | ToolSchema] = [],
+        json_output: Optional[bool] = None,
+        extra_create_args: Mapping[str, Any] = {},
+        cancellation_token: Optional[CancellationToken] = None,
+        max_consecutive_empty_chunk_tolerance: int = 0,
+    ) -> AsyncGenerator[Union[str, CreateResult], None]:
+        """
+        Creates an AsyncGenerator that yields a stream of completions based on the provided messages and tools.
+        """
+        # Copy create args and update with extra args
+        create_args = self._create_args.copy()
+        create_args.update(extra_create_args)
+
+        # Check for vision capability if images are present
+        if self.model_info["vision"] is False:
+            for message in messages:
+                if isinstance(message, UserMessage):
+                    if isinstance(message.content, list) and any(isinstance(x, Image) for x in message.content):
+                        raise ValueError("Model does not support vision and image was provided")
+
+        # Handle JSON output format
+        if json_output is not None:
+            if self.model_info["json_output"] is False and json_output is True:
+                raise ValueError("Model does not support JSON output")
+
+            if json_output is True:
+                create_args["response_format"] = {"type": "json_object"}
+
+        # Process system message separately
+        system_message = None
+        anthropic_messages: List[MessageParam] = []
+
+        for message in messages:
+            if isinstance(message, SystemMessage):
+                if system_message is not None:
+                    raise ValueError("Multiple system messages are not supported")
+                system_message = to_anthropic_type(message)
+            else:
+                anthropic_message = to_anthropic_type(message)
+                if isinstance(anthropic_message, list):
+                    anthropic_messages.extend(anthropic_message)
+                elif isinstance(anthropic_message, str):
+                    msg = MessageParam(
+                        role="user" if isinstance(message, UserMessage) else "assistant", content=anthropic_message
+                    )
+                    anthropic_messages.append(msg)
+                else:
+                    anthropic_messages.append(anthropic_message)
+
+        # Check for function calling support
+        if self.model_info["function_calling"] is False and len(tools) > 0:
+            raise ValueError("Model does not support function calling")
+
+        # Set up the request
+        request_args: Dict[str, Any] = {
+            "model": create_args["model"],
+            "messages": anthropic_messages,
+            "max_tokens": create_args.get("max_tokens", 4096),
+            "temperature": create_args.get("temperature", 1.0),
+            "stream": True,
+        }
+
+        # Add system message if present
+        if system_message is not None:
+            request_args["system"] = system_message
+
+        # Add tools if present
+        if len(tools) > 0:
+            converted_tools = convert_tools(tools)
+            request_args["tools"] = converted_tools
+
+        # Optional parameters
+        for param in ["top_p", "top_k", "stop_sequences", "metadata"]:
+            if param in create_args:
+                request_args[param] = create_args[param]
+
+        # Stream the response
+        stream_future: asyncio.Task[AsyncStream[RawMessageStreamEvent]] = asyncio.ensure_future(
+            cast(Coroutine[Any, Any, AsyncStream[RawMessageStreamEvent]], self._client.messages.create(**request_args))
+        )
+
+        if cancellation_token is not None:
+            cancellation_token.link_future(stream_future)  # type: ignore
+
+        stream: AsyncStream[RawMessageStreamEvent] = cast(AsyncStream[RawMessageStreamEvent], await stream_future)  # type: ignore
+
+        # Prepare variables for accumulating the response
+        text_content: List[str] = []
+        tool_calls: Dict[str, Dict[str, str]] = {}
+        input_tokens: int = 0
+        output_tokens: int = 0
+        stop_reason: Optional[str] = None
+
+        # Process the stream
+        async for chunk in stream:
+            # Handle different event types
+            if chunk.type == "content_block_delta":
+                # Text content
+                if chunk.delta.type == "text_delta":
+                    delta_text = chunk.delta.text
+                    text_content.append(delta_text)
+                    if delta_text:
+                        yield delta_text
+
+            elif chunk.type == "message_delta":
+                if hasattr(chunk, "usage") and hasattr(chunk.usage, "output_tokens"):
+                    output_tokens = chunk.usage.output_tokens
+                # End of message with stop reason
+                if hasattr(chunk.delta, "stop_reason") and chunk.delta.stop_reason:
+                    stop_reason = chunk.delta.stop_reason
+
+            elif chunk.type == "message_start":
+                if hasattr(chunk, "message") and hasattr(chunk.message, "usage"):
+                    input_tokens = chunk.message.usage.input_tokens
+                    output_tokens = chunk.message.usage.output_tokens
+
+            elif chunk.type == "message_stop":
+                pass
+
+        # Prepare the final result
+        usage = RequestUsage(
+            prompt_tokens=input_tokens,
+            completion_tokens=output_tokens,
+        )
+
+        # Determine content based on what was received
+        content: Union[str, List[FunctionCall]]
+        thought = None
+
+        if tool_calls:
+            # We received tool calls
+            if text_content:
+                # Text before tool calls is treated as thought
+                thought = "".join(text_content)
+
+            # Convert tool calls to FunctionCall objects
+            content = [
+                FunctionCall(
+                    id=call_data["id"],
+                    name=normalize_name(call_data["name"]),
+                    arguments=call_data["input"],
+                )
+                for call_data in tool_calls.values()
+            ]
+        else:
+            # Just text content
+            content = "".join(text_content)
+
+        # Create final result
+        result = CreateResult(
+            finish_reason=normalize_stop_reason(stop_reason),
+            content=content,
+            usage=usage,
+            cached=False,
+            thought=thought,
+        )
+
+        # Update usage statistics
+        self._total_usage = _add_usage(self._total_usage, usage)
+        self._actual_usage = _add_usage(self._actual_usage, usage)
+
+        yield result
+
+    def count_tokens(self, messages: Sequence[LLMMessage], *, tools: Sequence[Tool | ToolSchema] = []) -> int:
+        """
+        Estimate the number of tokens used by messages and tools.
+
+        Note: This is an estimation based on common tokenization patterns and may not perfectly
+        match Anthropic's exact token counting for Claude models.
+        """
+        # Use cl100k_base encoding as an approximation for Claude's tokenizer
+        try:
+            encoding = tiktoken.get_encoding("cl100k_base")
+        except Exception:
+            encoding = tiktoken.get_encoding("gpt2")  # Fallback
+
+        num_tokens = 0
+
+        # System message tokens (if any)
+        system_content = None
+        for message in messages:
+            if isinstance(message, SystemMessage):
+                system_content = message.content
+                break
+
+        if system_content:
+            num_tokens += len(encoding.encode(system_content)) + 15  # Approximate system message overhead
+
+        # Message tokens
+        for message in messages:
+            if isinstance(message, SystemMessage):
+                continue  # Already counted
+
+            # Base token cost per message
+            num_tokens += 10  # Approximate message role & formatting overhead
+
+            # Content tokens
+            if isinstance(message, UserMessage) or isinstance(message, AssistantMessage):
+                if isinstance(message.content, str):
+                    num_tokens += len(encoding.encode(message.content))
+                elif isinstance(message.content, list):
+                    # Handle different content types
+                    for part in message.content:
+                        if isinstance(part, str):
+                            num_tokens += len(encoding.encode(part))
+                        elif isinstance(part, Image):
+                            # Estimate vision tokens (simplified)
+                            num_tokens += 512  # Rough estimation for image tokens
+                        elif isinstance(part, FunctionCall):
+                            num_tokens += len(encoding.encode(part.name))
+                            num_tokens += len(encoding.encode(part.arguments))
+                            num_tokens += 10  # Function call overhead
+            elif isinstance(message, FunctionExecutionResultMessage):
+                for result in message.content:
+                    num_tokens += len(encoding.encode(result.content))
+                    num_tokens += 10  # Function result overhead
+
+        # Tool tokens
+        for tool in tools:
+            if isinstance(tool, Tool):
+                tool_schema = tool.schema
+            else:
+                tool_schema = tool
+
+            # Name and description
+            num_tokens += len(encoding.encode(tool_schema["name"]))
+            if "description" in tool_schema:
+                num_tokens += len(encoding.encode(tool_schema["description"]))
+
+            # Parameters
+            if "parameters" in tool_schema:
+                params = tool_schema["parameters"]
+
+                if "properties" in params:
+                    for prop_name, prop_schema in params["properties"].items():
+                        num_tokens += len(encoding.encode(prop_name))
+
+                        if "type" in prop_schema:
+                            num_tokens += len(encoding.encode(prop_schema["type"]))
+
+                        if "description" in prop_schema:
+                            num_tokens += len(encoding.encode(prop_schema["description"]))
+
+                        # Special handling for enums
+                        if "enum" in prop_schema:
+                            for value in prop_schema["enum"]:
+                                if isinstance(value, str):
+                                    num_tokens += len(encoding.encode(value))
+                                else:
+                                    num_tokens += 2  # Non-string enum values
+
+            # Tool overhead
+            num_tokens += 20
+
+        return num_tokens
+
+    def remaining_tokens(self, messages: Sequence[LLMMessage], *, tools: Sequence[Tool | ToolSchema] = []) -> int:
+        """Calculate the remaining tokens based on the model's token limit."""
+        token_limit = _model_info.get_token_limit(self._create_args["model"])
+        return token_limit - self.count_tokens(messages, tools=tools)
+
+    def actual_usage(self) -> RequestUsage:
+        return self._actual_usage
+
+    def total_usage(self) -> RequestUsage:
+        return self._total_usage
+
+    @property
+    def capabilities(self) -> ModelCapabilities:  # type: ignore
+        warnings.warn("capabilities is deprecated, use model_info instead", DeprecationWarning, stacklevel=2)
+        return self._model_info
+
+    @property
+    def model_info(self) -> ModelInfo:
+        return self._model_info
+
+
+class AnthropicChatCompletionClient(
+    BaseAnthropicChatCompletionClient, Component[AnthropicClientConfigurationConfigModel]
+):
+    """
+    Chat completion client for Anthropic's Claude models.
+
+    Args:
+        model (str): The Claude model to use (e.g., "claude-3-sonnet-20240229", "claude-3-opus-20240229")
+        api_key (str, optional): Anthropic API key. Required if not in environment variables.
+        base_url (str, optional): Override the default API endpoint.
+        max_tokens (int, optional): Maximum tokens in the response. Default is 4096.
+        temperature (float, optional): Controls randomness. Lower is more deterministic. Default is 1.0.
+        top_p (float, optional): Controls diversity via nucleus sampling. Default is 1.0.
+        top_k (int, optional): Controls diversity via top-k sampling. Default is -1 (disabled).
+        model_info (ModelInfo, optional): The capabilities of the model. Required if using a custom model.
+
+    To use this client, you must install the Anthropic extension:
+
+    .. code-block:: bash
+
+        pip install "autogen-ext[anthropic]"
+
+    Example:
+
+    .. code-block:: python
+
+        from autogen_ext.models.anthropic import AnthropicChatCompletionClient
+        from autogen_core.models import UserMessage
+
+        anthropic_client = AnthropicChatCompletionClient(
+            model="claude-3-sonnet-20240229",
+            api_key="your-api-key",  # Optional if ANTHROPIC_API_KEY is set in environment
+        )
+
+        result = await anthropic_client.create([UserMessage(content="What is the capital of France?", source="user")])
+        print(result)
+
+    To load the client from a configuration:
+
+    .. code-block:: python
+
+        from autogen_core.models import ChatCompletionClient
+
+        config = {
+            "provider": "AnthropicChatCompletionClient",
+            "config": {"model": "claude-3-sonnet-20240229"},
+        }
+
+        client = ChatCompletionClient.load_component(config)
+
+    The client supports function calling with Claude models that have the capability:
+
+    .. code-block:: python
+
+        from autogen_core.tools import FunctionTool
+
+
+        def get_weather(location: str) -> str:
+            '''Get the weather for a location'''
+            return f"The weather in {location} is sunny."
+
+
+        tool = FunctionTool(get_weather)
+
+        result = await anthropic_client.create(
+            [UserMessage(content="What's the weather in Paris?", source="user")],
+            tools=[tool],
+        )
+    """
+
+    component_type = "model"
+    component_config_schema = AnthropicClientConfigurationConfigModel
+    component_provider_override = "autogen_ext.models.anthropic.AnthropicChatCompletionClient"
+
+    def __init__(self, **kwargs: Unpack[AnthropicClientConfiguration]):
+        if "model" not in kwargs:
+            raise ValueError("model is required for AnthropicChatCompletionClient")
+
+        model_capabilities: Optional[ModelCapabilities] = None  # type: ignore
+        self._raw_config: Dict[str, Any] = dict(kwargs).copy()
+        copied_args = dict(kwargs).copy()
+
+        if "model_capabilities" in kwargs:
+            model_capabilities = kwargs["model_capabilities"]
+            del copied_args["model_capabilities"]
+
+        model_info: Optional[ModelInfo] = None
+        if "model_info" in kwargs:
+            model_info = kwargs["model_info"]
+            del copied_args["model_info"]
+
+        client = _anthropic_client_from_config(copied_args)
+        create_args = _create_args_from_config(copied_args)
+
+        super().__init__(
+            client=client,
+            create_args=create_args,
+            model_capabilities=model_capabilities,
+            model_info=model_info,
+        )
+
+    def __getstate__(self) -> Dict[str, Any]:
+        state = self.__dict__.copy()
+        state["_client"] = None
+        return state
+
+    def __setstate__(self, state: Dict[str, Any]) -> None:
+        self.__dict__.update(state)
+        self._client = _anthropic_client_from_config(state["_raw_config"])
+
+    def _to_config(self) -> AnthropicClientConfigurationConfigModel:
+        copied_config = self._raw_config.copy()
+        return AnthropicClientConfigurationConfigModel(**copied_config)
+
+    @classmethod
+    def _from_config(cls, config: AnthropicClientConfigurationConfigModel) -> Self:
+        copied_config = config.model_copy().model_dump(exclude_none=True)
+        return cls(**copied_config)
diff --git a/python/packages/autogen-ext/src/autogen_ext/models/anthropic/_model_info.py b/python/packages/autogen-ext/src/autogen_ext/models/anthropic/_model_info.py
new file mode 100644
index 000000000000..3f5200925b2d
--- /dev/null
+++ b/python/packages/autogen-ext/src/autogen_ext/models/anthropic/_model_info.py
@@ -0,0 +1,106 @@
+from typing import Dict
+
+from autogen_core.models import ModelFamily, ModelInfo
+
+# Mapping of model names to their capabilities
+# For Anthropic's Claude models based on:
+# https://docs.anthropic.com/claude/docs/models-overview
+_MODEL_INFO: Dict[str, ModelInfo] = {
+    # Claude 3 Opus (most powerful)
+    "claude-3-opus-20240229": {
+        "vision": True,
+        "function_calling": True,
+        "json_output": True,
+        "family": ModelFamily.CLAUDE_3_5_SONNET,
+    },
+    # Claude 3 Sonnet (balanced)
+    "claude-3-sonnet-20240229": {
+        "vision": True,
+        "function_calling": True,
+        "json_output": True,
+        "family": ModelFamily.CLAUDE_3_5_SONNET,
+    },
+    # Claude 3 Haiku (fastest)
+    "claude-3-haiku-20240307": {
+        "vision": True,
+        "function_calling": True,
+        "json_output": True,
+        "family": ModelFamily.CLAUDE_3_5_SONNET,
+    },
+    # Claude 3.5 Sonnet
+    "claude-3-5-sonnet-20240620": {
+        "vision": True,
+        "function_calling": True,
+        "json_output": True,
+        "family": ModelFamily.CLAUDE_3_5_SONNET,
+    },
+    # Claude 3.7 Sonnet
+    "claude-3-7-sonnet-20250219": {
+        "vision": True,
+        "function_calling": True,
+        "json_output": True,
+        "family": ModelFamily.CLAUDE_3_5_SONNET,
+    },
+    # Claude Instant v1 (legacy)
+    "claude-instant-1.2": {
+        "vision": False,
+        "function_calling": False,
+        "json_output": True,
+        "family": ModelFamily.CLAUDE_3_5_SONNET,
+    },
+    # Claude 2 (legacy)
+    "claude-2.0": {
+        "vision": False,
+        "function_calling": False,
+        "json_output": True,
+        "family": ModelFamily.CLAUDE_3_5_SONNET,
+    },
+    # Claude 2.1 (legacy)
+    "claude-2.1": {
+        "vision": False,
+        "function_calling": False,
+        "json_output": True,
+        "family": ModelFamily.CLAUDE_3_5_SONNET,
+    },
+}
+
+# Model token limits (context window size)
+_MODEL_TOKEN_LIMITS: Dict[str, int] = {
+    "claude-3-opus-20240229": 200000,
+    "claude-3-sonnet-20240229": 200000,
+    "claude-3-haiku-20240307": 200000,
+    "claude-3-5-sonnet-20240620": 200000,
+    "claude-3-7-sonnet-20250219": 200000,
+    "claude-instant-1.2": 100000,
+    "claude-2.0": 100000,
+    "claude-2.1": 200000,
+}
+
+
+def get_info(model: str) -> ModelInfo:
+    """Get the model information for a specific model."""
+    # Check for exact match first
+    if model in _MODEL_INFO:
+        return _MODEL_INFO[model]
+
+    # Check for partial match (for handling model variants)
+    for model_id in _MODEL_INFO:
+        if model.startswith(model_id.split("-2")[0]):  # Match base name
+            return _MODEL_INFO[model_id]
+
+    raise KeyError(f"Model '{model}' not found in model info")
+
+
+def get_token_limit(model: str) -> int:
+    """Get the token limit for a specific model."""
+    # Check for exact match first
+    if model in _MODEL_TOKEN_LIMITS:
+        return _MODEL_TOKEN_LIMITS[model]
+
+    # Check for partial match (for handling model variants)
+    for model_id in _MODEL_TOKEN_LIMITS:
+        if model.startswith(model_id.split("-2")[0]):  # Match base name
+            return _MODEL_TOKEN_LIMITS[model_id]
+
+    # Default to a reasonable limit if model not found
+    return 100000
diff --git a/python/packages/autogen-ext/src/autogen_ext/models/anthropic/config/__init__.py b/python/packages/autogen-ext/src/autogen_ext/models/anthropic/config/__init__.py
new file mode 100644
index 000000000000..58ff248487da
--- /dev/null
+++ b/python/packages/autogen-ext/src/autogen_ext/models/anthropic/config/__init__.py
@@ -0,0 +1,63 @@
+from typing import Any, Dict, List, Literal, Optional, Union
+
+from autogen_core.models import ModelCapabilities, ModelInfo  # type: ignore
+from pydantic import BaseModel
+from typing_extensions import TypedDict
+
+
+class ResponseFormat(TypedDict):
+    type: Literal["text", "json_object"]
+
+
+class CreateArguments(TypedDict, total=False):
+    model: str
+    max_tokens: Optional[int]
+    temperature: Optional[float]
+    top_p: Optional[float]
+    top_k: Optional[int]
+    stop_sequences: Optional[List[str]]
+    response_format: Optional[ResponseFormat]
+    metadata: Optional[Dict[str, str]]
+
+
+class BaseAnthropicClientConfiguration(CreateArguments, total=False):
+    api_key: str
+    base_url: Optional[str]
+    model_capabilities: ModelCapabilities  # type: ignore
+    model_info: ModelInfo
+    """What functionality the model supports, determined by default from model name but is overridden if value passed."""
+    timeout: Optional[float]
+    max_retries: Optional[int]
+    default_headers: Optional[Dict[str, str]]
+
+
+class AnthropicClientConfiguration(BaseAnthropicClientConfiguration, total=False):
+    tools: Optional[List[Dict[str, Any]]]
+    tool_choice: Optional[Union[Literal["auto", "any", "none"], Dict[str, Any]]]
+
+
+# Pydantic equivalents of the above TypedDicts
+class CreateArgumentsConfigModel(BaseModel):
+    model: str
+    max_tokens: int | None = 4096
+    temperature: float | None = 1.0
+    top_p: float | None = None
+    top_k: int | None = None
+    stop_sequences: List[str] | None = None
+    response_format: ResponseFormat | None = None
+    metadata: Dict[str, str] | None = None
+
+
+class BaseAnthropicClientConfigurationConfigModel(CreateArgumentsConfigModel):
+    api_key: str | None = None
+    base_url: str | None = None
+    model_capabilities: ModelCapabilities | None = None  # type: ignore
+    model_info: ModelInfo | None = None
+    timeout: float | None = None
+    max_retries: int | None = None
+    default_headers: Dict[str, str] | None = None
+
+
+class AnthropicClientConfigurationConfigModel(BaseAnthropicClientConfigurationConfigModel):
+    tools: List[Dict[str, Any]] | None = None
+    tool_choice: Union[Literal["auto", "any", "none"], Dict[str, Any]] | None = None
diff --git a/python/packages/autogen-ext/tests/models/test_anthropic_model_client.py b/python/packages/autogen-ext/tests/models/test_anthropic_model_client.py
new file mode 100644
index 000000000000..cdb53a55b48e
--- /dev/null
+++ b/python/packages/autogen-ext/tests/models/test_anthropic_model_client.py
@@ -0,0 +1,301 @@
+import asyncio
+import os
+from typing import List, Sequence
+
+import pytest
+from autogen_core import CancellationToken, FunctionCall
+from autogen_core.models import (
+    AssistantMessage,
+    CreateResult,
+    FunctionExecutionResult,
+    FunctionExecutionResultMessage,
+    SystemMessage,
+    UserMessage,
+)
+from autogen_core.models._types import LLMMessage
+from autogen_core.tools import FunctionTool
+from autogen_ext.models.anthropic import AnthropicChatCompletionClient
+
+
+def _pass_function(input: str) -> str:
+    """Simple passthrough function."""
+    return f"Processed: {input}"
+
+
+def _add_numbers(a: int, b: int) -> int:
+    """Add two numbers together."""
+    return a + b
+
+
+@pytest.mark.asyncio
+async def test_anthropic_basic_completion() -> None:
+    """Test basic message completion with Claude."""
+    api_key = os.getenv("ANTHROPIC_API_KEY")
+    if not api_key:
+        pytest.skip("ANTHROPIC_API_KEY not found in environment variables")
+
+    client = AnthropicChatCompletionClient(
+        model="claude-3-haiku-20240307",  # Use haiku for faster/cheaper testing
+        api_key=api_key,
+        temperature=0.0,  # Added temperature param to test
+        stop_sequences=["STOP"],  # Added stop sequence
+    )
+
+    # Test basic completion
+    result = await client.create(
+        messages=[
+            SystemMessage(content="You are a helpful assistant."),
+            UserMessage(content="What's 2+2? Answer with just the number.", source="user"),
+        ]
+    )
+
+    assert isinstance(result.content, str)
+    assert "4" in result.content
+    assert result.finish_reason == "stop"
+
+    # Test JSON output - add to existing test
+    json_result = await client.create(
+        messages=[
+            UserMessage(content="Return a JSON with key 'value' set to 42", source="user"),
+        ],
+        json_output=True,
+    )
+    assert isinstance(json_result.content, str)
+    assert "42" in json_result.content
+
+    # Check usage tracking
+    usage = client.total_usage()
+    assert usage.prompt_tokens > 0
+    assert usage.completion_tokens > 0
+
+
+@pytest.mark.asyncio
+async def test_anthropic_streaming() -> None:
+    """Test streaming capabilities with Claude."""
+    api_key = os.getenv("ANTHROPIC_API_KEY")
+    if not api_key:
+        pytest.skip("ANTHROPIC_API_KEY not found in environment variables")
+
+    client = AnthropicChatCompletionClient(
+        model="claude-3-haiku-20240307",
+        api_key=api_key,
+    )
+
+    # Test streaming completion
+    chunks: List[str | CreateResult] = []
+    async for chunk in client.create_stream(
+        messages=[
+            UserMessage(content="Count from 1 to 5. Each number on its own line.", source="user"),
+        ]
+    ):
+        chunks.append(chunk)
+
+    # Verify we got multiple chunks
+    assert len(chunks) > 1
+
+    # Check final result
+    final_result = chunks[-1]
+    assert isinstance(final_result, CreateResult)
+    assert final_result.finish_reason == "stop"
+
+    # Check content contains numbers 1-5
+    assert isinstance(final_result.content, str)
+    combined_content = final_result.content
+    for i in range(1, 6):
+        assert str(i) in combined_content
+
+
+@pytest.mark.asyncio
+async def test_anthropic_tool_calling() -> None:
+    """Test tool calling capabilities with Claude."""
+    api_key = os.getenv("ANTHROPIC_API_KEY")
+    if not api_key:
+        pytest.skip("ANTHROPIC_API_KEY not found in environment variables")
+
+    client = AnthropicChatCompletionClient(
+        model="claude-3-haiku-20240307",
+        api_key=api_key,
+    )
+
+    # Define tools
+    pass_tool = FunctionTool(_pass_function, description="Process input text", name="process_text")
+    add_tool = FunctionTool(_add_numbers, description="Add two numbers together", name="add_numbers")
+
+    # Test tool calling with instruction to use specific tool
+    messages: List[LLMMessage] = [
+        SystemMessage(content="Use the tools available to help the user."),
+        UserMessage(content="Process the text 'hello world' using the process_text tool.", source="user"),
+    ]
+
+    result = await client.create(messages=messages, tools=[pass_tool, add_tool])
+
+    # Check that we got a tool call
+    assert isinstance(result.content, list)
+    assert len(result.content) >= 1
+    assert isinstance(result.content[0], FunctionCall)
+
+    # Check that the correct tool was called
+    function_call = result.content[0]
+    assert function_call.name == "process_text"
+
+    # Test tool response handling
+    messages.append(AssistantMessage(content=result.content, source="assistant"))
+    messages.append(
+        FunctionExecutionResultMessage(
+            content=[
+                FunctionExecutionResult(content="Processed: hello world", call_id=result.content[0].id, is_error=False)
+            ]
+        )
+    )
+
+    # Get response after tool execution
+    after_tool_result = await client.create(messages=messages)
+
+    # Check we got a text response
+    assert isinstance(after_tool_result.content, str)
+
+    # Test multiple tool use
+    multi_tool_prompt: List[LLMMessage] = [
+        SystemMessage(content="Use the tools as needed to help the user."),
+        UserMessage(content="First process the text 'test' and then add 2 and 3.", source="user"),
+    ]
+
+    multi_tool_result = await client.create(messages=multi_tool_prompt, tools=[pass_tool, add_tool])
+
+    # We just need to verify we get at least one tool call
+    assert isinstance(multi_tool_result.content, list)
+    assert len(multi_tool_result.content) > 0
+    assert isinstance(multi_tool_result.content[0], FunctionCall)
+
+
+@pytest.mark.asyncio
+async def test_anthropic_token_counting() -> None:
+    """Test token counting functionality."""
+    api_key = os.getenv("ANTHROPIC_API_KEY")
+    if not api_key:
+        pytest.skip("ANTHROPIC_API_KEY not found in environment variables")
+
+    client = AnthropicChatCompletionClient(
+        model="claude-3-haiku-20240307",
+        api_key=api_key,
+    )
+
+    messages: Sequence[LLMMessage] = [
+        SystemMessage(content="You are a helpful assistant."),
+        UserMessage(content="Hello, how are you?", source="user"),
+    ]
+
+    # Test token counting
+    num_tokens = client.count_tokens(messages)
+    assert num_tokens > 0
+
+    # Test remaining token calculation
+    remaining = client.remaining_tokens(messages)
+    assert remaining > 0
+    assert remaining < 200000  # Claude's max context
+
+    # Test token counting with tools
+    tools = [
+        FunctionTool(_pass_function, description="Process input text", name="process_text"),
+        FunctionTool(_add_numbers, description="Add two numbers together", name="add_numbers"),
+    ]
+    tokens_with_tools = client.count_tokens(messages, tools=tools)
+    assert tokens_with_tools > num_tokens  # Should be more tokens with tools
+
+
+@pytest.mark.asyncio
+async def test_anthropic_cancellation() -> None:
+    """Test cancellation of requests."""
+    api_key = os.getenv("ANTHROPIC_API_KEY")
+    if not api_key:
+        pytest.skip("ANTHROPIC_API_KEY not found in environment variables")
+
+    client = AnthropicChatCompletionClient(
+        model="claude-3-haiku-20240307",
+        api_key=api_key,
+    )
+
+    # Create a cancellation token
+    cancellation_token = CancellationToken()
+
+    # Schedule cancellation after a short delay
+    async def cancel_after_delay() -> None:
+        await asyncio.sleep(0.5)  # Short delay
+        cancellation_token.cancel()
+
+    # Start task to cancel request
+    asyncio.create_task(cancel_after_delay())
+
+    # Create a request with long output
+    with pytest.raises(asyncio.CancelledError):
+        await client.create(
+            messages=[
+                UserMessage(content="Write a detailed 5-page essay on the history of computing.", source="user"),
+            ],
+            cancellation_token=cancellation_token,
+        )
+
+
+@pytest.mark.asyncio
+async def test_anthropic_multimodal() -> None:
+    """Test multimodal capabilities with Claude."""
+    api_key = os.getenv("ANTHROPIC_API_KEY")
+    if not api_key:
+        pytest.skip("ANTHROPIC_API_KEY not found in environment variables")
+
+    # Skip if PIL is not available
+    try:
+        from autogen_core import Image
+        from PIL import Image as PILImage
+    except ImportError:
+        pytest.skip("PIL or other dependencies not installed")
+
+    client = AnthropicChatCompletionClient(
+        model="claude-3-5-sonnet-latest",  # Use a model that supports vision
+        api_key=api_key,
+    )
+
+    # Use a simple test image that's reliable
+    # 1. Create a simple colored square image
+    width, height = 100, 100
+    color = (255, 0, 0)  # Red
+    pil_image = PILImage.new("RGB", (width, height), color)
+
+    # 2. Convert to autogen_core Image format
+    img = Image(pil_image)
+
+    # Test multimodal message
+    result = await client.create(
+        messages=[
+            UserMessage(content=["What color is this square? Answer in one word.", img], source="user"),
+        ]
+    )
+
+    # Verify we got a response describing the image
+    assert isinstance(result.content, str)
+    assert len(result.content) > 0
+    assert "red" in result.content.lower()
+    assert result.finish_reason == "stop"
+
+
+@pytest.mark.asyncio
+async def test_anthropic_serialization() -> None:
+    """Test serialization and deserialization of component."""
+
+    api_key = os.getenv("ANTHROPIC_API_KEY")
+    if not api_key:
+        pytest.skip("ANTHROPIC_API_KEY not found in environment variables")
+
+    client = AnthropicChatCompletionClient(
+        model="claude-3-haiku-20240307",
+        api_key=api_key,
+    )
+
+    # Serialize and deserialize
+    model_client_config = client.dump_component()
+    assert model_client_config is not None
+    assert model_client_config.config is not None
+
+    loaded_model_client = AnthropicChatCompletionClient.load_component(model_client_config)
+    assert loaded_model_client is not None
+    assert isinstance(loaded_model_client, AnthropicChatCompletionClient)


Code review comments:
diff_hunk: @@ -0,0 +1,987 @@
+import asyncio
+import base64
+import inspect
+import json
+import logging
+import re
+import warnings
+
+# from asyncio import Task
+from typing import (
+    Any,
+    AsyncGenerator,
+    Coroutine,
+    Dict,
+    List,
+    Literal,
+    Mapping,
+    Optional,
+    Sequence,
+    Set,
+    Union,
+    cast,
+)
+
+import tiktoken
+from anthropic import AsyncAnthropic, AsyncStream
+from anthropic.types import (
+    ContentBlock,
+    ImageBlockParam,
+    Message,
+    MessageParam,
+    RawMessageStreamEvent,  # type: ignore
+    TextBlock,
+    TextBlockParam,
+    ToolParam,
+    ToolResultBlockParam,
+    ToolUseBlock,
+)
+from anthropic.types.image_block_param import Source
+from autogen_core import (
+    EVENT_LOGGER_NAME,
+    TRACE_LOGGER_NAME,
+    CancellationToken,
+    Component,
+    FunctionCall,
+    Image,
+    MessageHandlerContext,
+)
+from autogen_core.logging import LLMCallEvent
+from autogen_core.models import (
+    AssistantMessage,
+    ChatCompletionClient,
+    CreateResult,
+    FinishReasons,
+    FunctionExecutionResultMessage,
+    LLMMessage,
+    ModelCapabilities,  # type: ignore
+    ModelFamily,
+    ModelInfo,
+    RequestUsage,
+    SystemMessage,
+    UserMessage,
+)
+from autogen_core.tools import Tool, ToolSchema
+from typing_extensions import Self, Unpack
+
+from . import _model_info
+from .config import AnthropicClientConfiguration, AnthropicClientConfigurationConfigModel
+
+logger = logging.getLogger(EVENT_LOGGER_NAME)
+trace_logger = logging.getLogger(TRACE_LOGGER_NAME)
+
+# Common parameters for message creation
+anthropic_message_params = {
+    "system",
+    "messages",
+    "max_tokens",
+    "temperature",
+    "top_p",
+    "top_k",
+    "stop_sequences",
+    "tools",
+    "tool_choice",
+    "stream",
+    "metadata",
+}
+disallowed_create_args = {"stream", "messages"}
+required_create_args: Set[str] = {"model"}
+
+anthropic_init_kwargs = set(inspect.getfullargspec(AsyncAnthropic.__init__).kwonlyargs)
+
+
+def _anthropic_client_from_config(config: Mapping[str, Any]) -> AsyncAnthropic:
+    # Filter config to only include valid parameters
+    client_config = {k: v for k, v in config.items() if k in anthropic_init_kwargs}
+    return AsyncAnthropic(**client_config)
+
+
+def _create_args_from_config(config: Mapping[str, Any]) -> Dict[str, Any]:
+    create_args = {k: v for k, v in config.items() if k in anthropic_message_params or k == "model"}
+    create_args_keys = set(create_args.keys())
+
+    if not required_create_args.issubset(create_args_keys):
+        raise ValueError(f"Required create args are missing: {required_create_args - create_args_keys}")
+
+    if disallowed_create_args.intersection(create_args_keys):
+        raise ValueError(f"Disallowed create args are present: {disallowed_create_args.intersection(create_args_keys)}")
+
+    return create_args
+
+
+def type_to_role(message: LLMMessage) -> str:
+    if isinstance(message, SystemMessage):
+        return "system"
+    elif isinstance(message, UserMessage):
+        return "user"
+    elif isinstance(message, AssistantMessage):
+        return "assistant"
+    else:
+        return "tool"
+
+
+def get_mime_type_from_image(image: Image) -> Literal["image/jpeg", "image/png", "image/gif", "image/webp"]:
+    """Get a valid Anthropic media type from an Image object."""
+    # Get base64 data first
+    base64_data = image.to_base64()
+
+    # Decode the base64 string
+    image_data = base64.b64decode(base64_data)
+
+    # Check the first few bytes for known signatures
+    if image_data.startswith(b"\xff\xd8\xff"):
+        return "image/jpeg"
+    elif image_data.startswith(b"\x89PNG\r\n\x1a\n"):
+        return "image/png"
+    elif image_data.startswith(b"GIF87a") or image_data.startswith(b"GIF89a"):
+        return "image/gif"
+    elif image_data.startswith(b"RIFF") and image_data[8:12] == b"WEBP":
+        return "image/webp"
+    else:
+        # Default to JPEG as a fallback
+        return "image/jpeg"
+
+
+def user_message_to_anthropic(message: UserMessage) -> MessageParam:
+    assert_valid_name(message.source)
+
+    if isinstance(message.content, str):
+        return {
+            "role": "user",
+            "content": message.content,
+        }
+    else:
+        blocks: List[Union[TextBlockParam, ImageBlockParam]] = []
+
+        for part in message.content:
+            if isinstance(part, str):
+                blocks.append(TextBlockParam(type="text", text=part))
+            elif isinstance(part, Image):
+                blocks.append(
+                    ImageBlockParam(
+                        type="image",
+                        source=Source(
+                            type="base64",
+                            media_type=get_mime_type_from_image(part),
+                            data=part.to_base64(),
+                        ),
+                    )
+                )
+            else:
+                raise ValueError(f"Unknown content type: {part}")
+
+        return {
+            "role": "user",
+            "content": blocks,
+        }
+
+
+def system_message_to_anthropic(message: SystemMessage) -> str:
+    return message.content
+
+
+def assistant_message_to_anthropic(message: AssistantMessage) -> MessageParam:
+    assert_valid_name(message.source)
+
+    if isinstance(message.content, list):
+        # Tool calls
+        tool_use_blocks: List[ToolUseBlock] = []
+
+        for func_call in message.content:
+            # Parse the arguments and convert to dict if it's a JSON string
+            args = func_call.arguments
+            if isinstance(args, str):
+                try:
+                    args_dict = json.loads(args)
+                except json.JSONDecodeError:
+                    args_dict = {"text": args}
+            else:
+                args_dict = args
+
+            tool_use_blocks.append(
+                ToolUseBlock(
+                    type="tool_use",
+                    id=func_call.id,
+                    name=func_call.name,
+                    input=args_dict,
+                )
+            )
+
+        # Include thought if available
+        content_blocks: List[ContentBlock] = []
+        if hasattr(message, "thought") and message.thought is not None:
+            content_blocks.append(TextBlock(type="text", text=message.thought))
+
+        content_blocks.extend(tool_use_blocks)
+
+        return {
+            "role": "assistant",
+            "content": content_blocks,
+        }
+    else:
+        # Simple text content
+        return {
+            "role": "assistant",
+            "content": message.content,
+        }
+
+
+def tool_message_to_anthropic(message: FunctionExecutionResultMessage) -> List[MessageParam]:
+    # Create a single user message containing all tool results
+    content_blocks: List[ToolResultBlockParam] = []
+
+    for result in message.content:
+        content_blocks.append(
+            ToolResultBlockParam(
+                type="tool_result",
+                tool_use_id=result.call_id,
+                content=result.content,
+            )
+        )
+
+    return [
+        {
+            "role": "user",  # Changed from "tool" to "user"
+            "content": content_blocks,
+        }
+    ]
+
+
+def to_anthropic_type(message: LLMMessage) -> Union[str, List[MessageParam], MessageParam]:
+    if isinstance(message, SystemMessage):
+        return system_message_to_anthropic(message)
+    elif isinstance(message, UserMessage):
+        return user_message_to_anthropic(message)
+    elif isinstance(message, AssistantMessage):
+        return assistant_message_to_anthropic(message)
+    else:
+        return tool_message_to_anthropic(message)
+
+
+def convert_tools(tools: Sequence[Tool | ToolSchema]) -> List[ToolParam]:
+    result: List[ToolParam] = []
+
+    for tool in tools:
+        if isinstance(tool, Tool):
+            tool_schema = tool.schema
+        else:
+            assert isinstance(tool, dict)
+            tool_schema = tool
+
+        # Convert parameters to match Anthropic's schema format
+        tool_params: Dict[str, Any] = {}
+        if "parameters" in tool_schema:
+            params = tool_schema["parameters"]
+
+            # Transfer properties
+            if "properties" in params:
+                tool_params["properties"] = params["properties"]
+
+            # Transfer required fields
+            if "required" in params:
+                tool_params["required"] = params["required"]
+
+            # Handle schema type
+            if "type" in params:
+                tool_params["type"] = params["type"]
+            else:
+                tool_params["type"] = "object"
+
+        result.append(
+            ToolParam(
+                name=tool_schema["name"],
+                input_schema=tool_params,
+                description=tool_schema.get("description", ""),
+            )
+        )
+
+        # Check if the tool has a valid name
+        assert_valid_name(tool_schema["name"])
+
+    return result
+
+
+def normalize_name(name: str) -> str:
+    """
+
+    def __init__(self, **kwargs: Unpack[AnthropicClientConfiguration]):
+        if "model" not in kwargs:
+            raise ValueError("model is required for AnthropicChatCompletionClient")
+
+        model_capabilities: Optional[ModelCapabilities] = None  # type: ignore
+        self._raw_config: Dict[str, Any] = dict(kwargs).copy()
+        copied_args = dict(kwargs).copy()
+
+        if "model_capabilities" in kwargs:
+            model_capabilities = kwargs["model_capabilities"]
+            del copied_args["model_capabilities"]
+
+        model_info: Optional[ModelInfo] = None
+        if "model_info" in kwargs:
+            model_info = kwargs["model_info"]
+            del copied_args["model_info"]
+
+        client = _anthropic_client_from_config(copied_args)
+        create_args = _create_args_from_config(copied_args)
+
+        super().__init__(
+            client=client,
+            create_args=create_args,
+            model_capabilities=model_capabilities,
+            model_info=model_info,
+        )
+
+    def __getstate__(self) -> Dict[str, Any]:
+        state = self.__dict__.copy()
+        state["_client"] = None
+        return state
+
+    def __setstate__(self, state: Dict[str, Any]) -> None:
+        self.__dict__.update(state)
+        self._client = _anthropic_client_from_config(state["_raw_config"])
+
+    def _to_config(self) -> AnthropicClientConfigurationConfigModel:
+        copied_config = self._raw_config.copy()
+        return AnthropicClientConfigurationConfigModel(**copied_config)
+
+    @classmethod
+    def _from_config(cls, config: AnthropicClientConfigurationConfigModel) -> Self:
+        copied_config = config.model_copy().model_dump(exclude_none=True)
+        return cls(**copied_config)
+    Normalize names by replacing invalid characters with underscore.
+    """
+    return re.sub(r"[^a-zA-Z0-9_-]", "_", name)[:64]
+
+
+def assert_valid_name(name: str) -> str:
+    """
+    Ensure that configured names are valid, raises ValueError if not.
+    """
+    if not re.match(r"^[a-zA-Z0-9_-]+$", name):
+        raise ValueError(f"Invalid name: {name}. Only letters, numbers, '_' and '-' are allowed.")
+    if len(name) > 64:
+        raise ValueError(f"Invalid name: {name}. Name must be less than 64 characters.")
+    return name
+
+
+def normalize_stop_reason(stop_reason: str | None) -> FinishReasons:
+    if stop_reason is None:
+        return "unknown"
+
+    # Convert to lowercase for comparison
+    stop_reason = stop_reason.lower()
+
+    # Map Anthropic stop reasons to standard reasons
+    KNOWN_STOP_MAPPINGS: Dict[str, FinishReasons] = {
+        "end_turn": "stop",
+        "max_tokens": "length",
+        "stop_sequence": "stop",
+        "tool_use": "function_calls",
+    }
+
+    return KNOWN_STOP_MAPPINGS.get(stop_reason, "unknown")
+
+
+def _add_usage(usage1: RequestUsage, usage2: RequestUsage) -> RequestUsage:
+    return RequestUsage(
+        prompt_tokens=usage1.prompt_tokens + usage2.prompt_tokens,
+        completion_tokens=usage1.completion_tokens + usage2.completion_tokens,
+    )
+
+
+class BaseAnthropicChatCompletionClient(ChatCompletionClient):
+    def __init__(
+        self,
+        client: AsyncAnthropic,
+        *,
+        create_args: Dict[str, Any],
+        model_capabilities: Optional[ModelCapabilities] = None,  # type: ignore
+        model_info: Optional[ModelInfo] = None,
+    ):
+        self._client = client
+
+        # Handle model info and capabilities
+        if model_capabilities is None and model_info is None:
+            try:
+                self._model_info = _model_info.get_info(create_args["model"])
+            except KeyError as err:
+                raise ValueError("model_info is required when model name is not recognized") from err
+        elif model_capabilities is not None and model_info is not None:
+            raise ValueError("model_capabilities and model_info are mutually exclusive")
+        elif model_capabilities is not None and model_info is None:
+            warnings.warn("model_capabilities is deprecated, use model_info instead", DeprecationWarning, stacklevel=2)
+            info = cast(ModelInfo, model_capabilities)
+            info["family"] = ModelFamily.CLAUDE_3_5_SONNET
+            self._model_info = info
+        elif model_capabilities is None and model_info is not None:
+            self._model_info = model_info
+
+        # Validate model_info
+        if not all(key in self._model_info for key in ["vision", "function_calling", "json_output", "family"]):
path: python/packages/autogen-ext/src/autogen_ext/models/anthropic/_anthropic_client.py
text: @user1:
There is a helper function autogen_core.models.validate_model_info that does the same thing. 

@author:
Added

diff_hunk: @@ -0,0 +1,106 @@
+from typing import Dict
+
+from autogen_core.models import ModelFamily, ModelInfo
+
+# Mapping of model names to their capabilities
+# For Anthropic's Claude models based on:
+# https://docs.anthropic.com/claude/docs/models-overview
+_MODEL_INFO: Dict[str, ModelInfo] = {
+    # Claude 3 Opus (most powerful)
+    "claude-3-opus-20240229": {
+        "vision": True,
+        "function_calling": True,
+        "json_output": True,
+        "family": ModelFamily.CLAUDE_3_5_SONNET,
+    },
+    # Claude 3 Sonnet (balanced)
+    "claude-3-sonnet-20240229": {
+        "vision": True,
+        "function_calling": True,
+        "json_output": True,
+        "family": ModelFamily.CLAUDE_3_5_SONNET,
+    },
+    # Claude 3 Haiku (fastest)
+    "claude-3-haiku-20240307": {
+        "vision": True,
+        "function_calling": True,
+        "json_output": True,
+        "family": ModelFamily.CLAUDE_3_5_SONNET,
+    },
+    # Claude 3.5 Sonnet
+    "claude-3-5-sonnet-20240620": {
+        "vision": True,
+        "function_calling": True,
+        "json_output": True,
+        "family": ModelFamily.CLAUDE_3_5_SONNET,
+    },
+    # Claude 3.7 Sonnet
+    "claude-3-7-sonnet-20250219": {
+        "vision": True,
+        "function_calling": True,
+        "json_output": True,
+        "family": ModelFamily.CLAUDE_3_5_SONNET,
path: python/packages/autogen-ext/src/autogen_ext/models/anthropic/_model_info.py
text: @user1:
Should we create a new model family for 3.7?

diff_hunk: @@ -0,0 +1,987 @@
+import asyncio
+import base64
+import inspect
+import json
+import logging
+import re
+import warnings
+
+# from asyncio import Task
+from typing import (
+    Any,
+    AsyncGenerator,
+    Coroutine,
+    Dict,
+    List,
+    Literal,
+    Mapping,
+    Optional,
+    Sequence,
+    Set,
+    Union,
+    cast,
+)
+
+import tiktoken
+from anthropic import AsyncAnthropic, AsyncStream
+from anthropic.types import (
+    ContentBlock,
+    ImageBlockParam,
+    Message,
+    MessageParam,
+    RawMessageStreamEvent,  # type: ignore
+    TextBlock,
+    TextBlockParam,
+    ToolParam,
+    ToolResultBlockParam,
+    ToolUseBlock,
+)
+from anthropic.types.image_block_param import Source
+from autogen_core import (
+    EVENT_LOGGER_NAME,
+    TRACE_LOGGER_NAME,
+    CancellationToken,
+    Component,
+    FunctionCall,
+    Image,
+    MessageHandlerContext,
+)
+from autogen_core.logging import LLMCallEvent
+from autogen_core.models import (
+    AssistantMessage,
+    ChatCompletionClient,
+    CreateResult,
+    FinishReasons,
+    FunctionExecutionResultMessage,
+    LLMMessage,
+    ModelCapabilities,  # type: ignore
+    ModelFamily,
+    ModelInfo,
+    RequestUsage,
+    SystemMessage,
+    UserMessage,
+)
+from autogen_core.tools import Tool, ToolSchema
+from typing_extensions import Self, Unpack
+
+from . import _model_info
+from .config import AnthropicClientConfiguration, AnthropicClientConfigurationConfigModel
+
+logger = logging.getLogger(EVENT_LOGGER_NAME)
+trace_logger = logging.getLogger(TRACE_LOGGER_NAME)
+
+# Common parameters for message creation
+anthropic_message_params = {
+    "system",
+    "messages",
+    "max_tokens",
+    "temperature",
+    "top_p",
+    "top_k",
+    "stop_sequences",
+    "tools",
+    "tool_choice",
+    "stream",
+    "metadata",
+}
+disallowed_create_args = {"stream", "messages"}
+required_create_args: Set[str] = {"model"}
+
+anthropic_init_kwargs = set(inspect.getfullargspec(AsyncAnthropic.__init__).kwonlyargs)
+
+
+def _anthropic_client_from_config(config: Mapping[str, Any]) -> AsyncAnthropic:
+    # Filter config to only include valid parameters
+    client_config = {k: v for k, v in config.items() if k in anthropic_init_kwargs}
+    return AsyncAnthropic(**client_config)
+
+
+def _create_args_from_config(config: Mapping[str, Any]) -> Dict[str, Any]:
+    create_args = {k: v for k, v in config.items() if k in anthropic_message_params or k == "model"}
+    create_args_keys = set(create_args.keys())
+
+    if not required_create_args.issubset(create_args_keys):
+        raise ValueError(f"Required create args are missing: {required_create_args - create_args_keys}")
+
+    if disallowed_create_args.intersection(create_args_keys):
+        raise ValueError(f"Disallowed create args are present: {disallowed_create_args.intersection(create_args_keys)}")
+
+    return create_args
+
+
+def type_to_role(message: LLMMessage) -> str:
+    if isinstance(message, SystemMessage):
+        return "system"
+    elif isinstance(message, UserMessage):
+        return "user"
+    elif isinstance(message, AssistantMessage):
+        return "assistant"
+    else:
+        return "tool"
+
+
+def get_mime_type_from_image(image: Image) -> Literal["image/jpeg", "image/png", "image/gif", "image/webp"]:
+    """Get a valid Anthropic media type from an Image object."""
+    # Get base64 data first
+    base64_data = image.to_base64()
+
+    # Decode the base64 string
+    image_data = base64.b64decode(base64_data)
+
+    # Check the first few bytes for known signatures
+    if image_data.startswith(b"\xff\xd8\xff"):
+        return "image/jpeg"
+    elif image_data.startswith(b"\x89PNG\r\n\x1a\n"):
+        return "image/png"
+    elif image_data.startswith(b"GIF87a") or image_data.startswith(b"GIF89a"):
+        return "image/gif"
+    elif image_data.startswith(b"RIFF") and image_data[8:12] == b"WEBP":
+        return "image/webp"
+    else:
+        # Default to JPEG as a fallback
+        return "image/jpeg"
+
+
+def user_message_to_anthropic(message: UserMessage) -> MessageParam:
+    assert_valid_name(message.source)
+
+    if isinstance(message.content, str):
+        return {
+            "role": "user",
+            "content": message.content,
+        }
+    else:
+        blocks: List[Union[TextBlockParam, ImageBlockParam]] = []
+
+        for part in message.content:
+            if isinstance(part, str):
+                blocks.append(TextBlockParam(type="text", text=part))
+            elif isinstance(part, Image):
+                blocks.append(
+                    ImageBlockParam(
+                        type="image",
+                        source=Source(
+                            type="base64",
+                            media_type=get_mime_type_from_image(part),
+                            data=part.to_base64(),
+                        ),
+                    )
+                )
+            else:
+                raise ValueError(f"Unknown content type: {part}")
+
+        return {
+            "role": "user",
+            "content": blocks,
+        }
+
+
+def system_message_to_anthropic(message: SystemMessage) -> str:
+    return message.content
+
+
+def assistant_message_to_anthropic(message: AssistantMessage) -> MessageParam:
+    assert_valid_name(message.source)
+
+    if isinstance(message.content, list):
+        # Tool calls
+        tool_use_blocks: List[ToolUseBlock] = []
+
+        for func_call in message.content:
+            # Parse the arguments and convert to dict if it's a JSON string
+            args = func_call.arguments
+            if isinstance(args, str):
+                try:
+                    args_dict = json.loads(args)
+                except json.JSONDecodeError:
+                    args_dict = {"text": args}
+            else:
+                args_dict = args
+
+            tool_use_blocks.append(
+                ToolUseBlock(
+                    type="tool_use",
+                    id=func_call.id,
+                    name=func_call.name,
+                    input=args_dict,
+                )
+            )
+
+        # Include thought if available
+        content_blocks: List[ContentBlock] = []
+        if hasattr(message, "thought") and message.thought is not None:
+            content_blocks.append(TextBlock(type="text", text=message.thought))
+
+        content_blocks.extend(tool_use_blocks)
+
+        return {
+            "role": "assistant",
+            "content": content_blocks,
+        }
+    else:
+        # Simple text content
+        return {
+            "role": "assistant",
+            "content": message.content,
+        }
+
+
+def tool_message_to_anthropic(message: FunctionExecutionResultMessage) -> List[MessageParam]:
+    # Create a single user message containing all tool results
+    content_blocks: List[ToolResultBlockParam] = []
+
+    for result in message.content:
+        content_blocks.append(
+            ToolResultBlockParam(
+                type="tool_result",
+                tool_use_id=result.call_id,
+                content=result.content,
+            )
+        )
+
+    return [
+        {
+            "role": "user",  # Changed from "tool" to "user"
+            "content": content_blocks,
+        }
+    ]
+
+
+def to_anthropic_type(message: LLMMessage) -> Union[str, List[MessageParam], MessageParam]:
+    if isinstance(message, SystemMessage):
+        return system_message_to_anthropic(message)
+    elif isinstance(message, UserMessage):
+        return user_message_to_anthropic(message)
+    elif isinstance(message, AssistantMessage):
+        return assistant_message_to_anthropic(message)
+    else:
+        return tool_message_to_anthropic(message)
+
+
+def convert_tools(tools: Sequence[Tool | ToolSchema]) -> List[ToolParam]:
+    result: List[ToolParam] = []
+
+    for tool in tools:
+        if isinstance(tool, Tool):
+            tool_schema = tool.schema
+        else:
+            assert isinstance(tool, dict)
+            tool_schema = tool
+
+        # Convert parameters to match Anthropic's schema format
+        tool_params: Dict[str, Any] = {}
+        if "parameters" in tool_schema:
+            params = tool_schema["parameters"]
+
+            # Transfer properties
+            if "properties" in params:
+                tool_params["properties"] = params["properties"]
+
+            # Transfer required fields
+            if "required" in params:
+                tool_params["required"] = params["required"]
+
+            # Handle schema type
+            if "type" in params:
+                tool_params["type"] = params["type"]
+            else:
+                tool_params["type"] = "object"
+
+        result.append(
+            ToolParam(
+                name=tool_schema["name"],
+                input_schema=tool_params,
+                description=tool_schema.get("description", ""),
+            )
+        )
+
+        # Check if the tool has a valid name
+        assert_valid_name(tool_schema["name"])
+
+    return result
+
+
+def normalize_name(name: str) -> str:
+    """
+
+    def __init__(self, **kwargs: Unpack[AnthropicClientConfiguration]):
+        if "model" not in kwargs:
+            raise ValueError("model is required for AnthropicChatCompletionClient")
+
+        model_capabilities: Optional[ModelCapabilities] = None  # type: ignore
+        self._raw_config: Dict[str, Any] = dict(kwargs).copy()
+        copied_args = dict(kwargs).copy()
+
+        if "model_capabilities" in kwargs:
+            model_capabilities = kwargs["model_capabilities"]
+            del copied_args["model_capabilities"]
+
+        model_info: Optional[ModelInfo] = None
+        if "model_info" in kwargs:
+            model_info = kwargs["model_info"]
+            del copied_args["model_info"]
+
+        client = _anthropic_client_from_config(copied_args)
+        create_args = _create_args_from_config(copied_args)
+
+        super().__init__(
+            client=client,
+            create_args=create_args,
+            model_capabilities=model_capabilities,
+            model_info=model_info,
+        )
+
+    def __getstate__(self) -> Dict[str, Any]:
+        state = self.__dict__.copy()
+        state["_client"] = None
+        return state
+
+    def __setstate__(self, state: Dict[str, Any]) -> None:
+        self.__dict__.update(state)
+        self._client = _anthropic_client_from_config(state["_raw_config"])
+
+    def _to_config(self) -> AnthropicClientConfigurationConfigModel:
+        copied_config = self._raw_config.copy()
+        return AnthropicClientConfigurationConfigModel(**copied_config)
+
+    @classmethod
+    def _from_config(cls, config: AnthropicClientConfigurationConfigModel) -> Self:
+        copied_config = config.model_copy().model_dump(exclude_none=True)
+        return cls(**copied_config)
+    Normalize names by replacing invalid characters with underscore.
+    """
+    return re.sub(r"[^a-zA-Z0-9_-]", "_", name)[:64]
+
+
+def assert_valid_name(name: str) -> str:
+    """
+    Ensure that configured names are valid, raises ValueError if not.
+    """
+    if not re.match(r"^[a-zA-Z0-9_-]+$", name):
+        raise ValueError(f"Invalid name: {name}. Only letters, numbers, '_' and '-' are allowed.")
+    if len(name) > 64:
+        raise ValueError(f"Invalid name: {name}. Name must be less than 64 characters.")
+    return name
+
+
+def normalize_stop_reason(stop_reason: str | None) -> FinishReasons:
+    if stop_reason is None:
+        return "unknown"
+
+    # Convert to lowercase for comparison
+    stop_reason = stop_reason.lower()
+
+    # Map Anthropic stop reasons to standard reasons
+    KNOWN_STOP_MAPPINGS: Dict[str, FinishReasons] = {
+        "end_turn": "stop",
+        "max_tokens": "length",
+        "stop_sequence": "stop",
+        "tool_use": "function_calls",
+    }
+
+    return KNOWN_STOP_MAPPINGS.get(stop_reason, "unknown")
+
+
+def _add_usage(usage1: RequestUsage, usage2: RequestUsage) -> RequestUsage:
+    return RequestUsage(
+        prompt_tokens=usage1.prompt_tokens + usage2.prompt_tokens,
+        completion_tokens=usage1.completion_tokens + usage2.completion_tokens,
+    )
+
+
+class BaseAnthropicChatCompletionClient(ChatCompletionClient):
+    def __init__(
+        self,
+        client: AsyncAnthropic,
+        *,
+        create_args: Dict[str, Any],
+        model_capabilities: Optional[ModelCapabilities] = None,  # type: ignore
path: python/packages/autogen-ext/src/autogen_ext/models/anthropic/_anthropic_client.py
text: @user1:
Let's not worry about model capabilities cause it's deprecated long ago

diff_hunk: @@ -0,0 +1,987 @@
+import asyncio
+import base64
+import inspect
+import json
+import logging
+import re
+import warnings
+
+# from asyncio import Task
+from typing import (
+    Any,
+    AsyncGenerator,
+    Coroutine,
+    Dict,
+    List,
+    Literal,
+    Mapping,
+    Optional,
+    Sequence,
+    Set,
+    Union,
+    cast,
+)
+
+import tiktoken
+from anthropic import AsyncAnthropic, AsyncStream
+from anthropic.types import (
+    ContentBlock,
+    ImageBlockParam,
+    Message,
+    MessageParam,
+    RawMessageStreamEvent,  # type: ignore
+    TextBlock,
+    TextBlockParam,
+    ToolParam,
+    ToolResultBlockParam,
+    ToolUseBlock,
+)
+from anthropic.types.image_block_param import Source
+from autogen_core import (
+    EVENT_LOGGER_NAME,
+    TRACE_LOGGER_NAME,
+    CancellationToken,
+    Component,
+    FunctionCall,
+    Image,
+    MessageHandlerContext,
+)
+from autogen_core.logging import LLMCallEvent
+from autogen_core.models import (
+    AssistantMessage,
+    ChatCompletionClient,
+    CreateResult,
+    FinishReasons,
+    FunctionExecutionResultMessage,
+    LLMMessage,
+    ModelCapabilities,  # type: ignore
+    ModelFamily,
+    ModelInfo,
+    RequestUsage,
+    SystemMessage,
+    UserMessage,
+)
+from autogen_core.tools import Tool, ToolSchema
+from typing_extensions import Self, Unpack
+
+from . import _model_info
+from .config import AnthropicClientConfiguration, AnthropicClientConfigurationConfigModel
+
+logger = logging.getLogger(EVENT_LOGGER_NAME)
+trace_logger = logging.getLogger(TRACE_LOGGER_NAME)
+
+# Common parameters for message creation
+anthropic_message_params = {
+    "system",
+    "messages",
+    "max_tokens",
+    "temperature",
+    "top_p",
+    "top_k",
+    "stop_sequences",
+    "tools",
+    "tool_choice",
+    "stream",
+    "metadata",
+}
+disallowed_create_args = {"stream", "messages"}
+required_create_args: Set[str] = {"model"}
+
+anthropic_init_kwargs = set(inspect.getfullargspec(AsyncAnthropic.__init__).kwonlyargs)
+
+
+def _anthropic_client_from_config(config: Mapping[str, Any]) -> AsyncAnthropic:
+    # Filter config to only include valid parameters
+    client_config = {k: v for k, v in config.items() if k in anthropic_init_kwargs}
+    return AsyncAnthropic(**client_config)
+
+
+def _create_args_from_config(config: Mapping[str, Any]) -> Dict[str, Any]:
+    create_args = {k: v for k, v in config.items() if k in anthropic_message_params or k == "model"}
+    create_args_keys = set(create_args.keys())
+
+    if not required_create_args.issubset(create_args_keys):
+        raise ValueError(f"Required create args are missing: {required_create_args - create_args_keys}")
+
+    if disallowed_create_args.intersection(create_args_keys):
+        raise ValueError(f"Disallowed create args are present: {disallowed_create_args.intersection(create_args_keys)}")
+
+    return create_args
+
+
+def type_to_role(message: LLMMessage) -> str:
+    if isinstance(message, SystemMessage):
+        return "system"
+    elif isinstance(message, UserMessage):
+        return "user"
+    elif isinstance(message, AssistantMessage):
+        return "assistant"
+    else:
+        return "tool"
+
+
+def get_mime_type_from_image(image: Image) -> Literal["image/jpeg", "image/png", "image/gif", "image/webp"]:
+    """Get a valid Anthropic media type from an Image object."""
+    # Get base64 data first
+    base64_data = image.to_base64()
+
+    # Decode the base64 string
+    image_data = base64.b64decode(base64_data)
+
+    # Check the first few bytes for known signatures
+    if image_data.startswith(b"\xff\xd8\xff"):
+        return "image/jpeg"
+    elif image_data.startswith(b"\x89PNG\r\n\x1a\n"):
+        return "image/png"
+    elif image_data.startswith(b"GIF87a") or image_data.startswith(b"GIF89a"):
+        return "image/gif"
+    elif image_data.startswith(b"RIFF") and image_data[8:12] == b"WEBP":
+        return "image/webp"
+    else:
+        # Default to JPEG as a fallback
+        return "image/jpeg"
+
+
+def user_message_to_anthropic(message: UserMessage) -> MessageParam:
+    assert_valid_name(message.source)
+
+    if isinstance(message.content, str):
+        return {
+            "role": "user",
+            "content": message.content,
+        }
+    else:
+        blocks: List[Union[TextBlockParam, ImageBlockParam]] = []
+
+        for part in message.content:
+            if isinstance(part, str):
+                blocks.append(TextBlockParam(type="text", text=part))
+            elif isinstance(part, Image):
+                blocks.append(
+                    ImageBlockParam(
+                        type="image",
+                        source=Source(
+                            type="base64",
+                            media_type=get_mime_type_from_image(part),
+                            data=part.to_base64(),
+                        ),
+                    )
+                )
+            else:
+                raise ValueError(f"Unknown content type: {part}")
+
+        return {
+            "role": "user",
+            "content": blocks,
+        }
+
+
+def system_message_to_anthropic(message: SystemMessage) -> str:
+    return message.content
+
+
+def assistant_message_to_anthropic(message: AssistantMessage) -> MessageParam:
+    assert_valid_name(message.source)
+
+    if isinstance(message.content, list):
+        # Tool calls
+        tool_use_blocks: List[ToolUseBlock] = []
+
+        for func_call in message.content:
+            # Parse the arguments and convert to dict if it's a JSON string
+            args = func_call.arguments
+            if isinstance(args, str):
+                try:
+                    args_dict = json.loads(args)
+                except json.JSONDecodeError:
+                    args_dict = {"text": args}
+            else:
+                args_dict = args
+
+            tool_use_blocks.append(
+                ToolUseBlock(
+                    type="tool_use",
+                    id=func_call.id,
+                    name=func_call.name,
+                    input=args_dict,
+                )
+            )
+
+        # Include thought if available
+        content_blocks: List[ContentBlock] = []
+        if hasattr(message, "thought") and message.thought is not None:
+            content_blocks.append(TextBlock(type="text", text=message.thought))
+
+        content_blocks.extend(tool_use_blocks)
+
+        return {
+            "role": "assistant",
+            "content": content_blocks,
+        }
+    else:
+        # Simple text content
+        return {
+            "role": "assistant",
+            "content": message.content,
+        }
+
+
+def tool_message_to_anthropic(message: FunctionExecutionResultMessage) -> List[MessageParam]:
+    # Create a single user message containing all tool results
+    content_blocks: List[ToolResultBlockParam] = []
+
+    for result in message.content:
+        content_blocks.append(
+            ToolResultBlockParam(
+                type="tool_result",
+                tool_use_id=result.call_id,
+                content=result.content,
+            )
+        )
+
+    return [
+        {
+            "role": "user",  # Changed from "tool" to "user"
+            "content": content_blocks,
+        }
+    ]
+
+
+def to_anthropic_type(message: LLMMessage) -> Union[str, List[MessageParam], MessageParam]:
+    if isinstance(message, SystemMessage):
+        return system_message_to_anthropic(message)
+    elif isinstance(message, UserMessage):
+        return user_message_to_anthropic(message)
+    elif isinstance(message, AssistantMessage):
+        return assistant_message_to_anthropic(message)
+    else:
+        return tool_message_to_anthropic(message)
+
+
+def convert_tools(tools: Sequence[Tool | ToolSchema]) -> List[ToolParam]:
+    result: List[ToolParam] = []
+
+    for tool in tools:
+        if isinstance(tool, Tool):
+            tool_schema = tool.schema
+        else:
+            assert isinstance(tool, dict)
+            tool_schema = tool
+
+        # Convert parameters to match Anthropic's schema format
+        tool_params: Dict[str, Any] = {}
+        if "parameters" in tool_schema:
+            params = tool_schema["parameters"]
+
+            # Transfer properties
+            if "properties" in params:
+                tool_params["properties"] = params["properties"]
+
+            # Transfer required fields
+            if "required" in params:
+                tool_params["required"] = params["required"]
+
+            # Handle schema type
+            if "type" in params:
+                tool_params["type"] = params["type"]
+            else:
+                tool_params["type"] = "object"
+
+        result.append(
+            ToolParam(
+                name=tool_schema["name"],
+                input_schema=tool_params,
+                description=tool_schema.get("description", ""),
+            )
+        )
+
+        # Check if the tool has a valid name
+        assert_valid_name(tool_schema["name"])
+
+    return result
+
+
+def normalize_name(name: str) -> str:
+    """
+
+    def __init__(self, **kwargs: Unpack[AnthropicClientConfiguration]):
+        if "model" not in kwargs:
+            raise ValueError("model is required for AnthropicChatCompletionClient")
+
+        model_capabilities: Optional[ModelCapabilities] = None  # type: ignore
+        self._raw_config: Dict[str, Any] = dict(kwargs).copy()
+        copied_args = dict(kwargs).copy()
+
+        if "model_capabilities" in kwargs:
+            model_capabilities = kwargs["model_capabilities"]
+            del copied_args["model_capabilities"]
+
+        model_info: Optional[ModelInfo] = None
+        if "model_info" in kwargs:
+            model_info = kwargs["model_info"]
+            del copied_args["model_info"]
+
+        client = _anthropic_client_from_config(copied_args)
+        create_args = _create_args_from_config(copied_args)
+
+        super().__init__(
+            client=client,
+            create_args=create_args,
+            model_capabilities=model_capabilities,
+            model_info=model_info,
+        )
+
+    def __getstate__(self) -> Dict[str, Any]:
+        state = self.__dict__.copy()
+        state["_client"] = None
+        return state
+
+    def __setstate__(self, state: Dict[str, Any]) -> None:
+        self.__dict__.update(state)
+        self._client = _anthropic_client_from_config(state["_raw_config"])
+
+    def _to_config(self) -> AnthropicClientConfigurationConfigModel:
+        copied_config = self._raw_config.copy()
+        return AnthropicClientConfigurationConfigModel(**copied_config)
+
+    @classmethod
+    def _from_config(cls, config: AnthropicClientConfigurationConfigModel) -> Self:
+        copied_config = config.model_copy().model_dump(exclude_none=True)
+        return cls(**copied_config)
+    Normalize names by replacing invalid characters with underscore.
+    """
+    return re.sub(r"[^a-zA-Z0-9_-]", "_", name)[:64]
+
+
+def assert_valid_name(name: str) -> str:
+    """
+    Ensure that configured names are valid, raises ValueError if not.
+    """
+    if not re.match(r"^[a-zA-Z0-9_-]+$", name):
+        raise ValueError(f"Invalid name: {name}. Only letters, numbers, '_' and '-' are allowed.")
+    if len(name) > 64:
+        raise ValueError(f"Invalid name: {name}. Name must be less than 64 characters.")
+    return name
+
+
+def normalize_stop_reason(stop_reason: str | None) -> FinishReasons:
+    if stop_reason is None:
+        return "unknown"
+
+    # Convert to lowercase for comparison
+    stop_reason = stop_reason.lower()
+
+    # Map Anthropic stop reasons to standard reasons
+    KNOWN_STOP_MAPPINGS: Dict[str, FinishReasons] = {
+        "end_turn": "stop",
+        "max_tokens": "length",
+        "stop_sequence": "stop",
+        "tool_use": "function_calls",
+    }
+
+    return KNOWN_STOP_MAPPINGS.get(stop_reason, "unknown")
+
+
+def _add_usage(usage1: RequestUsage, usage2: RequestUsage) -> RequestUsage:
+    return RequestUsage(
+        prompt_tokens=usage1.prompt_tokens + usage2.prompt_tokens,
+        completion_tokens=usage1.completion_tokens + usage2.completion_tokens,
+    )
+
+
+class BaseAnthropicChatCompletionClient(ChatCompletionClient):
+    def __init__(
+        self,
+        client: AsyncAnthropic,
+        *,
+        create_args: Dict[str, Any],
+        model_capabilities: Optional[ModelCapabilities] = None,  # type: ignore
+        model_info: Optional[ModelInfo] = None,
+    ):
+        self._client = client
+
+        # Handle model info and capabilities
+        if model_capabilities is None and model_info is None:
+            try:
+                self._model_info = _model_info.get_info(create_args["model"])
+            except KeyError as err:
+                raise ValueError("model_info is required when model name is not recognized") from err
+        elif model_capabilities is not None and model_info is not None:
+            raise ValueError("model_capabilities and model_info are mutually exclusive")
+        elif model_capabilities is not None and model_info is None:
+            warnings.warn("model_capabilities is deprecated, use model_info instead", DeprecationWarning, stacklevel=2)
+            info = cast(ModelInfo, model_capabilities)
+            info["family"] = ModelFamily.CLAUDE_3_5_SONNET
+            self._model_info = info
+        elif model_capabilities is None and model_info is not None:
+            self._model_info = model_info
+
+        # Validate model_info
+        if not all(key in self._model_info for key in ["vision", "function_calling", "json_output", "family"]):
+            raise ValueError("model_info must contain 'vision', 'function_calling', 'json_output', and 'family' keys")
+
+        self._create_args = create_args
+        self._total_usage = RequestUsage(prompt_tokens=0, completion_tokens=0)
+        self._actual_usage = RequestUsage(prompt_tokens=0, completion_tokens=0)
+
+    async def create(
+        self,
+        messages: Sequence[LLMMessage],
+        *,
+        tools: Sequence[Tool | ToolSchema] = [],
+        json_output: Optional[bool] = None,
+        extra_create_args: Mapping[str, Any] = {},
+        cancellation_token: Optional[CancellationToken] = None,
+    ) -> CreateResult:
+        # Copy create args and update with extra args
+        create_args = self._create_args.copy()
+        create_args.update(extra_create_args)
+
+        # Check for vision capability if images are present
+        if self.model_info["vision"] is False:
+            for message in messages:
+                if isinstance(message, UserMessage):
+                    if isinstance(message.content, list) and any(isinstance(x, Image) for x in message.content):
+                        raise ValueError("Model does not support vision and image was provided")
+
+        # Handle JSON output format
+        if json_output is not None:
+            if self.model_info["json_output"] is False and json_output is True:
+                raise ValueError("Model does not support JSON output")
+
+            if json_output is True:
+                create_args["response_format"] = {"type": "json_object"}
+
+        # Process system message separately
+        system_message = None
+        anthropic_messages: List[MessageParam] = []
+
+        for message in messages:
+            if isinstance(message, SystemMessage):
+                if system_message is not None:
+                    raise ValueError("Multiple system messages are not supported")
+                system_message = to_anthropic_type(message)
+            else:
+                anthropic_message = to_anthropic_type(message)
+                if isinstance(anthropic_message, list):
+                    anthropic_messages.extend(anthropic_message)
+                elif isinstance(anthropic_message, str):
+                    msg = MessageParam(
+                        role="user" if isinstance(message, UserMessage) else "assistant", content=anthropic_message
+                    )
+                    anthropic_messages.append(msg)
+                else:
+                    anthropic_messages.append(anthropic_message)
+
+        # Check for function calling support
+        if self.model_info["function_calling"] is False and len(tools) > 0:
+            raise ValueError("Model does not support function calling")
+
+        # Set up the request
+        request_args: Dict[str, Any] = {
+            "model": create_args["model"],
+            "messages": anthropic_messages,
+            "max_tokens": create_args.get("max_tokens", 4096),
+            "temperature": create_args.get("temperature", 1.0),
+        }
+
+        # Add system message if present
+        if system_message is not None:
+            request_args["system"] = system_message
+
+        has_tool_results = any(isinstance(msg, FunctionExecutionResultMessage) for msg in messages)
+
+        # Store and add tools if present
+        if len(tools) > 0:
+            converted_tools = convert_tools(tools)
+            self._last_used_tools = converted_tools
+            request_args["tools"] = converted_tools
+        elif has_tool_results:
+            # anthropic requires tools to be present even if there is any tool use
+            request_args["tools"] = self._last_used_tools
+
+        # Optional parameters
+        for param in ["top_p", "top_k", "stop_sequences", "metadata"]:
+            if param in create_args:
+                request_args[param] = create_args[param]
+
+        # Execute the request
+        future: asyncio.Task[Message] = asyncio.ensure_future(self._client.messages.create(**request_args))  # type: ignore
+
+        if cancellation_token is not None:
+            cancellation_token.link_future(future)  # type: ignore
+
+        result: Message = cast(Message, await future)  # type: ignore
+
+        # Extract usage statistics
+        usage = RequestUsage(
+            prompt_tokens=result.usage.input_tokens,
+            completion_tokens=result.usage.output_tokens,
+        )
+
+        # Log the event if in a handler context
+        try:
+            agent_id = MessageHandlerContext.agent_id()
+        except RuntimeError:
+            agent_id = None
+
+        logger.info(
+            LLMCallEvent(
+                messages=cast(Dict[str, Any], anthropic_messages),
+                response=result.model_dump(),
+                prompt_tokens=usage.prompt_tokens,
+                completion_tokens=usage.completion_tokens,
+                agent_id=agent_id,
+            )
+        )
+
+        # Process the response
+        content: Union[str, List[FunctionCall]]
+        thought = None
+
+        # Check if the response includes tool uses
+        tool_uses = [block for block in result.content if getattr(block, "type", None) == "tool_use"]
+
+        if tool_uses:
+            # Handle tool use response
+            content = []
+
+            # Check for text content that should be treated as thought
+            text_blocks: List[TextBlock] = [block for block in result.content if isinstance(block, TextBlock)]
+            if text_blocks:
+                thought = "".join([block.text for block in text_blocks])
+
+            # Process tool use blocks
+            for tool_use in tool_uses:
+                if isinstance(tool_use, ToolUseBlock):
+                    tool_input = tool_use.input
+                    if isinstance(tool_input, dict):
+                        tool_input = json.dumps(tool_input)
+                    else:
+                        tool_input = str(tool_input) if tool_input is not None else ""
+
+                    content.append(
+                        FunctionCall(
+                            id=tool_use.id,
+                            name=normalize_name(tool_use.name),
+                            arguments=tool_input,
+                        )
+                    )
+        else:
+            # Handle text response
+            content = "".join([block.text if isinstance(block, TextBlock) else "" for block in result.content])
+
+        # Create the final result
+        response = CreateResult(
+            finish_reason=normalize_stop_reason(result.stop_reason),
+            content=content,
+            usage=usage,
+            cached=False,
+            thought=thought,
+        )
+
+        # Update usage statistics
+        self._total_usage = _add_usage(self._total_usage, usage)
+        self._actual_usage = _add_usage(self._actual_usage, usage)
+
+        return response
+
+    async def create_stream(
+        self,
+        messages: Sequence[LLMMessage],
+        *,
+        tools: Sequence[Tool | ToolSchema] = [],
+        json_output: Optional[bool] = None,
+        extra_create_args: Mapping[str, Any] = {},
+        cancellation_token: Optional[CancellationToken] = None,
+        max_consecutive_empty_chunk_tolerance: int = 0,
+    ) -> AsyncGenerator[Union[str, CreateResult], None]:
+        """
+        Creates an AsyncGenerator that yields a stream of completions based on the provided messages and tools.
+        """
+        # Copy create args and update with extra args
+        create_args = self._create_args.copy()
+        create_args.update(extra_create_args)
+
+        # Check for vision capability if images are present
+        if self.model_info["vision"] is False:
+            for message in messages:
+                if isinstance(message, UserMessage):
+                    if isinstance(message.content, list) and any(isinstance(x, Image) for x in message.content):
+                        raise ValueError("Model does not support vision and image was provided")
+
+        # Handle JSON output format
+        if json_output is not None:
+            if self.model_info["json_output"] is False and json_output is True:
+                raise ValueError("Model does not support JSON output")
+
+            if json_output is True:
+                create_args["response_format"] = {"type": "json_object"}
+
+        # Process system message separately
+        system_message = None
+        anthropic_messages: List[MessageParam] = []
+
+        for message in messages:
+            if isinstance(message, SystemMessage):
+                if system_message is not None:
+                    raise ValueError("Multiple system messages are not supported")
+                system_message = to_anthropic_type(message)
+            else:
+                anthropic_message = to_anthropic_type(message)
+                if isinstance(anthropic_message, list):
+                    anthropic_messages.extend(anthropic_message)
+                elif isinstance(anthropic_message, str):
+                    msg = MessageParam(
+                        role="user" if isinstance(message, UserMessage) else "assistant", content=anthropic_message
+                    )
+                    anthropic_messages.append(msg)
+                else:
+                    anthropic_messages.append(anthropic_message)
+
+        # Check for function calling support
+        if self.model_info["function_calling"] is False and len(tools) > 0:
+            raise ValueError("Model does not support function calling")
+
+        # Set up the request
+        request_args: Dict[str, Any] = {
+            "model": create_args["model"],
+            "messages": anthropic_messages,
+            "max_tokens": create_args.get("max_tokens", 4096),
+            "temperature": create_args.get("temperature", 1.0),
+            "stream": True,
+        }
+
+        # Add system message if present
+        if system_message is not None:
+            request_args["system"] = system_message
+
+        # Add tools if present
+        if len(tools) > 0:
+            converted_tools = convert_tools(tools)
+            request_args["tools"] = converted_tools
+
+        # Optional parameters
+        for param in ["top_p", "top_k", "stop_sequences", "metadata"]:
+            if param in create_args:
+                request_args[param] = create_args[param]
+
+        # Stream the response
+        stream_future: asyncio.Task[AsyncStream[RawMessageStreamEvent]] = asyncio.ensure_future(
+            cast(Coroutine[Any, Any, AsyncStream[RawMessageStreamEvent]], self._client.messages.create(**request_args))
+        )
+
+        if cancellation_token is not None:
+            cancellation_token.link_future(stream_future)  # type: ignore
+
+        stream: AsyncStream[RawMessageStreamEvent] = cast(AsyncStream[RawMessageStreamEvent], await stream_future)  # type: ignore
+
+        # Prepare variables for accumulating the response
+        text_content: List[str] = []
+        tool_calls: Dict[str, Dict[str, str]] = {}
+        input_tokens: int = 0
+        output_tokens: int = 0
+        stop_reason: Optional[str] = None
+
+        # Process the stream
+        async for chunk in stream:
+            # Handle different event types
+            if chunk.type == "content_block_delta":
+                # Text content
+                if chunk.delta.type == "text_delta":
+                    delta_text = chunk.delta.text
+                    text_content.append(delta_text)
+                    if delta_text:
+                        yield delta_text
+
+            elif chunk.type == "message_delta":
+                if hasattr(chunk, "usage") and hasattr(chunk.usage, "output_tokens"):
+                    output_tokens = chunk.usage.output_tokens
+                # End of message with stop reason
+                if hasattr(chunk.delta, "stop_reason") and chunk.delta.stop_reason:
+                    stop_reason = chunk.delta.stop_reason
+
+            elif chunk.type == "message_start":
+                if hasattr(chunk, "message") and hasattr(chunk.message, "usage"):
+                    input_tokens = chunk.message.usage.input_tokens
+                    output_tokens = chunk.message.usage.output_tokens
+
+            elif chunk.type == "message_stop":
+                pass
+
+        # Prepare the final result
+        usage = RequestUsage(
+            prompt_tokens=input_tokens,
+            completion_tokens=output_tokens,
+        )
+
+        # Determine content based on what was received
+        content: Union[str, List[FunctionCall]]
+        thought = None
+
+        if tool_calls:
+            # We received tool calls
+            if text_content:
+                # Text before tool calls is treated as thought
+                thought = "".join(text_content)
+
+            # Convert tool calls to FunctionCall objects
+            content = [
+                FunctionCall(
+                    id=call_data["id"],
+                    name=normalize_name(call_data["name"]),
+                    arguments=call_data["input"],
+                )
+                for call_data in tool_calls.values()
+            ]
+        else:
+            # Just text content
+            content = "".join(text_content)
+
+        # Create final result
+        result = CreateResult(
path: python/packages/autogen-ext/src/autogen_ext/models/anthropic/_anthropic_client.py
text: @user1:
An LLMCallEvent should be logged here. 

@author:
This is done in create method (rather than create stream) when the full input and response is available.

https://github.com/microsoft/autogen/blob/c180410f863813000e29426c5889313ed1af181a/python/packages/autogen-ext/src/autogen_ext/models/anthropic/_anthropic_client.py#L528 


@user1:
I see. For a separate issue I think we need to create LLMStreamStartEvent and LLMStreamEndEvent to log streamming LLM call.