Repository: microsoft/autogen
Pull request: Adding bedrock chat completion for anthropic models

## Why are these changes needed?

Anthropic models are supported by AWS bedrock. ChatCompletionClient can be created for anthropic bedrock models using this changes. This enables the user to do the following
- Add any anthropic models and version from AWS bedrock
- Can use ChatCompletionClient for bedrock anthropic models

## Related issue number
Closes #5226 


Problem statement:
Add support for AWS Bedrock integration
### What feature would you like to be added?

- **New Module:** Create a new module for AWS Bedrock integration.
- **New Class:** Implement a new class `.autogen_ext.models.aws.BedrockChatCompletionClient` based on the `ChatCompletionClient` base class.
- **Reference:** Please refer to existing extensions such as `AzureAIChatCompletionClient` for guidance.



### Why is this needed?

**Benefits:**
- Official support for AWS Bedrock would ensure stability, ease of use, and proper maintenance.
- It would enhance the functionality of AutogenStudio for users who rely on AWS services.



**Additional Information:**
I am willing to contribute to the development of this feature. Please provide any guidance or steps on how I might be able to assist in making this feature officially supported.
In my environment, the setting for model_info is not passed from the JSON file, so I confirmed the operation by directly passing the arguments in the source code.


Thank you for your time and consideration.

Hints from issue discussion:
Thank you @Hama18, for start, you can take a look at the CONTRIBUTING.md for how to run tests locally. You can check with us when you are stuck. 

For API documentation, it would be good to have at least 1 runnable code block showing a hello world example. Would be good to also show a streaming example.

The two examples from their official doc is great:

https://docs.aws.amazon.com/code-library/latest/ug/python_3_bedrock-runtime_code_examples.html

For dependency, add `aws` for the `boto3` dependency, with a minimal verison bound set to the latest stable release verison. 

For unit test, create unit tests following those for the Azure AI client. Also create one that hits the actual endpoint but use `pytest.skip` to skip it if access was denied. See existing test against OpenAI and Gemini endpoints.
It is also possible to use Semantic Kernel client adapter for AWS Bedrock models. See: https://microsoft.github.io/autogen/stable/reference/python/autogen_ext.models.semantic_kernel.html
I apologize for any inconvenience, but I found that my requirements can be met with AWS Bedrock, which is compatible with OpenAI. 

From the discussion [below](https://github.com/microsoft/autogen/discussions/5090), I learned that it can be called from OpenAIChatCompletionClient, and I have completed what I needed to do.

 If no further extension of Bedrock is necessary, please feel free to close this issue.
> I apologize for any inconvenience, but I found that my requirements can be met with AWS Bedrock, which is compatible with OpenAI.
> 
> From the discussion [below](https://github.com/microsoft/autogen/discussions/5090), I learned that it can be called from OpenAIChatCompletionClient, and I have completed what I needed to do.
> 
> If no further extension of Bedrock is necessary, please feel free to close this issue.

Can you share your code? I still can not use bedrock model in autogen0.4
I apologize for the lack of explanation earlier. I am calling AWS Bedrock via Databricks' serving endpoint. Since the serving endpoint is OpenAI-compatible, I was able to invoke it using the `OpenAIChatCompletionClient`. It functions by assigning the Databricks access key to the `api_key` and the Databricks serving endpoint to the `base_url`.

My environment is autogen 0.4.1.


Thanks, I can also call bedrock by using both OpenAI-compatible or Semantic Kernel client 
> It is also possible to use Semantic Kernel client adapter for AWS Bedrock models. See: https://microsoft.github.io/autogen/stable/reference/python/autogen_ext.models.semantic_kernel.html


I am having some issues with using it through the agent approach. I can call the `.create()` method and it works well, but I can never get a response out of the `AssistantAgent`.

I am using `AssistantAgent` with `Swarm`. For transparency I am using agents across azure openai and bedrock. I figured this wouldn't be an issue but maybe that is problematic. If you have any thoughts on this anything would be appreciated! 


------------------------------------------

Tried solely Bedrock and still did not work.

@ekzhu 

Thanks for the tips. Would you like to help with creating a bedrock client? @Alex-Wenner-FHR 
> Thanks for the tips. Would you like to help with creating a bedrock client? [@Alex-Wenner-FHR](https://github.com/Alex-Wenner-FHR)

I would be happy to help. Are those efforts already underway? I thought I had seen some related pull requests. 

I also am just generally asking if I am misunderstanding something... I am using the Semantic Kernal wrapper around bedrock but am not getting any errors or anything, I just never get responses from the assistant agents, in swarm, using bedrock... let me know if you have any thoughts! 
Without having access to your code I can't know for sure what happened. Could you use a separate GitHub discussion to post your code?

Commit message: adding bedrock chat completion for anthropic models

Patch to review:
diff --git a/python/packages/autogen-ext/src/autogen_ext/models/anthropic/_anthropic_client.py b/python/packages/autogen-ext/src/autogen_ext/models/anthropic/_anthropic_client.py
index 9cb80e48fd8f..ff23816fb60e 100644
--- a/python/packages/autogen-ext/src/autogen_ext/models/anthropic/_anthropic_client.py
+++ b/python/packages/autogen-ext/src/autogen_ext/models/anthropic/_anthropic_client.py
@@ -25,7 +25,7 @@
 )
 
 import tiktoken
-from anthropic import AsyncAnthropic, AsyncStream
+from anthropic import AsyncAnthropic, AsyncStream, AnthropicBedrock
 from anthropic.types import (
     Base64ImageSourceParam,
     ContentBlock,
@@ -1073,3 +1073,118 @@ def _from_config(cls, config: AnthropicClientConfigurationConfigModel) -> Self:
             copied_config["api_key"] = config.api_key.get_secret_value()
 
         return cls(**copied_config)
+
+class AnthropicBedrockChatCompletionClient(
+    BaseAnthropicChatCompletionClient, Component[AnthropicClientConfigurationConfigModel]
+):
+    """
+    Chat completion client for Anthropic's Claude models.
+
+    Args:
+        model (str): The Claude model to use (e.g., "claude-3-sonnet-20240229", "claude-3-opus-20240229")
+        api_key (str, optional): Anthropic API key. Required if not in environment variables.
+        base_url (str, optional): Override the default API endpoint.
+        max_tokens (int, optional): Maximum tokens in the response. Default is 4096.
+        temperature (float, optional): Controls randomness. Lower is more deterministic. Default is 1.0.
+        top_p (float, optional): Controls diversity via nucleus sampling. Default is 1.0.
+        top_k (int, optional): Controls diversity via top-k sampling. Default is -1 (disabled).
+        model_info (ModelInfo, optional): The capabilities of the model. Required if using a custom model.
+
+    To use this client, you must install the Anthropic extension:
+
+    .. code-block:: bash
+
+        pip install "autogen-ext[anthropic]"
+
+    Example:
+
+    .. code-block:: python
+
+        import asyncio
+        from autogen_ext.models.anthropic import AnthropicBedrockChatCompletionClient
+        from autogen_core.models import UserMessage
+
+
+        async def main():
+            config =  {
+            "model": "anthropic.claude-3-5-sonnet-20240620-v1:0",
+            "temperature": 0.1,
+            "model_info": {
+                "vision": True,
+                "function_calling": True,
+                "json_output": True,
+                "family": ModelFamily.CLAUDE_3_5_SONNET,
+            },
+            "bedrock_info":{
+                "aws_access_key": "<aws_access_key>",
+                "aws_secret_key": "<aws_secret_key>",
+                "aws_session_token": "<aws_session_token>",
+                "aws_region": "<aws_region>"
+            }}
+            anthropic_client = AnthropicBedrockChatCompletionClient(**config)
+
+            result = await anthropic_client.create([UserMessage(content="What is the capital of France?", source="user")])  # type: ignore
+            print(result)
+
+        if __name__ == "__main__":
+            asyncio.run(main())
+    """
+
+    component_type = "model"
+    component_config_schema = AnthropicClientConfigurationConfigModel
+    component_provider_override = "autogen_ext.models.anthropic.AnthropicChatCompletionClient"
+
+    def __init__(self, **kwargs: Unpack[AnthropicClientConfiguration]):
+        if "model" not in kwargs:
+            raise ValueError("model is required for AnthropicChatCompletionClient")
+
+        self._raw_config: Dict[str, Any] = dict(kwargs).copy()
+        copied_args = dict(kwargs).copy()
+
+        model_info: Optional[ModelInfo] = None
+        if "model_info" in kwargs:
+            model_info = kwargs["model_info"]
+            del copied_args["model_info"]
+
+        if "bedrock_info" not in kwargs:
+            raise ValueError("bedrock_info is required for AnthropicBedrockChatCompletionClient")
+        # Authenticate by either providing the keys below or use the default AWS credential providers, such as
+        # using ~/.aws/credentials or the "AWS_SECRET_ACCESS_KEY" and "AWS_ACCESS_KEY_ID" environment variables.
+        aws_region = kwargs["bedrock_info"]["aws_region"]
+        aws_access_key=kwargs["bedrock_info"]["aws_access_key"]
+        aws_secret_key=kwargs["bedrock_info"]["aws_secret_key"]
+        aws_session_token=kwargs["bedrock_info"]["aws_session_token"]
+        client = AnthropicBedrock(aws_access_key=aws_access_key,
+                                  aws_secret_key=aws_secret_key,
+                                  aws_session_token=aws_session_token,
+                                  aws_region=aws_region)
+        create_args = _create_args_from_config(copied_args)
+
+        super().__init__(
+            client=client,
+            create_args=create_args,
+            model_info=model_info,
+        )
+
+    def __getstate__(self) -> Dict[str, Any]:
+        state = self.__dict__.copy()
+        state["_client"] = None
+        return state
+
+    def __setstate__(self, state: Dict[str, Any]) -> None:
+        self.__dict__.update(state)
+        self._client = _anthropic_client_from_config(state["_raw_config"])
+
+    def _to_config(self) -> AnthropicClientConfigurationConfigModel:
+        copied_config = self._raw_config.copy()
+        return AnthropicClientConfigurationConfigModel(**copied_config)
+
+    @classmethod
+    def _from_config(cls, config: AnthropicClientConfigurationConfigModel) -> Self:
+        copied_config = config.model_copy().model_dump(exclude_none=True)
+
+        # Handle api_key as SecretStr
+        if "api_key" in copied_config and isinstance(config.api_key, SecretStr):
+            copied_config["api_key"] = config.api_key.get_secret_value()
+
+        return cls(**copied_config)
\ No newline at end of file


Code review comments:
diff_hunk: @@ -1073,3 +1073,118 @@ def _from_config(cls, config: AnthropicClientConfigurationConfigModel) -> Self:
             copied_config["api_key"] = config.api_key.get_secret_value()
 
         return cls(**copied_config)
+
+class AnthropicBedrockChatCompletionClient(
+    BaseAnthropicChatCompletionClient, Component[AnthropicClientConfigurationConfigModel]
path: python/packages/autogen-ext/src/autogen_ext/models/anthropic/_anthropic_client.py
text: @user1:
Thanks, can you create a separate `BedrockAnthropicClientConfigurationConfigModel` for the additional parameters for the bedrock connection?

@user2:
It may also be a good idea that this `BedrockAnthropicClientConfigurationConfigModel` accepts `aws_profile` to pass to Anthropic’s [client SDKs](https://docs.anthropic.com/en/api/client-sdks) [constructor](https://github.com/anthropics/anthropic-sdk-python/blob/8b244157a7d03766bec645b0e1dc213c6d462165/src/anthropic/lib/bedrock/_client.py#L115) in `client = AnthropicBedrock(...)`
This would allow the option of using AWS CLI's specific configuration profiles to access AWS Bedrock service without storing any cleartext credentials elsewhere.
Also, perhaps certain properties of the ConfigModel should be SecretStr types, as with openai models' `api_key`