Repository: microsoft/autogen
Pull request: raise error for content_filter
<!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. -->

## Why are these changes needed?

Raise content_filter error to match the behavior in v0.1, following #757.

## Related issue number

Close #279

## Checks

- [ ] I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally.
- [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR.
- [ ] I've made sure all auto checks have passed.


Problem statement:
AttributeError: 'str' object has no attribute 'get' after the task is finished
Hi! First, thank you to all the devs who made autogen, it is so cool!

Unfortunately I have a persistent error (AttributeError: 'str' object has no attribute 'get')

It pops up sometimes before the Assistant is finished (when it writes python code for example), but when the task is primitive ( "come up with 5 possible names for SMMA that helps companies get leads") it manages to get it done and then spits out this error.

`user_proxy (to assistant):


come up with 5 possible names for SMMA that helps companies get leads


--------------------------------------------------------------------------------
assistant (to user_proxy):

1. Lead Generation Machine
2. Lead Generator
3. Sales Leads Machine
4. Business Leads Generator
5. Lead Finder

--------------------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\ihasdslr\Documents\auto_gen\autogen_init.py", line 36, in <module>
    user_proxy.initiate_chat(
  File "C:\Users\ihasdslr\anaconda3\envs\auto_gen\Lib\site-packages\autogen\agentchat\conversable_agent.py", line 531, in initiate_chat
    self.send(self.generate_init_message(**context), recipient, silent=silent)
  File "C:\Users\ihasdslr\anaconda3\envs\auto_gen\Lib\site-packages\autogen\agentchat\conversable_agent.py", line 334, in send
    recipient.receive(message, self, request_reply, silent)
  File "C:\Users\ihasdslr\anaconda3\envs\auto_gen\Lib\site-packages\autogen\agentchat\conversable_agent.py", line 464, in receive      
    self.send(reply, sender, silent=silent)
  File "C:\Users\ihasdslr\anaconda3\envs\auto_gen\Lib\site-packages\autogen\agentchat\conversable_agent.py", line 334, in send
    recipient.receive(message, self, request_reply, silent)
  File "C:\Users\ihasdslr\anaconda3\envs\auto_gen\Lib\site-packages\autogen\agentchat\conversable_agent.py", line 462, in receive
    reply = self.generate_reply(messages=self.chat_messages[sender], sender=sender)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ihasdslr\anaconda3\envs\auto_gen\Lib\site-packages\autogen\agentchat\conversable_agent.py", line 781, in generate_reply
    final, reply = reply_func(self, messages=messages, sender=sender, config=reply_func_tuple["config"])
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ihasdslr\anaconda3\envs\auto_gen\Lib\site-packages\autogen\agentchat\conversable_agent.py", line 606, in generate_oai_reply
    response = oai.ChatCompletion.create(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ihasdslr\anaconda3\envs\auto_gen\Lib\site-packages\autogen\oai\completion.py", line 799, in create
    response = cls.create(
               ^^^^^^^^^^^
  File "C:\Users\ihasdslr\anaconda3\envs\auto_gen\Lib\site-packages\autogen\oai\completion.py", line 830, in create
    return cls._get_response(params, raise_on_ratelimit_or_timeout=raise_on_ratelimit_or_timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ihasdslr\anaconda3\envs\auto_gen\Lib\site-packages\autogen\oai\completion.py", line 218, in _get_response
    response = openai_completion.create(**config)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ihasdslr\anaconda3\envs\auto_gen\Lib\site-packages\openai\api_resources\chat_completion.py", line 25, in create
    return super().create(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ihasdslr\anaconda3\envs\auto_gen\Lib\site-packages\openai\api_resources\abstract\engine_api_resource.py", line 155, in create
    response, _, api_key = requestor.request(
                           ^^^^^^^^^^^^^^^^^^
  File "C:\Users\ihasdslr\anaconda3\envs\auto_gen\Lib\site-packages\openai\api_requestor.py", line 299, in request
    resp, got_stream = self._interpret_response(result, stream)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ihasdslr\anaconda3\envs\auto_gen\Lib\site-packages\openai\api_requestor.py", line 710, in _interpret_response
    self._interpret_response_line(
  File "C:\Users\ihasdslr\anaconda3\envs\auto_gen\Lib\site-packages\openai\api_requestor.py", line 775, in _interpret_response_line
    raise self.handle_error_response(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ihasdslr\anaconda3\envs\auto_gen\Lib\site-packages\openai\api_requestor.py", line 428, in handle_error_response
    error_code=error_data.get("code"),
               ^^^^^^^^^^^^^^
AttributeError: 'str' object has no attribute 'get'`

I run the server with local LLMs via the LM studio. During this error my server logs say 
`[ERROR] Error: 'messages' array must only contain objects with a 'content' field that is not empty.`

pls help!

Hints from issue discussion:
Ihave the similar issue
Try setting `default_reply` of `UserProxyAgent` to a non-empty str.
> Try setting `default_reply` of `UserProxyAgent` to a non-empty str.

Unfortunately still the same error:

```
Traceback (most recent call last):
  File "c:\Users\ihasdslr\Documents\auto_gen\autogen_init.py", line 37, in <module>
    user_proxy.initiate_chat(
  File "C:\Users\ihasdslr\anaconda3\envs\auto_gen\Lib\site-packages\autogen\agentchat\conversable_agent.py", line 531, in initiate_chat
    self.send(self.generate_init_message(**context), recipient, silent=silent)
  File "C:\Users\ihasdslr\anaconda3\envs\auto_gen\Lib\site-packages\autogen\agentchat\conversable_agent.py", line 334, in send
    recipient.receive(message, self, request_reply, silent)
  File "C:\Users\ihasdslr\anaconda3\envs\auto_gen\Lib\site-packages\autogen\agentchat\conversable_agent.py", line 464, in receive
    self.send(reply, sender, silent=silent)
  File "C:\Users\ihasdslr\anaconda3\envs\auto_gen\Lib\site-packages\autogen\agentchat\conversable_agent.py", line 334, in send
    recipient.receive(message, self, request_reply, silent)
  File "C:\Users\ihasdslr\anaconda3\envs\auto_gen\Lib\site-packages\autogen\agentchat\conversable_agent.py", line 462, in receive
    reply = self.generate_reply(messages=self.chat_messages[sender], sender=sender)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ihasdslr\anaconda3\envs\auto_gen\Lib\site-packages\autogen\agentchat\conversable_agent.py", line 781, in generate_reply
    final, reply = reply_func(self, messages=messages, sender=sender, config=reply_func_tuple["config"])
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ihasdslr\anaconda3\envs\auto_gen\Lib\site-packages\autogen\agentchat\conversable_agent.py", line 606, in generate_oai_reply
    response = oai.ChatCompletion.create(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ihasdslr\anaconda3\envs\auto_gen\Lib\site-packages\autogen\oai\completion.py", line 799, in create
    response = cls.create(
               ^^^^^^^^^^^
  File "C:\Users\ihasdslr\anaconda3\envs\auto_gen\Lib\site-packages\autogen\oai\completion.py", line 830, in create
    return cls._get_response(params, raise_on_ratelimit_or_timeout=raise_on_ratelimit_or_timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ihasdslr\anaconda3\envs\auto_gen\Lib\site-packages\autogen\oai\completion.py", line 218, in _get_response
    response = openai_completion.create(**config)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ihasdslr\anaconda3\envs\auto_gen\Lib\site-packages\openai\api_resources\chat_completion.py", line 25, in create
    return super().create(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ihasdslr\anaconda3\envs\auto_gen\Lib\site-packages\openai\api_resources\abstract\engine_api_resource.py", line 155, in create
    response, _, api_key = requestor.request(
                           ^^^^^^^^^^^^^^^^^^
  File "C:\Users\ihasdslr\anaconda3\envs\auto_gen\Lib\site-packages\openai\api_requestor.py", line 299, in request
    resp, got_stream = self._interpret_response(result, stream)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ihasdslr\anaconda3\envs\auto_gen\Lib\site-packages\openai\api_requestor.py", line 710, in _interpret_response
    self._interpret_response_line(
  File "C:\Users\ihasdslr\anaconda3\envs\auto_gen\Lib\site-packages\openai\api_requestor.py", line 775, in _interpret_response_line
    raise self.handle_error_response(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ihasdslr\anaconda3\envs\auto_gen\Lib\site-packages\openai\api_requestor.py", line 428, in handle_error_response
    error_code=error_data.get("code"),
               ^^^^^^^^^^^^^^
AttributeError: 'str' object has no attribute 'get'
```

My current UserProxy definition now looks like this:

```
user_proxy = autogen.UserProxyAgent(
    name="user_proxy",
    human_input_mode="NEVER",
    default_auto_reply="default_auto_reply",
    max_consecutive_auto_reply=10,
    code_execution_config={"work_dir": "result"},
    llm_config=llm_config,
)
```


Do you intend for the user proxy agent to use llm-based reply? If not, please remove `llm_config` from its constructor.
> Do you intend for the user proxy agent to use llm-based reply? If not, please remove `llm_config` from its constructor.

Thanks so much for the help! It worked! Both assistants are now in an infinite loop sending the same code to each other :D But the error is gone. Thank you!
The user proxy agent is not supposed to send code. It will use `default_auto_reply` when no code is in the received msg.
> Ihave the similar issue

Same.

From the API I get the error:

`[ERROR] Error: 'messages' array must only contain objects with a 'content' field that is not empty.`  

And I can see that the repsonse contains multiple content fields which are empty:

`,
    {
      "content": "```",
      "role": "user",
      "name": "Planner"
    },
    {
      "content": "",
      "role": "user",
      "name": "Executor"
    },
    {
      "role": "system",
      "content": "Read the above conversation. Then select the next role from ['Admin', 'Engineer', 'Planner', 'Executor', 'Critic'] to play. Only return the role."
    }`
I'm getting this error trying to run the official example for visualizations with no substantial code changes. 
[example](https://github.com/microsoft/autogen/blob/main/notebook/agentchat_groupchat_vis.ipynb)

The only difference is that I'm using an LLM running on the local network rather than OAI.
`config_list_gpt4 = [
    {
        "api_type": "open_ai",
        "api_base": "http://###.###.##.##:1234/v1",
        "api_key": "NULL"
    }
]`

The LLM is receiving requests and returning results. This error seems to hit after the first cycle through the group.
Same reason as I said above. The local LLM doesn't allow empty content. The workaround is to set `default_auto_reply` to a non empty str.
can you write example? im new at this lol

Commit message: use the code field

Patch to review:
diff --git a/autogen/oai/client.py b/autogen/oai/client.py
index b4a139401ad2..e6ba8c26b724 100644
--- a/autogen/oai/client.py
+++ b/autogen/oai/client.py
@@ -245,7 +245,11 @@ def yes_or_no_filter(context, response):
                         continue  # filter is not passed; try the next config
             try:
                 response = self._completions_create(client, params)
-            except APIError:
+            except APIError as err:
+                error_code = err and getattr(err, "code", None)
+                if error_code == "content_filter":
+                    # raise the error for content_filter
+                    raise
                 logger.debug(f"config {i} failed", exc_info=1)
                 if i == last:
                     raise


Code review comments:
diff_hunk: @@ -245,7 +245,11 @@ def yes_or_no_filter(context, response):
                         continue  # filter is not passed; try the next config
             try:
                 response = self._completions_create(client, params)
-            except APIError:
+            except APIError as err:
+                error_code = err and getattr(err, "code", None)
path: autogen/oai/client.py
text: @user1:
do we need to do `err and`? if it has already caught the exception based on its type, it should not be None

```suggestion
                error_code = getattr(err, "code", None)
```