Repository: microsoft/autogen
Pull request: Fix R1 reasoning parser for openai client
<!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. -->
Hey @ekzhu, can you please have  a look at this. This resolves the issues with parsing of the R1 reasoning tokens at least for openai client there are still areas where it still uses the old parser which need to be look at in the future. This doesn't fully resolve all the issues mentioned previously however the parsing of reasoning tokens should now be resolved. 
<!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. -->

## Why are these changes needed?
R1 reasoning tokens were not parsed correctly for the openai client
<!-- Please give a short summary of the change and the problem this solves. -->

## Related issue number
#5941

<!-- For example: "Closes #1234" -->

## Checks

- [ ] I've included any doc changes needed for <https://microsoft.github.io/autogen/>. See <https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to build and test documentation locally.
- [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR.
- [ ] I've made sure all auto checks have passed.


Problem statement:
ERROR when using deepseek r1
### What happened?

**Describe the bug**

-----
{"error":{"message":"deepseek-reasoner does not support successive user or assistant messages (messages[1] and messages[2] in your input). You should interleave the user/assistant messages in the message sequence.","type":"invalid_request_error","param":null,"code":"invalid_request_error"}}

-----

Prompt "deepseek-reasoner does not support continuous user or assistant messages". The reason for this problem is that the roles (user and assistant) in the message sequence must strictly alternate, and messages with the same role cannot appear continuously.

**To Reproduce**

```python
analyst_model_client = OpenAIChatCompletionClient(model="deepseek-reasoner",
                                                  temperature=1.0,
                                                  api_key=analyst_model["api_key"],
                                                  base_url=analyst_model["base_url"],
                                                  max_tokens=analyst_model["max_tokens"],
                                                  model_capabilities={
                                                      "vision": False,
                                                      "function_calling": False,
                                                      "json_output": False,
                                                      "family": ModelFamily.R1,
                                                  })

agents = get_basic_financials_agent(planning_model_client, analyst_model_client)

research_team = RoundRobinGroupChat(agents, max_turns=2)
```
**Expected behavior**
DeepSeek V3 is ok.

**Screenshots**
NA

**Additional context**
NA

### Which packages was the bug in?

Python Core (autogen-core), Python AgentChat (autogen-agentchat>=0.4.0)

### AutoGen library version.

Python 0.4.7

### Other library version.

0.4.8.2

### Model used

DeepSeek R1

### Model provider

None

### Other model provider

DeepSeek

### Python version

3.12

### .NET version

None

### Operating system

Ubuntu

Hints from issue discussion:
Can you post a reproducible code example for this? Not a code snippet. An example that can run from command line. 

I do not have access to DeepSeek's hosted model -- I have access to locally hosted DeepSeek models. So I cannot test this for you. Can you reproduce this using a locally hosted DeepSeek model e.g., using Ollama? 
Same issue here
```
Traceback (most recent call last):
  File "C:\.......\Python313\Lib\site-packages\autogen_core\_single_threaded_agent_runtime.py", line 510, in _on_message   
    return await agent.on_message(
           ^^^^^^^^^^^^^^^^^^^^^^^
    ...<2 lines>...
    )
    ^
  File "C:\.......\Python313\Lib\site-packages\autogen_core\_base_agent.py", line 113, in on_message
    return await self.on_message_impl(message, ctx)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\.......\Python313\Lib\site-packages\autogen_agentchat\teams\_group_chat\_sequential_routed_agent.py", line 67, in on_message_impl
    return await super().on_message_impl(message, ctx)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\.......\Python313\Lib\site-packages\autogen_core\_routed_agent.py", line 485, in on_message_impl
    return await h(self, message, ctx)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\.......\Python313\Lib\site-packages\autogen_core\_routed_agent.py", line 268, in wrapper
    return_value = await func(self, message, ctx)  # type: ignore
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\.......\Python313\Lib\site-packages\autogen_agentchat\teams\_group_chat\_chat_agent_container.py", line 69, in handle_request
    async for msg in self._agent.on_messages_stream(self._message_buffer, ctx.cancellation_token):
    ...<11 lines>...
            )
  File "C:\.......\Python313\Lib\site-packages\autogen_agentchat\agents\_assistant_agent.py", line 748, in on_messages_stream
    async for inference_output in self._call_llm(
    ...<13 lines>...
            yield inference_output
  File "C:\.......\Python313\Lib\site-packages\autogen_agentchat\agents\_assistant_agent.py", line 870, in _call_llm       
    model_result = await model_client.create(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^
        llm_messages, tools=all_tools, cancellation_token=cancellation_token
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "C:\.......\Python313\Lib\site-packages\autogen_ext\models\openai\_openai_client.py", line 523, in create
    result: Union[ParsedChatCompletion[BaseModel], ChatCompletion] = await future
                                                                     ^^^^^^^^^^^^
  File "C:\.......\Python313\Lib\site-packages\openai\resources\chat\completions\completions.py", line 2000, in create     
    return await self._post(
           ^^^^^^^^^^^^^^^^^
    ...<43 lines>...
    )
    ^
  File "C:\.......\Python313\Lib\site-packages\openai\_base_client.py", line 1767, in post
    return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\.......\Python313\Lib\site-packages\openai\_base_client.py", line 1461, in request
    return await self._request(
           ^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "C:\.......\Python313\Lib\site-packages\openai\_base_client.py", line 1562, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.BadRequestError: Error code: 400 - {'error': {'message': 'deepseek-reasoner does not support successive user or assistant messages (messages[1] and messages[2] in your input). You should interleave the user/assistant messages in the message sequence.', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_request_error'}}


```



I also get the following error but this one comes in before the one above
```
 UserWarning: Could not find <think>..</think> field in model response content. No thought was extracted.
  thought, content = parse_r1_content(content)
```

Here is a sample response from the deepseek api 
`
{"id":"d35.....190","object":"chat.completion","created":1742010086,"model":"deepseek-reasoner","choices":[{"index":0,"message":{"role":"assistant","content":"Hello! How can I assist you today? 😊","reasoning_content":"Okay, the user started with \"Hi\". I should respond in a friendly and approachable way. Let me say hello and ask how I can help them today. Keeping it simple and open-ended should encourage them to share what they need assistance with."},"logprobs":null,"finish_reason":"stop"}],"usage":{"prompt_tokens":12,"completion_tokens":63,"total_tokens":75,"prompt_tokens_details":{"cached_tokens":0},"completion_tokens_details":{"reasoning_tokens":50},"prompt_cache_hit_tokens":0,"prompt_cache_miss_tokens":12},"system_fingerprint":"fp_.....25"}`

Which doesnt look like it matches how you are looking for the thinking tokens i.e. ` <think> .. </think>` as this has it as `"reasoning_content":"`
Fix is being worked on here: https://github.com/microsoft/autogen/pull/5941

Please test and give feedback

Commit message: fixex R1 reasoning parser for openai client

Patch to review:
diff --git a/python/packages/autogen-ext/src/autogen_ext/models/openai/_openai_client.py b/python/packages/autogen-ext/src/autogen_ext/models/openai/_openai_client.py
index 2301655b32d1..b02a32dd7aae 100644
--- a/python/packages/autogen-ext/src/autogen_ext/models/openai/_openai_client.py
+++ b/python/packages/autogen-ext/src/autogen_ext/models/openai/_openai_client.py
@@ -617,6 +617,8 @@ async def create(
         else:
             finish_reason = choice.finish_reason
             content = choice.message.content or ""
+            if choice.message.model_extra['reasoning_content'] is not None:
+                thought = choice.message.model_extra['reasoning_content']
 
         logprobs: Optional[List[ChatCompletionTokenLogprob]] = None
         if choice.logprobs and choice.logprobs.content:
@@ -630,8 +632,7 @@ async def create(
                 for x in choice.logprobs.content
             ]
 
-        if isinstance(content, str) and self._model_info["family"] == ModelFamily.R1:
-            thought, content = parse_r1_content(content)
+      
 
         response = CreateResult(
             finish_reason=normalize_stop_reason(finish_reason),
@@ -725,6 +726,7 @@ async def create_stream(
         stop_reason = None
         maybe_model = None
         content_deltas: List[str] = []
+        thought_deltas: List[str] = []
         full_tool_calls: Dict[int, FunctionCall] = {}
         completion_tokens = 0
         logprobs: Optional[List[ChatCompletionTokenLogprob]] = None
@@ -784,7 +786,10 @@ async def create_stream(
                 # NOTE: for OpenAI, tool_calls and content are mutually exclusive it seems, so we can skip the rest of the loop.
                 # However, this may not be the case for other APIs -- we should expect this may need to be updated.
                 continue
-
+            #thought: str | None = None
+            if choice.delta.model_extra['reasoning_content'] is not None:
+                thought_deltas.append(choice.delta.model_extra['reasoning_content'])              
+                yield choice.delta.model_extra['reasoning_content']
             # Otherwise, get tool calls
             if choice.delta.tool_calls is not None:
                 for tool_call_chunk in choice.delta.tool_calls:
@@ -837,22 +842,25 @@ async def create_stream(
         # Detect whether it is a function call or just text.
         content: Union[str, List[FunctionCall]]
         thought: str | None = None
+        # Determine the content and thought based on what was collected
         if full_tool_calls:
-            # This is a tool call.
+            # This is a tool call response
             content = list(full_tool_calls.values())
-            if len(content_deltas) > 1:
-                # Put additional text content in the thought field.
+            if content_deltas:
+                # Store any text alongside tool calls as thoughts
                 thought = "".join(content_deltas)
-        elif len(content_deltas) > 0:
-            # This is a text-only content.
-            content = "".join(content_deltas)
         else:
-            warnings.warn("No text content or tool calls are available. Model returned empty result.", stacklevel=2)
-            content = ""
+            # This is a text response (possibly with thoughts)
+            if content_deltas:
+                content = "".join(content_deltas)
+            else:
+                warnings.warn("No text content or tool calls are available. Model returned empty result.", stacklevel=2)
+                content = ""
+                
+        # Always set thoughts if we have any, regardless of other content types
+        if thought_deltas:
+            thought = "".join(thought_deltas)
 
-        # Parse R1 content if needed.
-        if isinstance(content, str) and self._model_info["family"] == ModelFamily.R1:
-            thought, content = parse_r1_content(content)
 
         # Create the result.
         result = CreateResult(


Code review comments:
diff_hunk: @@ -617,6 +617,8 @@ async def create(
         else:
             finish_reason = choice.finish_reason
             content = choice.message.content or ""
+            if choice.message.model_extra['reasoning_content'] is not None:
path: python/packages/autogen-ext/src/autogen_ext/models/openai/_openai_client.py
text: @author:
Hey @user1 my understanding is that model_extra is a property from the BaseModel of Pydantic (starting from v2.0.1 I think). It is a dictionary so if reasoning_content is not populated it might throw a spanner so I will change it to a .get but I think for the model_extra we should be good to keep as is. Let me know what you think and I can add the change in.

@user1:
Got it. `model_extra` is Pydantic field so it's fine to use it directly.