Repository: BerriAI/litellm
Pull request: fix: support azure o3 model family for fake streaming workaround
## Support o3 model family for fake streaming workaround from Azure
`o3-mini` is already available on Azure, but sadly it looks like it doesn't support streaming, just like the `o1` models.

So this PR updates the logic for the fake streaming workaround to also support the `o3` family. Note that there's a wider refactor to generalize the name of `AzureOpenAIO1Config` that I'm not making here.

## Relevant issues
Fixes https://github.com/BerriAI/litellm/issues/8158
<!-- e.g. "Fixes #000" -->

## Type
🐛 Bug Fix

## Changes
* Added `o3*` to the list of models for the Azure fake streaming workaround
<!-- List of changes -->

## [REQUIRED] Testing - Attach a screenshot of any new tests passing locally
I didn't see any obvious tests in this area, so manually tested the fake streaming is now working with `o3-mini`, `o1-mini`, as well as non-streaming on `o3-mini`. Also validated that real streaming is still working with `gpt-4o`.



Problem statement:
[Bug]: o3-mini - Azure OpenAI
### What happened?

Surprisingly, Azure OpenAI this time added support for the o3-mini model on the same day! 

But when a request is sent with `stream` set to `true`, the ability to fake the stream that worked with o1 models (behaving as if stream were false but returning full data by SSE in single message) doesn't work here. This won't work even with `enable_preview_features` set to `true`, and was likely hardcoded only for o1 models.

To reproduce this, simply sending a request with stream set to true will be sufficient.

### Request:

```json
{
  "model": "o3-mini",
  "messages": [
    {
      "role": "user",
      "content": "Who are you?"
    }
  ],
  "frequency_penalty": 0,
  "presence_penalty": 0,
  "temperature": 1,
  "top_p": 1,
  "stream": true
}
```

### Response:

```json
{
  "error": {
    "message": "litellm.BadRequestError: AzureException BadRequestError - Error code: 400 - {'error': {'message': \"Unsupported value: 'stream' does not support true with this model. Supported values are: false.\", 'type': 'invalid_request_error', 'param': 'stream', 'code': 'unsupported_value', 'callstack': '  File \...]}] LiteLLM Retried: 4 times, LiteLLM Max Retries: 5",
    "type": null,
    "param": null,
    "code": "400"
  }
}
```

### Relevant log output

```shell

```

### Are you a ML Ops Team?

No

### What LiteLLM version are you on ?

1.59.9

### Twitter / LinkedIn details

x.com/yigitkonur

Hints from issue discussion:

Commit message: fix: support azure o3 model family for fake streaming workaround

Patch to review:
diff --git a/litellm/litellm_core_utils/get_supported_openai_params.py b/litellm/litellm_core_utils/get_supported_openai_params.py
index e251784f4e18..87f7c037110d 100644
--- a/litellm/litellm_core_utils/get_supported_openai_params.py
+++ b/litellm/litellm_core_utils/get_supported_openai_params.py
@@ -81,7 +81,7 @@ def get_supported_openai_params(  # noqa: PLR0915
     elif custom_llm_provider == "openai":
         return litellm.OpenAIConfig().get_supported_openai_params(model=model)
     elif custom_llm_provider == "azure":
-        if litellm.AzureOpenAIO1Config().is_o1_model(model=model):
+        if litellm.AzureOpenAIO1Config().is_reasoning_model(model=model):
             return litellm.AzureOpenAIO1Config().get_supported_openai_params(
                 model=model
             )
diff --git a/litellm/llms/azure/chat/o1_handler.py b/litellm/llms/azure/chat/o1_handler.py
index 1cb6f888c3f7..b9d316de3bde 100644
--- a/litellm/llms/azure/chat/o1_handler.py
+++ b/litellm/llms/azure/chat/o1_handler.py
@@ -1,7 +1,7 @@
 """
-Handler file for calls to Azure OpenAI's o1 family of models
+Handler file for calls to Azure OpenAI's o1/o3 family of models
 
-Written separately to handle faking streaming for o1 models.
+Written separately to handle faking streaming for o1 and o3 models.
 """
 
 from typing import Optional, Union
diff --git a/litellm/llms/azure/chat/o1_transformation.py b/litellm/llms/azure/chat/o1_transformation.py
index 0b56aa1fb4d8..52e5e2f474db 100644
--- a/litellm/llms/azure/chat/o1_transformation.py
+++ b/litellm/llms/azure/chat/o1_transformation.py
@@ -1,5 +1,5 @@
 """
-Support for o1 model family 
+Support for o1 and o3 model families
 
 https://platform.openai.com/docs/guides/reasoning
 
@@ -47,5 +47,5 @@ def should_fake_stream(
 
         return True
 
-    def is_o1_model(self, model: str) -> bool:
-        return "o1" in model
+    def is_reasoning_model(self, model: str) -> bool:
+        return "o1" in model or "o3" in model
diff --git a/litellm/main.py b/litellm/main.py
index 503aeccee24b..7d6d0f6fb3cd 100644
--- a/litellm/main.py
+++ b/litellm/main.py
@@ -1201,7 +1201,7 @@ def completion(  # type: ignore # noqa: PLR0915
             if extra_headers is not None:
                 optional_params["extra_headers"] = extra_headers
 
-            if litellm.AzureOpenAIO1Config().is_o1_model(model=model):
+            if litellm.AzureOpenAIO1Config().is_reasoning_model(model=model):
                 ## LOAD CONFIG - if set
                 config = litellm.AzureOpenAIO1Config.get_config()
                 for k, v in config.items():
diff --git a/litellm/utils.py b/litellm/utils.py
index 5396e008f0d8..da1ca48ecefa 100644
--- a/litellm/utils.py
+++ b/litellm/utils.py
@@ -3489,7 +3489,7 @@ def _check_valid_arg(supported_params: List[str]):
             ),
         )
     elif custom_llm_provider == "azure":
-        if litellm.AzureOpenAIO1Config().is_o1_model(model=model):
+        if litellm.AzureOpenAIO1Config().is_reasoning_model(model=model):
             optional_params = litellm.AzureOpenAIO1Config().map_openai_params(
                 non_default_params=non_default_params,
                 optional_params=optional_params,
@@ -5997,7 +5997,7 @@ def get_provider_chat_config(  # noqa: PLR0915
         ):
             return litellm.AI21ChatConfig()
         elif litellm.LlmProviders.AZURE == provider:
-            if litellm.AzureOpenAIO1Config().is_o1_model(model=model):
+            if litellm.AzureOpenAIO1Config().is_reasoning_model(model=model):
                 return litellm.AzureOpenAIO1Config()
             return litellm.AzureOpenAIConfig()
         elif litellm.LlmProviders.AZURE_AI == provider:


Code review comments:
diff_hunk: @@ -5997,7 +5997,7 @@ def get_provider_chat_config(  # noqa: PLR0915
         ):
             return litellm.AI21ChatConfig()
         elif litellm.LlmProviders.AZURE == provider:
-            if litellm.AzureOpenAIO1Config().is_o1_model(model=model):
+            if litellm.AzureOpenAIO1Config().is_reasoning_model(model=model):
path: litellm/utils.py
text: @user1:
given the refactor will probably be to the `o-series` can we rename the helper util something like - `is_o_series_model` ? 

it would just be clearer what it's really trying to do