Repository: Lightning-AI/LitServe

Issue:
Add support for `reasoning_effort` in OpenAI spec for chat request
<!--

⚠️ BEFORE SUBMITTING, READ:

We're excited for your request! However, here are things we are not interested in:

- Decorators.
- Doing the same thing in multiple ways.    
- Adding more layers of abstraction... tree-depth should be 1 at most.
- Features that over-engineer or complicate the code internals.    
- Linters, and crud that complicates projects.    

-->

----

## 🚀 Feature

<!-- A clear and concise description of the feature proposal -->

**Add support for `reasoning_effort` in OpenAI spec for chat request**

`reasoning_effort` is a useful field for o-series reasoning models to control the level of reasoning (low, medium, high).
It helps optimize for latency or token usage depending on the use case.

Ref: https://platform.openai.com/docs/api-reference/chat/create#chat-create-reasoning_effort

### Motivation

<!--
Please outline the motivation for the proposal.
Is your feature request related to a problem? e.g., I'm always frustrated when [...].
If this is related to another GitHub issue, please link here too...
-->

### Pitch

<!-- A clear and concise description of what you want to happen. -->

### Alternatives

<!-- A clear and concise description of any alternative solutions or features you've considered, if any. -->

### Additional context

<!-- Add any other context or screenshots about the feature request here. -->



Discussion:

Patch:
diff --git a/src/litserve/specs/openai.py b/src/litserve/specs/openai.py
index 166d8d7a..4b3f9897 100644
--- a/src/litserve/specs/openai.py
+++ b/src/litserve/specs/openai.py
@@ -176,6 +176,7 @@ class ChatCompletionRequest(BaseModel):
     tools: Optional[List[Tool]] = None
     tool_choice: Optional[ToolChoice] = ToolChoice.auto
     response_format: Optional[ResponseFormat] = None
+    reasoning_effort: Optional[Literal["low", "medium", "high"]] = None
     metadata: Optional[Dict[str, str]] = None
 
 


Test patch:
diff --git a/tests/test_specs.py b/tests/test_specs.py
index b10b62c5..4fa85627 100644
--- a/tests/test_specs.py
+++ b/tests/test_specs.py
@@ -252,6 +252,34 @@ async def test_openai_spec_metadata_required_fail(openai_request_data):
             assert "Missing required metadata" in resp.text
 
 
+class TestAPIWithReasoningEffort(TestAPI):
+    def encode_response(self, output, context):
+        yield ChatMessage(
+            role="assistant",
+            content=f"This is a generated output with reasoning effort: {context.get('reasoning_effort', None)}",
+        )
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize("reasoning_effort", ["low", "medium", "high", None, "random"])
+async def test_openai_spec_reasoning_effort(reasoning_effort, openai_request_data):
+    server = ls.LitServer(TestAPIWithReasoningEffort(), spec=OpenAISpec())
+    openai_request_data["reasoning_effort"] = reasoning_effort
+    with wrap_litserve_start(server) as server:
+        async with LifespanManager(server.app) as manager, AsyncClient(
+            transport=ASGITransport(app=manager.app), base_url="http://test"
+        ) as ac:
+            resp = await ac.post("/v1/chat/completions", json=openai_request_data)
+            if reasoning_effort == "random":
+                assert resp.status_code == 422  # as random is not a valid reasoning effort
+            else:
+                assert resp.status_code == 200
+                assert (
+                    resp.json()["choices"][0]["message"]["content"]
+                    == f"This is a generated output with reasoning effort: {reasoning_effort}"
+                )
+
+
 class IncorrectAPI1(ls.LitAPI):
     def setup(self, device):
         self.model = None