Repository: openai/openai-agents-python

Issue:
Configure realtime agents with Azure OpenAI's Realtime API
### Please read this first

- **Have you read the docs?**[Agents SDK docs](https://openai.github.io/openai-agents-python/)
- **Have you searched for related issues?** Others may have faced similar issue
- 
### Describe the bug

When using the OpenAI Agents SDK in Python with Azure OpenAI as the default model (via the async Azure client), the SDK ignores the Azure configuration and still requires OPENAI_API_KEY. As a result, real-time mode fails because the SDK always attempts to authenticate against OpenAI’s public API instead of using the Azure client credentials.

### Debug information
- Agents SDK version: 0.2.9
- Python 3.11

### Repro steps

Using the Realtime Quickstart Guide and setting the AsyncAzureOpenAI client via `set_default_openai_client`

```python
import asyncio
import logging

from agents import set_default_openai_client
from agents.realtime import RealtimeAgent, RealtimeRunner
from openai import AsyncAzureOpenAI

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)


async def main():
    try:
        azure_openai_client = AsyncAzureOpenAI(
            api_key="****",
            api_version="2025-03-01-preview",
            azure_endpoint="https://***.openai.azure.com",
            # azure_deployment=azure_deployment, # somehow we don't need this anymore
        )
        set_default_openai_client(azure_openai_client)
        logger.info("✅ Azure OpenAI client configured successfully")
    except Exception as e:
        logger.error(f"❌ Failed to configure Azure OpenAI client: {e}")
        raise

    # Create the agent
    agent = RealtimeAgent(
        name="Assistant",
        instructions="You are a helpful voice assistant. Keep responses brief and conversational.",
    )

    # Set up the runner with configuration
    runner = RealtimeRunner(
        starting_agent=agent,
        config={
            "model_settings": {
                "model_name": "gpt-4o-mini-realtime-preview",
                "voice": "alloy",
                "modalities": ["text", "audio"],
                "input_audio_transcription": {"model": "whisper-1"},
                "turn_detection": {
                    "type": "server_vad",
                    "threshold": 0.5,
                    "prefix_padding_ms": 300,
                    "silence_duration_ms": 200,
                },
            }
        },
    )

    # Start the session
    session = await runner.run(model_config={})

    async with session:
        print("Session started! The agent will stream audio responses in real-time.")

        # Process events
        async for event in session:
            if event.type == "response.audio_transcript.done":
                print(f"Assistant: {event.transcript}")
            elif event.type == "conversation.item.input_audio_transcription.completed":
                print(f"User: {event.transcript}")
            elif event.type == "error":
                print(f"Error: {event.error}")
                break


if __name__ == "__main__":
    asyncio.run(main())
```


### Expected behavior

If the default model is configured with the Azure client, the SDK should use the provided Azure credentials for real-time mode and shouldn't ask for `OPENAI_API_KEY`.



Discussion:
Hi, thanks for writing in. When using a different endpoint than openai.com one,

>session = await runner.run(model_config={})

The `model_config` here needs to be properly configured. Specifically, the `api_key` and `url` needs to be passed to override the default OpenAI ones.
 
@seratch Thanks for the fast response, it worked. But i stumbled on a different error where Azure is rejecting the websocket connection. Any hints or tips on this ? 

```
Traceback (most recent call last):
  File "C:\Workspaces\Playground\python-openai-agents-sdk\example.py", line 72, in <module>
    asyncio.run(main())
  File "C:\Users\parndt\AppData\Local\Programs\Python\Python311\Lib\asyncio\runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "C:\Users\parndt\AppData\Local\Programs\Python\Python311\Lib\asyncio\runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\parndt\AppData\Local\Programs\Python\Python311\Lib\asyncio\base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "C:\Workspaces\Playground\python-openai-agents-sdk\example.py", line 57, in main
    async with session:
  File "C:\Workspaces\Playground\python-openai-agents-sdk\.venv\Lib\site-packages\agents\realtime\session.py", line 131, in __aenter__
    await self._model.connect(model_config)
  File "C:\Workspaces\Playground\python-openai-agents-sdk\.venv\Lib\site-packages\agents\realtime\openai_realtime.py", line 189, in connect
    self._websocket = await websockets.connect(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Workspaces\Playground\python-openai-agents-sdk\.venv\Lib\site-packages\websockets\asyncio\client.py", line 543, in __await_impl__
    await self.connection.handshake(
  File "C:\Workspaces\Playground\python-openai-agents-sdk\.venv\Lib\site-packages\websockets\asyncio\client.py", line 114, in handshake
    raise self.protocol.handshake_exc
  File "C:\Workspaces\Playground\python-openai-agents-sdk\.venv\Lib\site-packages\websockets\client.py", line 325, in parse
    self.process_response(response)
  File "C:\Workspaces\Playground\python-openai-agents-sdk\.venv\Lib\site-packages\websockets\client.py", line 142, in process_response
    raise InvalidStatus(response)
websockets.exceptions.InvalidStatus: server rejected WebSocket connection: HTTP 200
```

Tried using the URL from [Azure Docs](https://learn.microsoft.com/en-us/azure/ai-foundry/openai/how-to/realtime-audio-websockets#connection-and-authentication) but it says not found when omitting the query parameters i get the error above for getting rejected with HTTP 200 Code.

@seratch I've managed to get it working.

I think there is a bug in the SDK where it's not using the provided API Key from `model_config` .

```python
        runner = RealtimeRunner(agent)
        session_context = await runner.run(
            model_config={
                "api_key": settings.azure_openai_api_key,
                "url": f"wss://****.openai.azure.com/openai/realtime?api-version=2025-04-01-preview&deployment=gpt-4o-mini-realtime-preview&api-key={settings.azure_openai_api_key}",
            }
        )
```

When passing in the API Key as URL Query parameter it seems to work! 

Thanks for sharing this. Using a query string is okay, but my pull request https://github.com/openai/openai-agents-python/pull/1633 will enable you to pass request headers instead.



Patch:
diff --git a/src/agents/realtime/model.py b/src/agents/realtime/model.py
index d7ebe4ffa..c0632aa9b 100644
--- a/src/agents/realtime/model.py
+++ b/src/agents/realtime/model.py
@@ -118,6 +118,12 @@ class RealtimeModelConfig(TypedDict):
     the OpenAI Realtime model will use the default OpenAI WebSocket URL.
     """
 
+    headers: NotRequired[dict[str, str]]
+    """The headers to use when connecting. If unset, the model will use a sane default.
+    Note that, when you set this, authorization header won't be set under the hood.
+    e.g., {"api-key": "your api key here"} for Azure OpenAI Realtime WebSocket connections.
+    """
+
     initial_model_settings: NotRequired[RealtimeSessionModelSettings]
     """The initial model settings to use when connecting."""
 
diff --git a/src/agents/realtime/openai_realtime.py b/src/agents/realtime/openai_realtime.py
index 766c49f8d..b9048a1ec 100644
--- a/src/agents/realtime/openai_realtime.py
+++ b/src/agents/realtime/openai_realtime.py
@@ -188,15 +188,23 @@ async def connect(self, options: RealtimeModelConfig) -> None:
         else:
             self._tracing_config = "auto"
 
-        if not api_key:
-            raise UserError("API key is required but was not provided.")
-
         url = options.get("url", f"wss://api.openai.com/v1/realtime?model={self.model}")
 
-        headers = {
-            "Authorization": f"Bearer {api_key}",
-            "OpenAI-Beta": "realtime=v1",
-        }
+        headers: dict[str, str] = {}
+        if options.get("headers") is not None:
+            # For customizing request headers
+            headers.update(options["headers"])
+        else:
+            # OpenAI's Realtime API
+            if not api_key:
+                raise UserError("API key is required but was not provided.")
+
+            headers.update(
+                {
+                    "Authorization": f"Bearer {api_key}",
+                    "OpenAI-Beta": "realtime=v1",
+                }
+            )
         self._websocket = await websockets.connect(
             url,
             user_agent_header=_USER_AGENT,
@@ -490,9 +498,7 @@ async def _handle_ws_event(self, event: dict[str, Any]):
         try:
             if "previous_item_id" in event and event["previous_item_id"] is None:
                 event["previous_item_id"] = ""  # TODO (rm) remove
-            parsed: AllRealtimeServerEvents = self._server_event_type_adapter.validate_python(
-                event
-            )
+            parsed: AllRealtimeServerEvents = self._server_event_type_adapter.validate_python(event)
         except pydantic.ValidationError as e:
             logger.error(f"Failed to validate server event: {event}", exc_info=True)
             await self._emit_event(
@@ -583,11 +589,13 @@ async def _handle_ws_event(self, event: dict[str, Any]):
         ):
             await self._handle_output_item(parsed.item)
         elif parsed.type == "input_audio_buffer.timeout_triggered":
-            await self._emit_event(RealtimeModelInputAudioTimeoutTriggeredEvent(
-                item_id=parsed.item_id,
-                audio_start_ms=parsed.audio_start_ms,
-                audio_end_ms=parsed.audio_end_ms,
-            ))
+            await self._emit_event(
+                RealtimeModelInputAudioTimeoutTriggeredEvent(
+                    item_id=parsed.item_id,
+                    audio_start_ms=parsed.audio_start_ms,
+                    audio_end_ms=parsed.audio_end_ms,
+                )
+            )
 
     def _update_created_session(self, session: OpenAISessionObject) -> None:
         self._created_session = session


Test patch:
diff --git a/tests/realtime/test_openai_realtime.py b/tests/realtime/test_openai_realtime.py
index 4c410bf6e..08b8d878f 100644
--- a/tests/realtime/test_openai_realtime.py
+++ b/tests/realtime/test_openai_realtime.py
@@ -84,8 +84,45 @@ def mock_create_task_func(coro):
 
                 # Verify internal state
                 assert model._websocket == mock_websocket
-                assert model._websocket_task is not None
-                assert model.model == "gpt-4o-realtime-preview"
+        assert model._websocket_task is not None
+        assert model.model == "gpt-4o-realtime-preview"
+
+    @pytest.mark.asyncio
+    async def test_connect_with_custom_headers_overrides_defaults(self, model, mock_websocket):
+        """If custom headers are provided, use them verbatim without adding defaults."""
+        # Even when custom headers are provided, the implementation still requires api_key.
+        config = {
+            "api_key": "unused-because-headers-override",
+            "headers": {"api-key": "azure-key", "x-custom": "1"},
+            "url": "wss://custom.example.com/realtime?model=custom",
+            # Use a valid realtime model name for session.update to validate.
+            "initial_model_settings": {"model_name": "gpt-4o-realtime-preview"},
+        }
+
+        async def async_websocket(*args, **kwargs):
+            return mock_websocket
+
+        with patch("websockets.connect", side_effect=async_websocket) as mock_connect:
+            with patch("asyncio.create_task") as mock_create_task:
+                mock_task = AsyncMock()
+
+                def mock_create_task_func(coro):
+                    coro.close()
+                    return mock_task
+
+                mock_create_task.side_effect = mock_create_task_func
+
+                await model.connect(config)
+
+                # Verify WebSocket connection used the provided URL
+                called_url = mock_connect.call_args[0][0]
+                assert called_url == "wss://custom.example.com/realtime?model=custom"
+
+                # Verify headers are exactly as provided and no defaults were injected
+                headers = mock_connect.call_args.kwargs["additional_headers"]
+                assert headers == {"api-key": "azure-key", "x-custom": "1"}
+                assert "Authorization" not in headers
+                assert "OpenAI-Beta" not in headers
 
     @pytest.mark.asyncio
     async def test_connect_with_callable_api_key(self, model, mock_websocket):