Repository: openai/openai-agents-js (2025-08-09T21:27:08Z)

Issue:
RealtimeSession.updateAgent reverts audio input/output format to pcm16
### Describe the bug

This is a fatal issue regarding Twilio call support + the `updateAgent` method.

When using the `RealtimeSession`'s `updateAgent` method, the original session's `config` values are not maintained. As a result, if you had an  `inputAudioFormat` of `g711_ulaw`, the session update will revert back to `pcm16`. Twilio requires `g711_ulaw`, and sending the wrong data will cause the user to hear loud static.

There is no clear workaround that enables updating the agent using this method.

The error begins inside the `updateAgent` function, where it calls `updateSessionConfig` with an unfaithful copy of the config generated by the `getSessionConfig` function. 

[The exact location of the issue is here](https://github.com/openai/openai-agents-js/blob/191b82a9e77e3667422a727b37f9262e1928c1e7/packages/agents-realtime/src/realtimeSession.ts#L331).

The `getSessionConfig` function returns a "manual" copy of the config, like so:
```typescript
return {
      instructions,
      voice: this.#currentAgent.voice,
      model: this.options.model,
      tools: this.#currentTools,
      tracing: tracingConfig,
      ...additionalConfig, // this is a function argument, which is not passed when `updateSessionConfig` calls it
};
```

As a result of this manual copy of the config ignoring important session config properties like `inputAudioFormat` and `outputAudioFormat`, the server reverts to its default of `pcm16`.

In the context of a phonecall with Twilio, this causes erroneous data to be sent to Twilio and the caller hears loud static.

### Proposed Resolution

Update the `getSessionConfig` method to more accurately get the entire set of values for the session config.

### Debug information

- Agents SDK version: (e.g. `v0.0.15`)
- Runtime environment (e.g. `Node.js 24`)

### Repro steps

```typescript
session.updateAgent(
	agent.clone({
		name: 'Agent Bob',
		instructions:
			'You are now speaking as Bob. Greet the user by saying "Howdy, I am Bob" when you join the call.',
	}),
);
```

### Expected behavior

Updating the agent should not impact any other parts of the RealtimeSession configuration.



Discussion:

Patch:
diff --git a/.changeset/preserve-audio-formats-on-agent-update.md b/.changeset/preserve-audio-formats-on-agent-update.md
new file mode 100644
index 00000000..de1d458e
--- /dev/null
+++ b/.changeset/preserve-audio-formats-on-agent-update.md
@@ -0,0 +1,5 @@
+---
+'@openai/agents-realtime': patch
+---
+
+fix(realtime-session): preserve audio format & other session config fields on agent update
diff --git a/packages/agents-realtime/src/realtimeSession.ts b/packages/agents-realtime/src/realtimeSession.ts
index 3c4f22ef..1c7da253 100644
--- a/packages/agents-realtime/src/realtimeSession.ts
+++ b/packages/agents-realtime/src/realtimeSession.ts
@@ -183,6 +183,13 @@ export class RealtimeSession<
   #shouldIncludeAudioData: boolean;
   #interruptedByGuardrail: Record<string, boolean> = {};
   #audioStarted = false;
+  // Keeps track of the last full session config we sent (camelCase keys) so that
+  // subsequent updates (e.g. during agent handoffs) preserve properties that are
+  // not explicitly recalculated here (such as inputAudioFormat, outputAudioFormat,
+  // modalities, speed, toolChoice, turnDetection, etc.). Without this, updating
+  // the agent would drop audio format overrides (e.g. g711_ulaw) and revert to
+  // transport defaults causing issues for integrations like Twilio.
+  #lastSessionConfig: Partial<RealtimeSessionConfig> | null = null;
 
   constructor(
     public readonly initialAgent:
@@ -313,14 +320,34 @@ export class RealtimeSession<
       );
     }
 
-    return {
+    // Start from any previously-sent config (so we preserve values like audio formats)
+    // and the original options.config provided by the user. Preference order:
+    // 1. Last session config we sent (#lastSessionConfig)
+    // 2. Original options.config
+    // 3. Additional config passed into this invocation (explicit overrides)
+    // Finally we overwrite dynamic fields (instructions, voice, model, tools, tracing)
+    // to ensure they always reflect the current agent & runtime state.
+    const base: Partial<RealtimeSessionConfig> = {
+      ...(this.#lastSessionConfig ?? {}),
+      ...(this.options.config ?? {}),
+      ...(additionalConfig ?? {}),
+    };
+
+    // Note: Certain fields cannot be updated after the session begins, such as voice and model
+    const fullConfig: Partial<RealtimeSessionConfig> = {
+      ...base,
       instructions,
       voice: this.#currentAgent.voice,
       model: this.options.model,
       tools: this.#currentTools,
       tracing: tracingConfig,
-      ...additionalConfig,
     };
+
+    // Update our cache so subsequent updates inherit the full set including any
+    // dynamic fields we just overwrote.
+    this.#lastSessionConfig = fullConfig;
+
+    return fullConfig;
   }
 
   async updateAgent(newAgent: RealtimeAgent<TBaseContext>) {
@@ -689,6 +716,9 @@ export class RealtimeSession<
       url: options.url,
       initialSessionConfig: await this.#getSessionConfig(this.options.config),
     });
+    // Ensure the cached lastSessionConfig includes everything passed as the initial session config
+    // (the call above already set it via #getSessionConfig but in case additional overrides were
+    // passed directly here in the future we could merge them). For now it's a no-op.
 
     this.#history = [];
     this.emit('history_updated', this.#history);


Test patch:
diff --git a/packages/agents-realtime/test/realtimeSession.test.ts b/packages/agents-realtime/test/realtimeSession.test.ts
index e39df378..9aa656d5 100644
--- a/packages/agents-realtime/test/realtimeSession.test.ts
+++ b/packages/agents-realtime/test/realtimeSession.test.ts
@@ -300,4 +300,20 @@ describe('RealtimeSession', () => {
     } as any);
     expect(startEvents).toBe(2);
   });
+
+  it('preserves custom audio formats across updateAgent', async () => {
+    const t = new FakeTransport();
+    const agent = new RealtimeAgent({ name: 'Orig', handoffs: [] });
+    const s = new RealtimeSession(agent, {
+      transport: t,
+      config: { inputAudioFormat: 'g711_ulaw', outputAudioFormat: 'g711_ulaw' },
+    });
+    await s.connect({ apiKey: 'test' });
+    const newAgent = new RealtimeAgent({ name: 'Next', handoffs: [] });
+    await s.updateAgent(newAgent);
+    // Find the last updateSessionConfig call
+    const last = t.updateSessionConfigCalls.at(-1)!;
+    expect(last.inputAudioFormat).toBe('g711_ulaw');
+    expect(last.outputAudioFormat).toBe('g711_ulaw');
+  });
 });