Repository: langchain4j/langchain4j (2026-04-09T22:50:24Z)

Issue:
[BUG] OpenAiStreamingResponseBuilder drops additional tool calls from same delta, causing AiMessage/tool-execution mismatch
**Describe the bug**
When using `OpenAiStreamingChatModel` on the OpenAI-compatible streaming path, tool execution can diverge from the final `AiMessage`:

- all tool calls are emitted and processed via `onCompleteToolCall(...)`
- but `onCompleteResponse(...).aiMessage().toolExecutionRequests()` may contain only the first tool call from a delta

This creates inconsistent history/memory: assistant `AiMessage` contains fewer tool requests than tool results produced for the same turn.

**Log and Stack trace**
No exception is thrown (logical/state mismatch, not a crash).

Observed callback sequence:

```text
onCompleteToolCall(index=0, id=tool-1, ...)
onCompleteToolCall(index=1, id=tool-2, ...)
onCompleteResponse.aiMessage.toolExecutionRequests.size=1
executedToolResults.size=2
```

Resulting history for that turn:

- assistant `AiMessage`: 1 tool request
- tool result messages: 2 entries

**To Reproduce**
Two minimal ways:

1) Black-box runtime repro (no code-heavy setup)

- Use any OpenAI-compatible streaming model/tool setup.
- Ask model to call tools in parallel, e.g.:
  - `Run tool run_sql for each query in parallel: SELECT 1, SELECT 2, ..., SELECT 7. Return only tool calls.`
- Observe:
  - multiple `onCompleteToolCall(...)` callbacks
  - final `onCompleteResponse(...).aiMessage().toolExecutionRequests().size()` can be smaller (often `1`) in affected versions.

2) Low-level unit-test repro (preferred for maintainers)

```java
package dev.langchain4j.model.openai;

import static org.assertj.core.api.Assertions.assertThat;

import dev.langchain4j.model.chat.response.ChatResponse;
import dev.langchain4j.model.openai.internal.chat.*;
import java.util.List;
import org.junit.jupiter.api.Test;

class OpenAiStreamingResponseBuilderRegressionTest {

    @Test
    void should_keep_all_tool_calls_from_same_delta() {
        OpenAiStreamingResponseBuilder builder = new OpenAiStreamingResponseBuilder();

        ToolCall tc1 = ToolCall.builder()
                .id("call_1")
                .index(0)
                .type(ToolType.FUNCTION)
                .function(FunctionCall.builder().name("run_sql").arguments("{\"q\":\"SELECT 1\"}").build())
                .build();

        ToolCall tc2 = ToolCall.builder()
                .id("call_2")
                .index(1)
                .type(ToolType.FUNCTION)
                .function(FunctionCall.builder().name("run_sql").arguments("{\"q\":\"SELECT 2\"}").build())
                .build();

        ChatCompletionResponse partial = ChatCompletionResponse.builder()
                .id("resp_1")
                .model("openai-compatible-model")
                .choices(List.of(ChatCompletionChoice.builder()
                        .index(0)
                        .delta(Delta.builder().toolCalls(List.of(tc1, tc2)).build())
                        .build()))
                .build();

        builder.append(partial);
        ChatResponse response = builder.build();

        assertThat(response.aiMessage().toolExecutionRequests()).hasSize(2);
        assertThat(response.aiMessage().toolExecutionRequests())
                .extracting(it -> it.id())
                .containsExactly("call_1", "call_2");
    }
}
```

**Expected behavior**
`ChatResponse.aiMessage().toolExecutionRequests()` must contain the same complete set of tool calls that were emitted/executed for that model turn.

**Please complete the following information:**
- LangChain4j version: `1.13.0-SNAPSHOT` (observed in current environment; likely affects versions with same `OpenAiStreamingResponseBuilder.append` logic)
- LLM(s) used: Mistral via OpenAI-compatible API
- Java version: 17
- Spring Boot version (if applicable): N/A

**Additional context**
Likely root cause in affected versions is asymmetry between streaming callback handling and final response building:

- `OpenAiStreamingChatModel.handle(...)` iterates all `delta.toolCalls()`
- `OpenAiStreamingResponseBuilder.append(...)` keeps only first tool call from a multi-call delta

Suggested fix:

- In `OpenAiStreamingResponseBuilder.append(...)`, iterate all `delta.toolCalls()` entries and reuse existing per-entry accumulation logic.

Suggested regression test:

- Add a test in `OpenAiStreamingResponseBuilderTest`
- feed one streamed delta containing 2 tool calls
- assert final `aiMessage.toolExecutionRequests()` size is 2 and IDs/names/arguments for both entries are preserved

Possibly related:

- #4573
- #4528



Discussion:

Patch:
diff --git a/langchain4j-open-ai/src/main/java/dev/langchain4j/model/openai/OpenAiStreamingResponseBuilder.java b/langchain4j-open-ai/src/main/java/dev/langchain4j/model/openai/OpenAiStreamingResponseBuilder.java
index ec2f4679558..9256efc3794 100644
--- a/langchain4j-open-ai/src/main/java/dev/langchain4j/model/openai/OpenAiStreamingResponseBuilder.java
+++ b/langchain4j-open-ai/src/main/java/dev/langchain4j/model/openai/OpenAiStreamingResponseBuilder.java
@@ -162,38 +162,39 @@ public void append(ChatCompletionResponse partialResponse) {
             }
         }
 
-        if (delta.toolCalls() != null && !delta.toolCalls().isEmpty()) {
-            ToolCall toolCall = delta.toolCalls().get(0);
-            int toolCallIndex = toolCall.index() != null ? toolCall.index() : fallbackToolCallIndex.get();
-
-            ToolExecutionRequestBuilder builder = this.indexToToolExecutionRequestBuilder.computeIfAbsent(
-                    toolCallIndex, idx -> new ToolExecutionRequestBuilder());
-
-            // When index is null and a different tool call id appears, increment the fallback index
-            if (toolCall.index() == null
-                    && toolCall.id() != null
-                    && !builder.idBuilder.isEmpty()
-                    && !builder.idBuilder.toString().equals(toolCall.id())) {
-                toolCallIndex = fallbackToolCallIndex.incrementAndGet();
-                builder = this.indexToToolExecutionRequestBuilder.computeIfAbsent(
+        if (delta.toolCalls() != null) {
+            for (ToolCall toolCall : delta.toolCalls()) {
+                int toolCallIndex = toolCall.index() != null ? toolCall.index() : fallbackToolCallIndex.get();
+
+                ToolExecutionRequestBuilder builder = this.indexToToolExecutionRequestBuilder.computeIfAbsent(
                         toolCallIndex, idx -> new ToolExecutionRequestBuilder());
-            }
 
-            if (toolCall.id() != null) {
-                if (accumulateToolCallId) {
-                    builder.idBuilder.append(toolCall.id());
-                } else {
-                    builder.idBuilder.setLength(0);
-                    builder.idBuilder.append(toolCall.id());
+                // When index is null and a different tool call id appears, increment the fallback index
+                if (toolCall.index() == null
+                        && toolCall.id() != null
+                        && !builder.idBuilder.isEmpty()
+                        && !builder.idBuilder.toString().equals(toolCall.id())) {
+                    toolCallIndex = fallbackToolCallIndex.incrementAndGet();
+                    builder = this.indexToToolExecutionRequestBuilder.computeIfAbsent(
+                            toolCallIndex, idx -> new ToolExecutionRequestBuilder());
                 }
-            }
 
-            FunctionCall functionCall = toolCall.function();
-            if (functionCall.name() != null) {
-                builder.nameBuilder.append(functionCall.name());
-            }
-            if (functionCall.arguments() != null) {
-                builder.argumentsBuilder.append(functionCall.arguments());
+                if (toolCall.id() != null) {
+                    if (accumulateToolCallId) {
+                        builder.idBuilder.append(toolCall.id());
+                    } else {
+                        builder.idBuilder.setLength(0);
+                        builder.idBuilder.append(toolCall.id());
+                    }
+                }
+
+                FunctionCall functionCall = toolCall.function();
+                if (functionCall.name() != null) {
+                    builder.nameBuilder.append(functionCall.name());
+                }
+                if (functionCall.arguments() != null) {
+                    builder.argumentsBuilder.append(functionCall.arguments());
+                }
             }
         }
     }


Test patch:
diff --git a/langchain4j-open-ai/src/test/java/dev/langchain4j/model/openai/OpenAiStreamingResponseBuilderTest.java b/langchain4j-open-ai/src/test/java/dev/langchain4j/model/openai/OpenAiStreamingResponseBuilderTest.java
index 7a1675cba9f..86795b78ac2 100644
--- a/langchain4j-open-ai/src/test/java/dev/langchain4j/model/openai/OpenAiStreamingResponseBuilderTest.java
+++ b/langchain4j-open-ai/src/test/java/dev/langchain4j/model/openai/OpenAiStreamingResponseBuilderTest.java
@@ -2,6 +2,7 @@
 
 import static org.assertj.core.api.Assertions.assertThat;
 
+import dev.langchain4j.agent.tool.ToolExecutionRequest;
 import dev.langchain4j.model.chat.response.ChatResponse;
 import dev.langchain4j.model.openai.internal.chat.*;
 import java.util.List;
@@ -103,6 +104,55 @@ void should_handle_non_null_tool_call_index() {
                 .isEqualTo("getTemperature");
     }
 
+    @Test
+    void should_keep_all_tool_calls_from_same_delta() {
+
+        // given
+        OpenAiStreamingResponseBuilder builder = new OpenAiStreamingResponseBuilder();
+
+        ToolCall tc1 = ToolCall.builder()
+                .id("call_1")
+                .index(0)
+                .type(ToolType.FUNCTION)
+                .function(FunctionCall.builder()
+                        .name("tool_name")
+                        .arguments("{}")
+                        .build())
+                .build();
+
+        ToolCall tc2 = ToolCall.builder()
+                .id("call_2")
+                .index(1)
+                .type(ToolType.FUNCTION)
+                .function(FunctionCall.builder()
+                        .name("tool_name")
+                        .arguments("{}")
+                        .build())
+                .build();
+
+        ChatCompletionResponse partial = ChatCompletionResponse.builder()
+                .id("resp_1")
+                .model("openai-compatible-model")
+                .choices(List.of(ChatCompletionChoice.builder()
+                        .index(0)
+                        .delta(Delta.builder()
+                                .toolCalls(List.of(tc1, tc2))
+                                .build())
+                        .build()))
+                .build();
+
+        // when
+        builder.append(partial);
+        ChatResponse response = builder.build();
+
+        // then
+        List<ToolExecutionRequest> toolExecutionRequests = response.aiMessage().toolExecutionRequests();
+        assertThat(toolExecutionRequests).hasSize(2);
+        assertThat(toolExecutionRequests)
+                .extracting(ToolExecutionRequest::id)
+                .containsExactly("call_1", "call_2");
+    }
+
     private static ChatCompletionResponse chatCompletionResponse(ToolCall toolCall) {
         return ChatCompletionResponse.builder()
                 .id("resp_1")