Repository: langchain4j/langchain4j (Java)

Issue:
[BUG] java.lang.NullPointerException - "tokenUsageAccumulator" is null - when using LocalAI function calling
<!-- Please provide as many details as possible, this will help us to deliver a fix as soon as possible. Thank you! -->

**Describe the bug**
<!-- A clear and concise description of what the bug is. -->
When implementing function calling using LocalAI a NPE is thrown stating "tokenUsageAccumulator" is null

**Log and Stack trace**
<!-- Please provide a log and a stack trace (with exception), if applicable. -->
```shell
Exception in thread "main" java.lang.NullPointerException: Cannot invoke "dev.langchain4j.model.output.TokenUsage.add(dev.langchain4j.model.output.TokenUsage)" because "tokenUsageAccumulator" is null
        at dev.langchain4j.service.DefaultAiServices$1.invoke(DefaultAiServices.java:175)
        at $Proxy3.chat(Unknown Source)
        at AiServicesFunctionCalling.main(AiServicesFunctionCalling.java:71)
```


**To Reproduce**
<!-- Please provide a relevant code snippets to reproduce this bug. -->

Following is the jbang script

```java
//DEPS dev.langchain4j:langchain4j:0.29.1
//DEPS dev.langchain4j:langchain4j-local-ai:0.29.1


import java.time.Duration;
import dev.langchain4j.agent.tool.Tool;
import dev.langchain4j.memory.ChatMemory;
import dev.langchain4j.memory.chat.MessageWindowChatMemory;
import dev.langchain4j.model.chat.ChatLanguageModel;
import dev.langchain4j.model.localai.LocalAiChatModel;
import dev.langchain4j.service.AiServices;

class AiServicesFunctionCalling {

    private static final String MODEL = "gpt-4";
    private static final String BASE_URL = "http://localhost:8080";
    private static final Duration timeout = Duration.ofSeconds(120);
    private static String question;

    static class Calculator {

        @Tool("Calculates the length of a string")
        int stringLength(String s) {
            System.out.println("Called stringLength with s='" + s + "'");
            return s.length();
        }

        @Tool("Calculates the sum of two numbers")
        int add(int a, int b) {
            System.out.println("Called add with a=" + a + ", b=" + b);
            return a + b;
        }

        @Tool("Calculates the square root of a number")
        double sqrt(int x) {
            System.out.println("Called sqrt with x=" + x);
            return Math.sqrt(x);
        }
    }

    interface ChatMinion {
        String chat(String message);
    }

    public static void main(String[] args) {

        ChatLanguageModel model = LocalAiChatModel.builder()
                .baseUrl(BASE_URL)
                .modelName(MODEL)
                .timeout(timeout)
                .temperature(0.0)
                .build();
        ChatMemory memory = MessageWindowChatMemory.withMaxMessages(10);

        ChatMinion minion = AiServices.builder(ChatMinion.class)
                .chatLanguageModel(model)
                .tools(new Calculator())
                .chatMemory(memory)
                .build();
        question = "What is the square root 226?";
        System.out.println("Asking question: " + question);
        String text = minion.chat(question);
        System.out.println("\n\nResponse:\n\n" + text);
    }
}
```

**Expected behavior**
<!-- A clear and concise description of what you expected to happen. -->
Should call the function and return the result

**Please complete the following information:**
- LangChain4j version: 0.29.1
- LLM(s) used: LocalAI (gpt-4)
- Java version: 21
- Spring Boot version (if applicable): NA 

**Additional context**
<!-- Add any other context about the problem here. -->



Discussion: dev.langchain4j.model.localai.LocalAiChatModel#generate(java.util.List<dev.langchain4j.data.message.ChatMessage>, java.util.List<dev.langchain4j.agent.tool.ToolSpecification>, dev.langchain4j.agent.tool.ToolSpecification)

```java
private Response<AiMessage> generate(List<ChatMessage> messages,
                                         List<ToolSpecification> toolSpecifications,
                                         ToolSpecification toolThatMustBeExecuted
    ) {
        ChatCompletionRequest.Builder requestBuilder = ChatCompletionRequest.builder()
                .model(modelName)
                .messages(toOpenAiMessages(messages))
                .temperature(temperature)
                .topP(topP)
                .maxTokens(maxTokens);
        ...
        return Response.from(
                aiMessageFrom(response),
                null,
                finishReasonFrom(response.choices().get(0).finishReason())
        );
    }
```
the response of `generate`  returned that `tokenUsage` is null
So, is null here the correct situation? I have considered two ways
- proxy method `invoke` compatible null value
- diffrent `TokenUsage` implement

I hope to try to solve it~
Related to https://github.com/langchain4j/langchain4j/issues/921
@Kugaaa I think AI Service should be changed to handle `TokenUsage == null` situation well, not all LLM providers return it.

Ideally, `LocalAiChatModel` should return `TokenUsage`, if LocalAI returns it (need to check it).
Please raise priority
@ohadpinch it should have been fixed by https://github.com/langchain4j/langchain4j/pull/939/files, could you please check the latest version (or snapshot)?
> @ohadpinch it should have been fixed by https://github.com/langchain4j/langchain4j/pull/939/files, could you please check the latest version (or snapshot)?

I checked it, it's fixed with those changes.
@ohadpinch thank you!
Fixed by #939
@langchain4j sorry for the mix up!
The issue still exist on https://github.com/langchain4j/langchain4j/pull/939, I ment I checked it with  https://github.com/langchain4j/langchain4j/pull/942 and it seems solved.

> @Kugaaa I think AI Service should be changed to handle `TokenUsage == null` situation well, not all LLM providers return it.
> 
> Ideally, `LocalAiChatModel` should return `TokenUsage`, if LocalAI returns it (need to check it).

@langchain4j I think [#939](https://github.com/langchain4j/langchain4j/pull/939) not compatible with this situation

[#942](https://github.com/langchain4j/langchain4j/pull/942) If the solution is just return a null value, it appears that too little information is provided, so I want to use the null object mode return a `NullTokenUsage`, it will throw an exception and more infomation about used LLM not supported token usage.



Fix (patch):
diff --git a/langchain4j-core/src/main/java/dev/langchain4j/model/output/TokenUsage.java b/langchain4j-core/src/main/java/dev/langchain4j/model/output/TokenUsage.java
index c4f034503c5..63000b9a111 100644
--- a/langchain4j-core/src/main/java/dev/langchain4j/model/output/TokenUsage.java
+++ b/langchain4j-core/src/main/java/dev/langchain4j/model/output/TokenUsage.java
@@ -79,6 +79,33 @@ public Integer totalTokenCount() {
         return totalTokenCount;
     }
 
+    /**
+     * Adds two token usages.
+     * <br>
+     * If one of the token usages is null, the other is returned without changes.
+     * <br>
+     * Fields which are null in both responses will be null in the result.
+     *
+     * @param first The first token usage to add.
+     * @param second The second token usage to add.
+     * @return a new {@link TokenUsage} instance with the sum of token usages.
+     */
+    public static TokenUsage sum(TokenUsage first, TokenUsage second) {
+        if (first == null) {
+            return second;
+        }
+
+        if (second == null) {
+            return first;
+        }
+
+        return new TokenUsage(
+                sum(first.inputTokenCount, second.inputTokenCount),
+                sum(first.outputTokenCount, second.outputTokenCount),
+                sum(first.totalTokenCount, second.totalTokenCount)
+        );
+    }
+
     /**
      * Adds the token usage of two responses together.
      *
@@ -86,7 +113,9 @@ public Integer totalTokenCount() {
      *
      * @param that The token usage to add to this one.
      * @return a new {@link TokenUsage} instance with the token usage of both responses added together.
+     * @deprecated use {@link #sum(TokenUsage, TokenUsage)} instead
      */
+    @Deprecated
     public TokenUsage add(TokenUsage that) {
         if (that == null) {
             return new TokenUsage(inputTokenCount, outputTokenCount, totalTokenCount);
diff --git a/langchain4j/src/main/java/dev/langchain4j/service/AiServiceStreamingResponseHandler.java b/langchain4j/src/main/java/dev/langchain4j/service/AiServiceStreamingResponseHandler.java
index 64e3c818400..b9eb74fa28d 100644
--- a/langchain4j/src/main/java/dev/langchain4j/service/AiServiceStreamingResponseHandler.java
+++ b/langchain4j/src/main/java/dev/langchain4j/service/AiServiceStreamingResponseHandler.java
@@ -10,7 +10,6 @@
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
-import java.util.List;
 import java.util.function.Consumer;
 
 import static dev.langchain4j.internal.ValidationUtils.ensureNotNull;
@@ -82,14 +81,14 @@ public void onComplete(Response<AiMessage> response) {
                             tokenHandler,
                             completionHandler,
                             errorHandler,
-                            tokenUsage.add(response.tokenUsage())
+                            TokenUsage.sum(tokenUsage, response.tokenUsage())
                     )
             );
         } else {
             if (completionHandler != null) {
                 completionHandler.accept(Response.from(
                         aiMessage,
-                        tokenUsage.add(response.tokenUsage()),
+                        TokenUsage.sum(tokenUsage, response.tokenUsage()),
                         response.finishReason())
                 );
             }
diff --git a/langchain4j/src/main/java/dev/langchain4j/service/DefaultAiServices.java b/langchain4j/src/main/java/dev/langchain4j/service/DefaultAiServices.java
index 0c1fc57d43a..9c3d1fd2f31 100644
--- a/langchain4j/src/main/java/dev/langchain4j/service/DefaultAiServices.java
+++ b/langchain4j/src/main/java/dev/langchain4j/service/DefaultAiServices.java
@@ -187,7 +187,7 @@ public Object invoke(Object proxy, Method method, Object[] args) throws Exceptio
                             }
 
                             response = context.chatModel.generate(messages, context.toolSpecifications);
-                            tokenUsageAccumulator = tokenUsageAccumulator.add(response.tokenUsage());
+                            tokenUsageAccumulator = TokenUsage.sum(tokenUsageAccumulator, response.tokenUsage());
                         }
 
                         response = Response.from(response.content(), tokenUsageAccumulator, response.finishReason());