AI/ML - LangChain4j - AiServices
Tags: machinelearning, langchain4j, java, ollama
Published: 2024-04-08 06:00:00

The [previous article][1] our focus has been on delving into foundational elements of [LangChain4j][2] such as `ChatLanguageModel`, `ChatMessage`, `ChatMemory`, and others. Working with components at this level offers great flexibility and complete control, but it comes with the added burden of writing extensive boilerplate code. LLM-driven applications typically necessitate a multitude of interconnected components rather than a single component. Those components includes prompt templates, chat memory, LLMs, output parsers, RAG components: embedding models and stores. It frequently involves numerous interactions between these components and coordinating them all becomes an intricate and laborious task.

LangChain4j aims to simplify the development process by allowing the developers to concentrate on the core business logic without getting bogged down in intricate implementation details. To achieve this, LangChain4j provides two essential high-level abstractions: `AiServices` and `Chains`. These concepts are designed to streamline the workflow, enables to leverage the power of AI effectively while minimizing the complexity of low-level operations.

## AiServices and Chains

### AiServices

LangChain4j introduces a novel approach called `AiServices`, tailored specifically for the Java ecosystem. The primary objective of AI Services is to abstract away the complexities associated with interacting with Large Language Models (LLMs) and other components, providing a simple and intuitive API.

This approach draws inspiration from popular frameworks like Spring Data JPA and Retrofit, where developers can declaratively define an interface representing the desired API, and LangChain4j automatically generates an object (proxy) that implements this interface. `AiServices` can be perceived as a component within the service layer of the application, designed to provide AI-powered services, hence the name.

`AiServices` streamline common operations such as formatting inputs for LLMs and parsing outputs from LLMs. Furthermore, they support advanced features like chat memory, tools (Function Calling), and Retrieval-Augmented Generation (RAG).

`AiServices` can be leveraged to build stateful interactive applications that facilitate back-and-forth interactions, as well as to automate processes where each call to the LLM is isolated and self-contained. This versatility empowers developers to harness the power of AI in a seamless and efficient manner, without the need to delve into low-level implementation details.

### Chains

LangChain4j also offers another composable approach called `Chains`. The concept of Chains originates from [Python's LangChain][3]. `Chains` combine multiple low-level components and orchestrate the interactions between them, enabling the creation of more complex and customized workflows.

However, one potential drawback of `Chains` is their inherent rigidity, which can pose challenges when customization is required. Currently, LangChain4j has implemented only two types of Chains: `ConversationalChain` and `ConversationalRetrievalChain`.

The `ConversationalChain` facilitates back-and-forth conversations with an LLM, maintaining context and memory across multiple interactions. On the other hand, the `ConversationalRetrievalChain` extends this functionality by incorporating a retrieval component (`ContentRetriever`), allowing the LLM to access and leverage external data sources during the conversation.

LangChain4J recommends using the `AiServices` instead of `Chains` as it is more flexible, declarative and provides a simple API.

In this article, we will see various examples of using `AiServices`

First let's see a basic AiServices examples

```java
//DEPS dev.langchain4j:langchain4j:0.29.1
//DEPS dev.langchain4j:langchain4j-ollama:0.29.1

import java.io.Console;
import java.time.Duration;
import java.util.Set;

import dev.langchain4j.memory.ChatMemory;
import dev.langchain4j.memory.chat.MessageWindowChatMemory;
import dev.langchain4j.model.chat.ChatLanguageModel;
import dev.langchain4j.model.ollama.OllamaChatModel;
import dev.langchain4j.service.AiServices;

class AiServicesBasic {

    private static final String MODEL = "mistral";
    private static final String BASE_URL = "http://localhost:11434";
    private static final Duration timeout = Duration.ofSeconds(120);

    interface ChatMinion {
        String chat(String message);
    }

    public static void main(String[] args) {

        ChatLanguageModel model = OllamaChatModel.builder()
                .baseUrl(BASE_URL)
                .modelName(MODEL)
                .temperature(0.2)
                .timeout(timeout)
                .build();
        ChatMemory memory = MessageWindowChatMemory.withMaxMessages(10);

        ChatMinion minion = AiServices.builder(ChatMinion.class)
                .chatLanguageModel(model)
                .chatMemory(memory)
                .build();
        Console console = System.console();
        String question = console.readLine("\n\nPlease enter your question: ");

        Set<String> set = Set.of("exit", "quit");
        while (!set.contains(question)) {
            String response = minion.chat(question);
            System.out.println(response);
            question = console.readLine("\n\nPlease enter your question: ");
        }

    }
}
```

We define the simple interface.

```java
interface ChatMinion {
    String chat(String message);
}
```

Next come the low-level components `ChatLanguageModel` and `ChatMemory`

```java
ChatLanguageModel model = OllamaChatModel.builder()
        .baseUrl(BASE_URL)
        .modelName(MODEL)
        .temperature(0.2)
        .timeout(timeout)
        .build();
ChatMemory memory = MessageWindowChatMemory.withMaxMessages(10);
```

Finally, create the `AiServices` object using the low-level components we created above.

```java
ChatMinion minion = AiServices.builder(ChatMinion.class)
        .chatLanguageModel(model)
        .chatMemory(memory)
        .build();
```

LangChain4j's `AiServices` utility enables creating the proxy objects that implement custom interfaces defined (`ChatMinion` in this case). `AiServices` should be provided the `Class` of the interface along with the low-level components to be integrated (`ChatLanguageModel` and `ChatMemory`). `AiServices` then generates a proxy object that implements this interface using reflection.

This proxy object handles the necessary conversions for inputs and outputs, abstracting away the complexities of working with low-level components. Here the `ChatMinion` interface has a `chat` method that accepts a `String` as input. However, the underlying `ChatLanguageModel` component expects a `ChatMessage` object as input. In this scenario, `AiServices` will automatically convert the `String` input into a `UserMessage` before invoking the `ChatLanguageModel`.

Similarly, when the `ChatLanguageModel` returns an `AiMessage`, `AiServices` will convert it into a `String` before returning the result from the `chat` method. This seamless conversion process allows working with familiar data types in the application code, while `AiServices` handles the underlying transformations and interactions with the low-level components transparently.

```java
String response = minion.chat(question);
```

To execute the statement above, `AiServices` does the heavylifting just described. 

The next code example is the stream version of the `AiServices` where the responses are streamed unlike the previous one.

One additional point to note is that the LLM responds in a sarcastic tone! That is achieved using the `SystemMessage` annotation. `AiServices` takes care of passing the `SystemMessage` to the LLM

```java
interface ChatMinion {
    @SystemMessage("Answer in a sarcastic tone.")
    TokenStream chat(String message);
}
```

Here's the full code.

```java
//DEPS dev.langchain4j:langchain4j:0.29.1
//DEPS dev.langchain4j:langchain4j-ollama:0.29.1

import java.io.Console;
import java.time.Duration;
import java.util.Set;
import java.util.concurrent.CompletableFuture;

import dev.langchain4j.data.message.AiMessage;
import dev.langchain4j.memory.ChatMemory;
import dev.langchain4j.memory.chat.MessageWindowChatMemory;
import dev.langchain4j.model.chat.StreamingChatLanguageModel;
import dev.langchain4j.model.ollama.OllamaStreamingChatModel;
import dev.langchain4j.model.output.Response;
import dev.langchain4j.service.AiServices;
import dev.langchain4j.service.SystemMessage;
import dev.langchain4j.service.TokenStream;

class AiServicesStream {

    private static final String MODEL = "mistral";
    private static final String BASE_URL = "http://localhost:11434";
    private static final Duration timeout = Duration.ofSeconds(120);
    private static String question;

    interface ChatMinion {
        @SystemMessage("Answer in a sarcastic tone.")
        TokenStream chat(String message);
    }

    public static void main(String[] args) {

        StreamingChatLanguageModel model = OllamaStreamingChatModel.builder()
                .baseUrl(BASE_URL)
                .modelName(MODEL)
                .timeout(timeout)
                .temperature(0.0)
                .build();
        ChatMemory memory = MessageWindowChatMemory.withMaxMessages(10);

        ChatMinion minion = AiServices.builder(ChatMinion.class)
                .streamingChatLanguageModel(model)
                .chatMemory(memory)
                .build();
        Console console = System.console();
        question = console.readLine("\n\nPlease enter your question: ");
        Set<String> set = Set.of("exit", "quit");
        while (!set.contains(question.toLowerCase())) {
            CompletableFuture<Response<AiMessage>> future = new CompletableFuture<>();
            TokenStream stream = minion.chat(question);

            stream.onNext(System.out::print)
                    .onComplete(response -> {
                        future.complete(response);
                    })
                    .onError(error -> {
                        future.completeExceptionally(error);
                    })
                    .start();
            future.join();
            question = console.readLine("\n\nPlease enter your question: ");
        }
        System.exit(0);
    }
}
```
Sample input and output for the above

```
Please enter your question: Hello my name is Kevin

Oh, hello there! I'm just the most interesting person you'll ever meet. 
You know, I have a pet unicorn that I ride to work every day. 
And I can solve a Rubik's cube in under 5 seconds, no big deal. 
So, what's your superpower? Oh wait, you don't have one? 
Well, I guess we'll just have to be impressed by my extraordinary abilities then.

Please enter your question: What is my name?

Oh, right! I almost forgot. 
Your name is... oh, who am I kidding? 
It doesn't matter what your name is. 
I bet you're still more boring than a snail's race. 
But hey, keep trying to impress me with your mundane existence! 
It's always a good laugh.
```

While the current examples have focused on text-based interactions with Large Language Models (LLMs), the power of `AiServices` extends far beyond handling plain text data. `AiServices` can leverage the capabilities of LLMs to work with various types of structured data, such as Plain Old Java Objects (POJOs), Collection classes, and more.

By leveraging the versatility of LLMs, `AiServices` can seamlessly convert structured data objects into a format suitable for the LLM, process the data, and then convert the LLM's output back into the desired structured data format. This powerful feature enables developers to harness the power of LLMs for a wide range of tasks involving structured data, such as data processing, analysis, transformation, and generation.

For instance, you could define an interface that accepts or returns a POJO representing a complex data structure, and `AiServices` will handle the conversion between the POJO and the LLM's input/output format transparently. Similarly, you could work with Collection classes like Lists or Maps, allowing the LLM to process and manipulate the data within these collections.

This capability opens up numerous possibilities for integrating LLMs into various domains and applications, extending their utility beyond pure text-based tasks. With `AiServices`, you can leverage the power of LLMs to process and manipulate structured data in a seamless and intuitive manner, without being constrained by the limitations of traditional text-based interfaces.

In the following examples, let's build a crude profanity filter in the text.

```java
//DEPS dev.langchain4j:langchain4j:0.29.1
//DEPS dev.langchain4j:langchain4j-open-ai:0.29.1

import java.util.Map;

import dev.langchain4j.model.chat.ChatLanguageModel;
import dev.langchain4j.model.openai.OpenAiChatModel;
import dev.langchain4j.service.AiServices;
import dev.langchain4j.service.UserMessage;

class AiServicesWordAnalysis {

    enum WordAnalysis {
        OFFENSIVE, BAD, NEUTRAL, GOOD
    }

    interface WordModerator {
        @UserMessage("Analyze the profanity of {{it}}")
        WordAnalysis analyzeWords(String text);

        @UserMessage("Does {{it}} have a profanity?")
        boolean isProfane(String text);

        @UserMessage("Provide alternate better words for the profane words in {{it}}")
        Map<String, String> alternateWords(String text);
    }

    public static void main(String[] args) {

        ChatLanguageModel model = OpenAiChatModel.withApiKey("demo");

        WordModerator moderator = AiServices.create(WordModerator.class, model);

        WordAnalysis analysis =moderator.analyzeWords("He is shit");
        System.out.println("Analysis: " + analysis);

        boolean isProfane = moderator.isProfane("He is a dumbo");
        System.out.println("Is Profane: " + isProfane);

        Map<String, String> replacements = moderator.alternateWords("He is not intelligent but a shit and dumbo");
        System.out.println(replacements);
    }
}
```

Here we define our interface `WordModerator` that defines three methods and the enum `WordAnalysis` to be supplied to the `AiServices` 

```java
enum WordAnalysis {
    OFFENSIVE, BAD, NEUTRAL, GOOD
}

interface WordModerator {
    @UserMessage("Analyze the profanity of {{it}}")
    WordAnalysis analyzeWords(String text);

    @UserMessage("Does {{it}} have a profanity?")
    boolean isProfane(String text);

    @UserMessage("Provide alternate better words for the profane words in {{it}}")
    Map<String, String> alternateWords(String text);
}
```
- `analyzeWords` determines if the degree of profanity in the given text as defined in the enum `WordAnalysis` - `OFFENSIVE`, `BAD`, `NEUTRAL`, `GOOD`.
- `isProfane` returns `true` or `false` indicating the presence of profanity in the given text.
- `alternateWords` finds out the list of words that are profane and the LLM provides alternate words to be replaced for those words

As mentioned above, `AiServices` handles the conversion of input/output between the application and the LLM in a transparent manner.

The following is what the LLM returns when running the code

```
Analysis: OFFENSIVE
Is Profane: true
{shit=fool, dumbo=dimwit}
```

In the next example, we will see how `AiServices` deals with POJOs. We will try to create a primitive resume screener and the names extractor from the text. It is quite similar to the previous example

```java
//DEPS dev.langchain4j:langchain4j:0.29.1
//DEPS dev.langchain4j:langchain4j-ollama:0.29.1

import java.time.Duration;
import java.util.List;

import dev.langchain4j.model.chat.ChatLanguageModel;
import dev.langchain4j.model.ollama.OllamaChatModel;
import dev.langchain4j.service.AiServices;
import dev.langchain4j.service.UserMessage;

class AiServicesCandidateInfo {

    private static final String MODEL = "mistral";
    private static final String BASE_URL = "http://localhost:11434";
    private static final Duration timeout = Duration.ofSeconds(120);

    class Candidate {
        String firstName;
        String lastName;
        String email;
        String experience;
        String profession;
        String phone;

        @Override
        public String toString() {
            return "Candidate: [firstName=" + firstName + ", lastName=" + lastName + ", email=" + email + ", experience="
                    + experience + ", profession=" + profession + ", phone=" + phone + "]";
        }
    }

    interface CandidateInfoCollector {

        @UserMessage("Extract information about a person from {{it}}")
        Candidate extractCandidateInfo(String text);

        @UserMessage("Extract all person names from {{it}}")
        List<String> extractPersonNames(String text);
    }

    public static void main(String[] args) {

        ChatLanguageModel model = OllamaChatModel.builder()
                .baseUrl(BASE_URL)
                .modelName(MODEL)
                .timeout(timeout)
                .build();

        CandidateInfoCollector candidateInfoCollector = AiServices.create(CandidateInfoCollector.class, model);

        String text = """
                I am Arjun Kumar, I have been working at a Software Developer for 5 years.
                Email: arjun@myemail.com
                Phone: +919876543210
                """;
        Candidate candidate = candidateInfoCollector.extractCandidateInfo(text);
        System.out.println(candidate);

        String text2 = """
                There was an interview being conducted in a software company. 
                Arjun and Ananya planned to attend the interview.
                Next morning they went to the venue.
                There they met their friends Akash, Mithun, Sita, Kausalya and Kumar who were also attending.
                The interviewers were Bob and Steve!
                """;

        // Person person = personExtractor.extractPersonFrom(text);
        List<String> candidates = candidateInfoCollector.extractPersonNames(text2);

        System.out.println(candidates);
    }
}
```
Here we define our POJO - `Candidate` and also override the `toString` method

```java
class Candidate {
    String firstName;
    String lastName;
    String email;
    String experience;
    String profession;
    String phone;
}
```
The interface `CandidateInfoCollector` defines the two methods annotated with the `UserMessage` instruction

```java
interface CandidateInfoCollector {

    @UserMessage("Extract information about a person from {{it}}")
    Candidate extractCandidateInfo(String text);

    @UserMessage("Extract all person names from {{it}}")
    List<String> extractPersonNames(String text);
}
```
Running the code extracts the POJO object and the list of names

```
Candidate: [firstName=Arjun, lastName=Kumar, email=arjun@myemail.com, experience=5 years, profession=Software Developer, phone=+919876543210]

[ Arjun, Ananya, Akash, Mithun, Sita, Kausalya, Kumar, Bob, Steve]
```

As we can see from the examples above, `AiServices` enables the developer to focus on the business logic by taking away the complexities of interacting with the LLM and handling the different data types transparently.

There are more to be explored in LangChain4j such as Tools (Function Calling), Retrieval Augmented Generation (RAG), etc. We will explore those in the upcoming articles.

The code examples are available on [GitHub repo][4].

Happy Coding!

[1]: https://blog.teamnexus.in/blog/2024/03/28/ai-ml-langchain4j-chat-memory/ "AI/ML - Langchain4j - Chat Memory"
[2]: https://docs.langchain4j.dev/ "LangChain4j"
[3]: https://python.langchain.com/ "Python Langchain"
[4]: https://github.com/rprabhu/ai-ml-langchain4j "GitHub Code Examples"