Managing AI-powered Java App With API Management
Tags: webdev, chatgpt, ai, java
Published: 2023-06-23 07:06:58

In this article, we will **explore how to integrate** [OpenAI's ChatGPT API](https://platform.openai.com/docs/api-reference/introduction)s with a [Spring Boot](https://spring.io/projects/spring-boot) application and manage the APIs using [Apache APISIX](https://apisix.apache.org/), an **open-source API gateway**. This integration will allow us to leverage the power of ChatGPT, a state-of-the-art language model developed by [OpenAI](https://openai.com/), in our Spring Boot application, while APISIX will provide a robust, scalable, and secure way to manage the APIs.

## OpenAI ChatGPT APIs

[OpenAI's ChatGPT API](https://platform.openai.com/docs/api-reference/introduction) is a powerful tool that we can use to **integrate the capabilities of the ChatGPT model** into our own applications, or services. The API allows us to send a series of messages and receive an AI model-generated message in response via REST. It offers a bunch of APIs to create text responses in a chatbot, code completion, generate images, or answer questions in a conversational interface. In this tutorial, we will use [chat completion API](https://platform.openai.com/docs/api-reference/chat/create) to generate responses to a prompt (basically we can ask anything). Before starting with the tutorial, you can explore the API to have an understanding of how to [authenticate to the API](https://platform.openai.com/docs/api-reference/authentication) using API keys, how API request parameters and response look like.

A sample cURL request to [chat completion API](https://platform.openai.com/docs/api-reference/chat/create)  would look like this. You replace `OPENAI_API_KEY`  with your own [API key](https://platform.openai.com/account/api-keys) and place it in the *Authorization* header while calling the API.

```bash
curl https://api.openai.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [{"role": "user", "content": "What is Apache APISIX?"}]
}'
```

Here's a sample JSON response:

```json
{
  "id": "chatcmpl-7PtycrYOTJGv4jw8FQPD7LCCw0tOE",
  "object": "chat.completion",
  "created": 1686407730,
  "model": "gpt-3.5-turbo-0301",
  "usage": {
    "prompt_tokens": 15,
    "completion_tokens": 104,
    "total_tokens": 119
  },
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "Apache APISIX is a dynamic, real-time, high-performance API gateway designed to facilitate the management and routing of microservices and APIs. It provides features such as load balancing, rate limiting, authentication, authorization, and traffic control, all of which help to simplify the management of microservices and APIs. Apache APISIX is built on top of the Nginx server and can support high levels of traffic with low latency and high availability. It is open source and released under the Apache 2.0 license."
      },
      "finish_reason": "stop",
      "index": 0
    }
  ]
}
```

## Project code example

The tutorial consists of two parts. The first part covers the setting up Spring Boot application and creating a new API endpoint that can handle our API calls to [chat completion API](https://platform.openai.com/docs/api-reference/chat/create) programmatically. In the second part, we will introduce APISIX features such as security, and traffic control to Spring Boot API. The full code examples for this tutorial are available on the GitHub repository called [apisix-java-chatgpt-openaiapi](https://github.com/Boburmirzo/apisix-java-chatgpt-openaiapi).

### Prerequisites

Before we start, make sure you have the following:

- Create an [OpenAI API Key](https://platform.openai.com/docs/api-reference/authentication): To access the OpenAI API, you will need to create an API Key. You can do this by logging into the OpenAI website and navigating to the API Key management page.
- [Docker](https://www.docker.com/products/docker-desktop/) is installed on your machine to run APISIX and Spring Boot.

### Step 1: Setting up your Spring Boot application

First, we need to set up a new Spring Boot application. You can use [Spring Initializr](https://start.spring.io/) to generate a new Maven project with the necessary dependencies. For this tutorial, we will need the [Spring Boot Starter Web](https://mvnrepository.com/artifact/org.springframework.boot/spring-boot-starter-web) dependency. To integrate the ChatGPT API, we will use the OpenAI Java client. There is an open-source [community Java library](https://github.com/TheoKanning/openai-java). It provides service classes that create and calls the OpenAI's GPT APIs client in Java. Of course, you can write your own implementation in Spring that interacts with OpenAI APIs too. See other [client libraries](https://platform.openai.com/docs/libraries) for different programming languages. 

```xml
<dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>com.theokanning.openai-gpt3-java</groupId>
            <artifactId>service</artifactId>
            <version>0.12.0</version>
        </dependency>
</dependencies>
```

### Step 2: Create a Controller class

In the [ChatCompletionController.java](https://github.com/Boburmirzo/apisix-java-chatgpt-openaiapi/blob/main/openaiapi/src/main/java/com/company/project/controllers/ChatCompletionController.java) class, you can use the OpenAI service to send a request to the ChatGPT API. 

```java
import com.theokanning.openai.completion.chat.ChatCompletionChoice;
import com.theokanning.openai.completion.chat.ChatCompletionRequest;
import com.theokanning.openai.completion.chat.ChatMessage;
import com.theokanning.openai.completion.chat.ChatMessageRole;
import com.theokanning.openai.service.OpenAiService;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RestController;

import java.util.ArrayList;
import java.util.List;

@RestController
public class ChatCompletionController {

    @Value("${openai.model}")
    private String model;

    @Value("${openai.api.key}")
    private String openaiApiKey;

    @PostMapping("/ai-chat")
    public String chat(@RequestBody String prompt) {
        OpenAiService service = new OpenAiService(openaiApiKey);

        final List<ChatMessage> messages = new ArrayList<>();
        final ChatMessage systemMessage = new ChatMessage(
						 ChatMessageRole.USER.value(), prompt);
        messages.add(systemMessage);

        ChatCompletionRequest chatCompletionRequest = ChatCompletionRequest
            .builder()
            .model(model)
            .messages(messages)
            .maxTokens(250)
            .build();

        List<ChatCompletionChoice> choices = service
						.createChatCompletion(chatCompletionRequest).getChoices();

        if (choices == null || choices.isEmpty()) {
            return "No response";
        }

        return choices.get(0).getMessage().getContent();
    }
}
```

Chat API endpoint `/ai-chat` handles POST requests creates a chat request, and sends it to the OpenAI API. Then, it returns the first message from the API response.

### Step 3: Define application properties

Next, we provide the properties for the API like `model` and `API key` in the *[application.properties](https://github.com/Boburmirzo/apisix-java-chatgpt-openaiapi/blob/main/openaiapi/src/main/resources/application.properties)* file:

```yaml
openai.model=gpt-3.5-turbo
openai.api.key=YOUR_OPENAI_API_TOKEN
```

### Step 4: Run the Spring Boot app

We can now run the [Application.java](https://github.com/Boburmirzo/apisix-java-chatgpt-openaiapi/blob/main/openaiapi/src/main/java/com/company/project/Application.java) and test it with Postman or the cURL command.

![AI response from the Spring Boot](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6wmykn27qfj7dca0garv.png)

As we can see, the application generated a response to our question in the prompt request body.

### Step 5: Create a Dockerfile

We use a Docker container to wrap our Spring Boot application and use it together with other APISIX containers in [docker-compose.yml](https://github.com/Boburmirzo/apisix-java-chatgpt-openaiapi/blob/main/docker-compose.yml). To do so, we can create a [Dockerfile](https://github.com/Boburmirzo/apisix-java-chatgpt-openaiapi/blob/main/openaiapi/Dockerfile) to build a JAR and execute it. See how to [dockerize Spring Boot app](https://spring.io/guides/topicals/spring-boot-docker/) tutorial. Then, register the service in docker compose yaml file.

```yaml
openaiapi:
    build: openaiapi
    ports:
      - "8080:8080"
    networks:
      apisix:
```

### Step 6: Setting up Apache APISIX

To set up APISIX, we can simply run `docker compose up` command. Because we have already defined all necessary services in [docker-compose.yml](https://github.com/Boburmirzo/apisix-java-chatgpt-openaiapi/blob/main/docker-compose.yml). This file defines only 2 containers one for APISIX, and another for the Spring boot application we implemented in the previous steps. In this sample project, we run APISIX in [standalone mode](https://apisix.apache.org/docs/apisix/2.15/stand-alone/). There are other APISIX [installation options](https://apisix.apache.org/docs/apisix/installation-guide/) and [deployment modes](https://apisix.apache.org/docs/apisix/deployment-modes/) as well. Now APISIX as a separate service is running on `localhost:9080` and Spring Boot app on **`localhost:8080`**

### Step 7: Securing the API with APISIX

Once APISIX is set up, we can add security features to our existing Spring boot API `/ai-chat` so that only allowed API consumers can access this API. APISIX provides several plugins to secure your APIs. For example, you can use the [jwt-auth](https://apisix.apache.org/docs/apisix/plugins/jwt-auth/) plugin to require a JWT token for all requests. Here is an example of how to add a route with an upstream and plugins using the `apisix.yml` file:

```yaml
upstreams:
  - id: 1
    type: roundrobin
    nodes:
      "openaiapi:8080": 1
routes:
  - uri: /ask-me-anything
    upstream_id: 1
    plugins:
      proxy-rewrite:
        uri: /ai-chat
      jwt-auth: {}
  - uri: /login
    plugins:
      public-api:
        uri: /apisix/plugin/jwt/sign
consumers:
  - username: appsmithuser
    plugins:
        jwt-auth:
            key: appsmithuser@gmail.com
            secret: my-secret-key
```

> After we specify upstreams, routes, and consumer objects and routing rules in the APISIX config, the config file is loaded into memory immediately after the APISIX node service starts in Docker. Apisix periodically tries to detect whether the file content is updated, if there is an update, it **reloads automatically changes**.

With this configuration, we added one upstream, two routes and one consumer object. In the first route, all requests to `/ask-me-anything` (which is a custom URI path, you can define any URI there) must include the `Authorization: JWT_TOKEN` in the header. Then, APISIX rewrites the custom URI path to the actual API `/ai-chat` automatically with the help of [proxy-rewrite](https://apisix.apache.org/docs/apisix/plugins/proxy-rewrite/) plugin and forwards requests to the Spring Boot application running on `localhost:8080`.

If you try to request the APISIX route, it will reject our requests by returning an authorized error:

```bash
curl -i http://localhost:9080/ask-me-anything -X POST -d '
{
   "prompt":"What is Apache APISIX?"
}'
```

The result of the above request:

```bash
HTTP/1.1 401 Unauthorized
{"message":"Missing JWT token in request"}
```

In the second route config, we enabled [public-api](https://apisix.apache.org/docs/apisix/plugins/public-api/) plugin to expose a new endpoint `/login` to sign new JWT tokens. Because APISIX can act as an identity provider to generate and validate a new token from the API consumer or client apps. See **Step 8**, how we claim a new token for our API consumer.

```bash
  - uri: /login
    plugins:
      public-api:
        uri: /apisix/plugin/jwt/sign
```

If you noticed in the same config file, we registered an API consumer to use `/ask-me-anything` AI-powered API and our users can claim APISIX using their secret to generate a JWT token to access the API:

```bash
consumers:
  - username: appsmithuser
    plugins:
        jwt-auth:
            key: appsmithuser@gmail.com
            secret: my-secret-key
```

### Step 8: Claim a new JWT token

Now we can claim a new JWT token for our existing API consumer with the key:

```bash
curl -i http://127.0.0.1:9080/login?key=user-key -i
```

We will get the new token as a response from APISIX:

```bash
Server: APISIX/3.0.0
eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJrZXkiOiJ1c2VyLWtleSIsImV4cCI6MTY4NjU5MjE0NH0.4Kn9c2DBYKthyUx824Ah97-z0Eu2Ul9WGO2WB3IfURA
```

### Step 9: Request the API with the JWT token

Finally, we can send a request to the API `/ask-me-anything` with the JWT token in the header we obtained in the previous step.

```bash
curl -i http://localhost:9080/ask-me-anything -H 'Authorization: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJrZXkiOiJ1c2VyLWtleSIsImV4cCI6MTY4NjU5Mjk4N30.lhom9db3XMkcVd86ScpM6s4eP1_YzR-tfmXPckszsYo' -X POST -d '
{
   "prompt":"What is Apache APISIX?"
}'
```

Or using Postman, we will get AI response but this time response comes through APISIX Gateway:

![AI response from APISIX](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6bu426xjmowx6jnf7e80.png)

## Conclusion

In this tutorial, we explored the OpenAI ChatGPT API to generate responses to prompts. We created a Spring Boot application that calls the API to generate responses to prompts. Next, you can introduce [additional features](https://github.com/Boburmirzo/apisix-java-chatgpt-openaiapi/tree/main/additional-features) to your integration by updating the existing [apisix.yml](https://github.com/Boburmirzo/apisix-java-chatgpt-openaiapi/blob/with-frontend/apisix/apisix.yml) file. Also, you check out the branch name called [with-frontend](https://github.com/Boburmirzo/apisix-java-chatgpt-openaiapi/tree/with-frontend) and run the project to see the UI interface built using [Appsmith](https://www.appsmith.com/) that works with APISIX.

### Related resources

- [OpenAI API Java library](https://github.com/TheoKanning/openai-java)
- [OpenAI Chat Completion API](https://platform.openai.com/docs/api-reference/chat/create)

### Recommended content

- [Powering AI Capabilities with Apache APISIX and OpenAI API](https://api7.ai/blog/power-ai-capabilities-with-apache-apisix)
- [Why Is Apache APISIX the Best API Gateway?](https://api7.ai/blog/why-is-apache-apisix-the-best-api-gateway)

### Community

🙋 [Join the Apache APISIX Community](https://apisix.apache.org/docs/general/join/)
🐦 [Follow us on Twitter](https://twitter.com/ApacheAPISIX)
📝 [Find us on Slack](https://join.slack.com/t/the-asf/shared_invite/zt-vlfbf7ch-HkbNHiU_uDlcH_RvaHv9gQ)
💁 [How to contribute page](https://apisix.apache.org/docs/general/how-to-contribute/)

### About the author

Visit my blog: [www.iambobur.com](https://www.iambobur.com/)