Otello: how to make Chat-GPT interact with your API
Tags: openai, api, javascript, chatgpt
Published: 2024-03-01 13:28:29

During my internship I had to build a chatbot capable of connecting to a private API. Building an LLM from scratch was out of question, so I made a little reseach and found out there is not much literacy about this task. 
The two most promising services were OpenAI API and SuperagentAI. 
I opted for the first one, because of a better documentation and greater support.

So here is how i built **Otello**:


## What is it
First of all, let me specify what we are talking about: 
Otello is a chatbot that enables the user  to make questions about the information present in our company Web app.
It gives the user an easy and natural way to get the data without the need of navigating trough the interface.


## How it's made
The chatbot is composed of three sequential components:
1. **`AskCompletion`** calls the OpenAI **Completion** to identify the resources requested by the user.
You need to list all the possible resources and configure a `function call` tool

```javascript
const completion = await openai.chat.completions.create({
        messages: [{ role: "system", content: context }, { role: "user", content: user_request }],
        model: "ft:gpt-3.5-turbo-0613:personal::XXXXXXXX",
        tools: [
            {
                "type": "function",
                "function": {
                    "name": "api_fetch",
                    "description": "fetches the API and returns the requested resource",
                    "parameters": {
                        "type": "object",
                        "properties": {
                            "resource": {
                                "type": "string",
                                "description": "requested resource name"
                            }
                        },
                        "required": [
                            "resources"
                        ]
                    }
                }
            }
        ],
        tool_choice: "auto",
    });
```
2. **`FetchAPI`**: makes a call to our API based on the resources identified, and retrive the data in JSON format

3. **`AskAssitant`**: send a request to a custom OpenAI **Assistant** with the user question and the data needed to aswer it.


### Why Completion? 
Because Completion is simple, fast and, with a small drawback, it allows you to use a fine tuned model: a base GPT model trained on your data, perfect to have more control on it. (yes, you need to create a dataset from scratch with a decent nubmer of chats to train the model)

```javascript
{"messages": [{"role": "user", "content": "User question 1"},{"role": "assistant", "function_call": {"name": "logica_fetch", "arguments": "{\"resource\": \"presenze_bloccate\"}"}}],"functions": [{"name": "logica_fetch","description": "fetches the API and returns the requested resource","parameters": {"type": "object","properties": {"resource": {"type": "string","description": "requested resource name"}},"required": ["resources"]}}]}
{"messages": [{"role": "user", "content": "User question 2"},{"role": "assistant", "function_call": {"name": "logica_fetch", "arguments": "{\"resource\": \"presenze_bloccate\"}"}}],"functions": [{"name": "logica_fetch","description": "fetches the API and returns the requested resource","parameters": {"type": "object","properties": {"resource": {"type": "string","description": "requested resource name"}},"required": ["resources"]}}]}
{"messages": [{"role": "user", "content": "User question 3"},{"role": "assistant", "function_call": {"name": "logica_fetch", "arguments": "{\"presenze_bloccate\": \"presenze\"}"}}],"functions": [{"name": "logica_fetch","description": "fetches the API and returns the requested resource","parameters": {"type": "object","properties": {"resource": {"type": "string","description": "requested resource namea"}},"required": ["resources"]}}]}
...
```
 
### But...
Ok, now that the architecture is clear, let's talk about the **main issue** with this approach: _how do you make an OpenAI GPT answer question based on your data_? or even better, _how do you upload the data it needs_?
I found two possible approaches:
- **Big model** - the easy way, just use a GPT which allows a large amount of input tokens (long questions) and add them to the user request. It is easy but definitly not cheap nor scalable for large quantities of data.
Some examples are: `gpt-3.5-turbo-16k, gpt-4-32k`

```javascript
const message = await openai.beta.threads.messages.create(thread_id, 
    {
        role: "user",
        content: user_request + ' The data you need are: ' + data,
    }
);
const run = await openai.beta.threads.runs.create(thread_id, { assistant_id });
```
- **File assistant** - the hard way, you can upload a file in a supported format (like .JSON) and tell the assistant to use its content in the answer. It is harder to implement and slower but far more cheap and supports up to 150 Gb files.

```javascript
let file = await openai.files.create({ file: output_files[0].file, purpose: "assistants" });
file = await openai.beta.assistants.files.create(assistantF_id, { file_id: file.id });
file_ids.push(file.id);
redirection = `the data you need ${output_files[0].label} are in the file: ${file.id}`;

const thread = await openai.beta.threads.create();

await openai.beta.threads.messages.create(
    thread.id,
        {
            role: "user",
            content: user_request + redirection,
            content: `${user_request}. ${redirection}`,
            file_ids
        }
    );
let run = await openai.beta.threads.runs.create(thread.id, { assistant_id: assistantF_id });
```

I built the backend with Node.js, but Python could have been an alternative too, and after a simple frontent in pure Javascript, it was time to deploy it.

## Deployment
The deployment of the backend had been in an AWS Lambda function, even if it is not suited well for projects like this, where is needed a high processing time.

## Conclusion
Even if the technologies are still quite new, building a personalized chatbot might be easier than you think, if you don't want anything too fancy.

If this caught your attention and you want to know more, you'll find the complete code with all the possible variations here: [github-repo](https://github.com/sinapsidev/test-openai)


Hope this helps