Build a Chatbot With OpenAI, Vercel AI and Xata
Tags: openai, vercel, tutorial, webdev
Published: 2023-10-13 16:33:43

In today's data-driven world, efficient interaction with databases is a crucial aspect of many applications. But what if we could go beyond conventional search methods and enable a natural language conversation with our databases?

At Xata, we aim to provide developers with the tools to build powerful applications that can interact with data in a natural way. Built-in with our core APIs and SDKs, we offer a powerful [ask endpoint](https://xata.io/docs/sdk/ask) that allows you to ask questions about your data and get the answers that matter most.

However, how would you integrate Xata with an existing application built with OpenAI? In this tutorial, we'll show you how to integrate OpenAI's [function calling](https://platform.openai.com/docs/guides/gpt/function-calling) feature with Xata's TypeScript SDK to create a chatbot that can use search to answer questions about your data.

In a [previous post](https://xata.io/blog/chatgpt-on-your-data), we introduced the concept of using our built-in [ask endpoint](https://xata.io/docs/sdk/ask) to simplify the process of querying your data.

## Prerequisites

Before we begin, make sure you have the following prerequisites:

- Existing project configured with the [Xata CLI](https://xata.io/docs/getting-started/installation)
- [Xata API key](https://app.xata.io/settings)
- [OpenAI API key](https://platform.openai.com/account/api-keys)

In your preferred serverless environment, make sure you install the [OpenAI API Library](https://github.com/openai/openai-node) and [Vercel AI library](https://github.com/vercel-labs/ai) to get started.

After ensuring your prerequisites are met, you can integrate Xata with your existing OpenAI application in three steps: Define a search function for AI, ask questions about your data, and run completions while streaming the results.

## Step 1 - Defining a search function

First, we'll define a function that allows us to search our database using OpenAI's [function calling](https://platform.openai.com/docs/guides/gpt/function-calling) feature.

```ts
const functions: CompletionCreateParams.Function[] = [
  {
    name: 'full_text_search',
    description: 'Full text search on a branch',
    parameters: {
      type: 'object',
      properties: {
        query: {
          type: 'string',
          description: 'The search query'
        }
      },
      required: ['query']
    }
  }
];
```

If we want to allow OpenAI to fine-tune the search results, we can add more options to the `parameters` object. For example, we can add a `fuzziness` parameter to allow for fuzzy search.

```ts
const functions: CompletionCreateParams.Function[] = [
  {
    name: 'full_text_search',
    description: 'Full text search on a branch',
    parameters: {
      type: 'object',
      properties: {
        query: {
          type: 'string',
          description: 'The search query'
        },
        fuzziness: {
          type: 'number',
          description: 'Maximum levenshtein distance for fuzzy search, minimum 0, maximum 2'
        }
      },
      required: ['query']
    }
  }
];
```

## Step 2 - Ask a question about your data

Now that we have our search function defined, we can use the [OpenAI](https://www.npmjs.com/package/openai) library to ask a question about our data.

```ts
const openai = new OpenAI({
  // Make sure to properly load and set your OpenAI API key here
  apiKey: process.env.OPENAI_API_KEY
});

const model = 'gpt-3.5-turbo';

const response = await openai.chat.completions.create({
  model,
  messages: [
    {
      role: 'user',
      content: question
    }
  ],
  functions
});
```

With the functions defined, the AI will be able to call the `full_text_search` function and pass the `query` parameter with the parts of the question that are relevant to the search.

To enhance the results, we can include additional information in the `messages` array as system messages. For instance, we can provide instructions to the AI or offer hints related to our database.

```ts
const { schema } = await api.branches.getBranchDetails({ workspace, region, database, branch });

const response = await openai.chat.completions.create({
  model,
  stream: true,
  messages: [
    {
      role: 'user',
      content: question
    },
    {
      role: 'system',
      content: `
        Workspace: ${workspace}
        Region: ${region}
        Database: ${database}
        Branch: ${branch}
        Schema: ${JSON.stringify(schema)}

        Reply to the user about the data in the database, do not reply about other topics.
        Only use the functions you have been provided with, and use them in the way they are documented.
      `
    }
  ],
  functions
});
```

OpenAI provides a variety of models, which you can find listed [here](https://platform.openai.com/docs/models/overview). If you require a different model that aligns more closely with your specific use case, you can easily switch the `model` parameter.

## Step 3 - Running the completion and streaming the results

Finally, we can run the completion and stream the results to the client.

```ts
const stream = OpenAIStream(response, {
  experimental_onFunctionCall: async ({ name, arguments: args }, createFunctionCallMessages) => {
    switch (name) {
      case 'full_text_search': {
        const response = await api.searchAndFilter.searchBranch({
          workspace,
          region,
          database,
          branch,
          query: args.query as string,
          fuzziness: args.fuzziness as number
        });

        const newMessages = createFunctionCallMessages(response);

        return openai.chat.completions.create({
          messages: [...messages, ...newMessages],
          stream: true,
          model,
          functions
        });
      }
      default:
        throw new Error('Unknown OpenAI function call name');
    }
  }
});

return new StreamingTextResponse(stream);
```

We have chosen to stream the results to the client, but you can also wait for the completion to finish and return the results as a single response.

### Bonus - Building an interactive chatbot UI

With React and the [`useChat` hook](https://sdk.vercel.ai/docs/api-reference/use-chat) you can easily create a chatbot that can answer questions about your data.

```ts
const { messages, append, reload, stop, isLoading } = useChat({
  // The route to the endpoint we have just created
  api: '/api/chat',
  initialMessages
});
```

### Conclusion

Congratulations! You've just built a powerful system that combines the capabilities of OpenAI, Vercel AI and Xata's database API. Users can now engage in natural language conversations with their databases, and OpenAI will utilize the provided functions to perform searches and retrieve relevant information.

By following this tutorial, you've learned how to integrate multiple APIs, handle requests and create interactive responses. This foundation can be extended to create even more sophisticated systems that enable seamless human-machine interactions with data.

If you want to learn more about Xata's database API, check out our [documentation](https://xata.io/docs) and come say hi on our [Discord](https://xata.io/discord) if you have any questions.

Happy coding! 🦋