How to build and deploy an AI Chatbot like ChatGPT without a credit card
Tags: webdev, devops, ai, nextjs
Published: 2023-07-18 18:21:52

## Overview

-   We will start from this [template](https://github.com/vercel-labs/ai-chatbot) provided by vercel which is built with [NextJs 13 App Directory](https://nextjs.org/docs/app) with server actions enabled and [Vercel AI SDK](https://sdk.vercel.ai/docs) for streaming chat UI.
-   Configure all necessary services step by step.
-   All the services has a free plan with no payment method requirement.
-   Implement configurable rate limiting to prevent abuse on your deployment.
-   We will use the vercel platform, but familiarity with vercel is not a prerequisite.
-   The code is available on [github](https://github.com/renzhamin/next-ai-chatbot)


## Table of Contents

- [Configuring services](#configuring-services)
  * [Model Provider](#model-provider)
  * [Auth](#auth)
  * [Database](#database)
  * [Upstash](#upstash)
- [Rate Limiting](#rate-limiting)
  * [Per Account](#per-account)
  * [Per IP](#per-ip)
- [Deployment](#deploy)
  * [Set up a new project on vercel](#set-up-a-new-project-on-vercel)
  * [Configure Github OAuth App](#configure-github-oauth-app)
- [Switch LLM](#switch-llm)



## Configuring services

### Model Provider

We will use hugging face for our model.

Why use hugging face instead of OpenAI ? Simply because it has a free plan and you don't need to attach any payment method.

1. Create an account on [hugging face](https://huggingface.co/join).
2. Go to account [settings](https://huggingface.co/settings/tokens) and create a token.
3. Save the token as `HUGGINGFACE_API_KEY` in the .env file.

### Auth

We will use github oauth for authentication via [NextAuth](https://next-auth.js.org/).

1.  Register a new oauth app on github from [here](https://github.com/settings/applications/new)
2.  Give a name for the app and fill the rest like this:<img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fcfemz8u2kpmbof0tldx.jpeg" width="500"/>
3.  Note that, this is for local development, for deployment register a new app and replace all instance of `http://localhost:3000` with the url of that deployment.
4.  Generate a new client secret.
5.  Copy Client ID as `AUTH_GITHUB_ID` and Client secret as `AUTH_GITHUB_SECRET` in the .env file.
6.  Go to this [link](https://generate-secret.vercel.app/32) to generate a secure random string and save it as `AUTH_SERCRET` in the .env file. This is needed for NextAuth to work in production.

### Database

For storing chats we will use vercel's KV database.

1. [Sign up](https://vercel.com/signup) or [login](https://vercel.com/login) in vercel.
2. If you signed up with email, connect your github account from [settings](https://vercel.com/account/login-connections).
3. Go to [storage](https://vercel.com/dashboard/stores) and Create a KV datbase. Note that you can only have one KV database on free plan.
4. Select the database you created and click on .env.local tab. Copy the contents to your .env file.

### Upstash

We will use upstash for rate limiting. if you don't need rate limiting, you can skip this step.

1. Go to [upstash console](https://console.upstash.com/).
2. Click on Create Database, chose your preferred region and enable the `Eviction` option.
3. In Details, go to `REST API` section and click on the .env tab.
4. Copy the values and paste it into your .env file.

## Rate Limiting

### Per Account

app/api/chat/route.ts:

```ts
const ratelimit = new Ratelimit({
  redis: redis,
  limiter: Ratelimit.slidingWindow(15, '1 d')
})

export async function POST(req: NextRequest) {
    const { userId } = auth()

    const { success, reset } = await ratelimit.limit(userId!)
    if (!success) {
        return new Response(
            `Your rate limit has been exceeded. You can chat again from ${new Date(
                reset
            ).toLocaleString()} GMT`
        )
    }
    .
    .
    .
}
```

This is the portion of code responsible for rate limiting. You can chose the values to suit your own needs. Here, I have set it to 15 requests per day. You can use 'm' for minutes, 's' for seconds. You can also modify the message shown when rate limit exceeded. The `reset` variable returns a unix timestamp in miliseconds of when the rate limit will be reset for this user which is converted to a string such as `7/18/2023, 6:00:00 AM`

### Per IP

You can rate limit per IP address as well. Instead of using userId, pass ip as the parameter to ratelimit.limit.

```ts
export async function POST(req: NextRequest) {
    const ip = req.ip ?? "127.0.0.1"
    const { success, reset } = await ratelimit.limit(ip)
    .
    .
    .
}
```

`req.ip` is undefined on localhost, thats why we are placing `127.0.0.1` as the fallback value.

If you don't want rate limiting, remove the imports of redis and RateLimit at the top of the file and delete all lines references those.

## Deployment

Before proceeding further, make sure that you have populated the values of all necessary environment variables in the [.env file](https://github.com/renzhamin/next-ai-chatbot/blob/main/.env.example). If you want to run localy, clone the [repo](https://github.com/renzhamin/next-ai-chatbot) and install packages with `pnpm install`. Then run the dev server with `pnpm dev`


### Set up a new project on vercel

1. Fork the [repo](https://github.com/renzhamin/next-ai-chatbot).
2. Create a new project on [vercel](https://vercel.com/new).
3. Import the forked repo.
4. Set `Environment Variables` according to your .env file. Note that you dont need to copy the key value pairs one by one, just copy the whole file, put the cursor in the Name field and Ctrl-v.
5. Click on deploy

But we are not done yet. We hit deploy so that a new project is created. Vercel assigns unique domains to deployments. You need use a fixed domain for github oauth to work.

### Configure Github OAuth App

1. Go to vercel [dashboard](https://vercel.com/dashboard). and select your project.
2. Click on the `Settings` tab and go to the `Domains` section.
3. Set the domain to your preference and copy the full url.
4. Register a new github oath app from [here](https://github.com/settings/applications/new).
5. Set Homapage URL as the domain you copied earlier.
6. Set Authorization callback URL as `your-domain-url/api/auth/callback`.
7. Now go to project Settings on vercel and then go to `Environment Variables` section.
8. Edit `AUTH_GITHUB_ID` and `AUTH_GITHUB_SECRET` and replace it with the value from the new github oauth app as before.
9. Ideally, you should use different `AUTH_SERCRET` in local and production environment for better security. Get a random string from [here](https://generate-secret.vercel.app/32) and edit `AUTH_SERCRET`.


## Switch LLM

You can easily switch models by changing the model parameter in `Hf.textGenerationStream()` on `/api/chat/rout.ts`. Hugging face provides access to a lot of [models](https://huggingface.co/models). You can use a model without any major code modification if that model supports streaming and has a small size. You have to modify `buildOpenAssistantPrompt` prompt method to format the prompts according to the model's specification. The model we are using takes user prompt as `
${prompt}
` and the previous replies of the model is expected in this format `
${content}
`. This format will vary depending on the model.

You can use LLM providers other than hugging face such as [Anthropic](https://anthropic.com), [Langchain](https://js.langchain.com/) adn OpenAI as well. Follow this [doc](https://sdk.vercel.ai/docs) to change the route handler(/api/chat/route.ts) and refer to the section of your chosen LLM provider. You can test and compare the models in [Vercel AI Playground](https://sdk.vercel.ai)


Find me on
💻 [Github](https://github.com/renzhamin)
🔘 [LinkedIn](https://www.linkedin.com/in/renzhamin)
⭕ [Twitter](https://twitter.com/renzhamin)