Build a chat room with custom bots powered by OpenAI/Gemini
Tags: go, ai, programming, tutorial
Published: 2024-06-14 07:49:09

![Chatty bots](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2ioggewpu5bi487moowp.jpg)

This app combines popular chat platforms like Discord/Slack, and the powerful minds of LLMs like OpenAI and Google Gemini.

It lets you create your own AI bots with unique personalities that can seamlessly engage in conversations with each other and with users.

**Demo app:** [Try the Demo version here](https://chatty.encore.dev)

In this guide we'll walk through how to run it locally, deploy to Encore's free dev cloud, integrate with your Slack/Discord, and how to develop the application further.

### TL;DR - What we'll build

* **Open Source:** Go based application built using [Encore](https://encore.dev) ([See Open Source Repo](https://github.com/encoredev/examples/edit/main/ai-chat/README.md))
* **Multi-platform:** Run locally or deploy to your cloud / AWS / GCP / Encore's free dev cloud
* **Multi-model:** Use your OpenAI or Gemini API key to try out each model
* **Integrations:** Add your favorite bots to your Discord or Slack

## 🏁 Getting started

### 💽 Install Encore

Install the Encore CLI to run your local environment:
- **macOS:** `brew install encoredev/tap/encore`
- **Linux:** `curl -L https://encore.dev/install.sh | bash`
- **Windows:** `iwr https://encore.dev/install.ps1 | iex`


### 👉 Create your app

Create your Encore app and clone the example:

```bash
encore app create my-ai-chat --example=ai-chat
```
(Feel free to replace `my-ai-chat` with a name that tickles your fancy)

### 🔐 Set Your LLM API Key

**OpenAI:**
To use OpenAI's models, you need an OpenAI API key. If you don't already have one, you can [get one here](https://platform.openai.com/api-keys).

Then use Encore's built-in secrets manager to securely store it:

```bash
cd my-ai-chat
encore secret set OpenAIKey --type dev,local,pr,prod
```
Paste your OpenAI API key when prompted.

**Gemini:**
Setting up Gemini is a longer process, see the instructions in the [appendix](#appendix-adding-gemini-credentials).

### 🕹 Run Your App Locally
To run your application locally, first make sure you have [Docker](https://docker.com) installed and run.

Then, with Encore, you only need to use a single command to start your entire system including multiple microservices, databases, pub/sub, and other infrastructure.

Run it from your app's root directory:
```bash
encore run
```
Encore will build and start your application, providing you with a local URL (e.g., <http://localhost:4000>). Open this URL in your browser to see your creation - the local chat interface!

## How it works

### Local Development Dashboard

Encore comes with a local development dashboard, when your app is running, open <http://localhost:9400> in your browser to access it.

It comes with tools like an API explorer, Service Catalog, architecture diagrams, and local tracing. All to help you test and understand the behavior of your application.

Go ahead and open it up now.

### Architecture

If you open the "Flow" architecture diagram in the local dev dashboard, you should see this: 

![Architecture diagram](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6rd9k0gbe4uuqit5jup2.png)

As you can see, AI Chat is a microservices-based application. 

Each service handling a specific aspect of the chatbot ecosystem. The services uses a combination of Encore APIs, pub/sub messaging, and WebSocket communication to orchestrate the flow of messages between chat platforms and LLM providers.

### Service Catalog

If you open the Service Catalog, you'll get an overview of all the services and endpoints in the application.

**Here are the key components:**
- Chat Service: The orchestrator service, routing messages between chat platforms and LLM providers.
- Discord Service: Handles the integration with the Discord API.
- Slack Service: Manages the art of conversation with the Slack API.
- Local Service: Provides a cozy web-based chat interface for testing and development.
- Bot Service: Responsible for creating, storing, and managing bot profiles.
- LLM Service: Formats prompts for LLMs, processes responses, and gracefully handles multiple LLM providers.
- OpenAI Service: Interfaces with OpenAI's API for chat completions and image generation.
- Gemini Service: Integrates with Google Gemini for even more chat completion options.

**Main Flow:**
- A user sends a message in a connected chat channel
- The corresponding chat integration (Discord, Slack, or local) receives the message
- The integration publishes a message to the chat service
- The chat service identifies any bots in the channel and fetches their profiles and the channel's message history
- The chat service sends the message to the LLM service
- The LLM service crafts a prompt including the bot's persona and the ongoing conversation
- The prompt is sent to the chosen LLM provider (OpenAI or Gemini)
- The LLM provider streams responses through pubsub back to the LLM service
- The LLM service parses the responses and relays them back to the chat service
- The chat service delivers the bot's witty (or not-so-witty) responses to the appropriate chat integration

### Deploy to the Cloud

Ready to share your bots with the world? Encore makes deploying to a free dev environment a breeze, you only need to use one command.

Run it from your app's root directory:
```bash
git push
```

Encore will automatically deploy your app and set up all the necessary infrastructure. You'll see a link to the deployment in Encore's [Cloud Dashboard](https://app.encore.dev).

![Deploy](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pufk5ppv4uhafkzvpjs7.png)

Once the deployment is complete, click Overview and copy the URL to see your bots in action on the web!

![Live deploy](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/i28n0cehfkstkmmd6i0a.png)

From the Cloud Dashboard you can also connect your own AWS/GCP accounts and deploy there, all with the same automated process.

## Add your bots to Discord/Slack

The application is designed to make it easy to integrate with any chat platform, and comes pre-configured to work with Discord and Slack.

### Integrate with Slack
To be able to use Slack as a chat platform, you'll need to create a Slack app and add it to your workspace. Here's how you can do it:

#### 1. **Create the Encore App:**

Complete the steps in the [Getting Started](#getting-started) section to create your Encore app.

#### 2. **Create a Slack App:**

* Visit [https://api.slack.com/apps](https://api.slack.com/apps) and click `Create New App`.
* Choose `From an app manifest` and click `Next`.
* Pick the workspace for your bot and click `Next`.
* Copy the [bot manifest](./chat/provider/slack/bot-manifest.json) and paste it into the text box.
* Replace the `<bot-domain>` placeholder with your cloud environment's url (e.g. `staging-my-chatbot-tur3.encr.app`) 
* Click `Next` and then `Create`.

#### 3. **Activate Bot Events:**

* On the bot settings page, click `Event Subscriptions`.
* Start the Encore app.
* If the `Request URL` is yellow, click on `Retry`.

#### 4. **Install the App to Your Workspace:**

* On the settings page, click `OAuth & Permissions` and then `Install to Workspace`.
* Select a channel for your bot and click `Allow`.

#### 5. **Add the Slack Bot Token:**

* Copy the `Bot User OAuth Token` from the `OAuth & Permissions` page.
* Add it as an Encore secret:
```bash
encore secret set SlackToken --type dev,local,pr,prod
```
#### 6. **Create Your Chat Bots**

Proceed to the [Create Your Chat Bots](#create-your-chat-bots) section to add bots to your channels.

### Integrate with Discord

To be able to use Discord as a chat platform, you'll need to create a Discord bot and add it to your server. Here's how you can do it:

#### 1. **Create the Encore App:**

Complete the steps in the [Getting Started](#getting-started) section to create your Encore app.

#### 2. **Create a Discord Bot:**

* Go to [Developer Portal Applications](https://discord.com/developers/applications) and click `New Application`.
* Give your Discord app a name and click `Create`.

#### 3. **Configure Install Settings:**

* Click `Installation`.
* Select `Discord Provided Link` in `Install Link`.
* Under `Default Install Settings`, add the `bot` scope and these permissions:
* Connect
* Manage Web Hooks
* Read Message History
* Read Messages/View Channels
* Send Messages

#### 4. **Grant Privileged Gateway Intents:**

* Click `Bot` and then `Privileged Gateway Intents`.
* Enable these intents:
* Server Members Intent
* Message Content Intent

#### 5. **Copy the Bot Token:**

* On the `Bot` page, click `Reset Token`.
* Copy the token and add it as an Encore secret:
```bash
encore secret set DiscordToken --type dev,local,pr,prod
```

#### 6. **Install the Bot:**

* Copy the Install Link and paste it into your browser.
* Grant your bot access to a server.

#### 7. **Invite the Bot to a Channel (Optional):**

If you want your bot to join private conversations, invite it to specific channels.

#### 8. **Create Your Chat Bots**

Proceed to the [Create Your Chat Bots](#create-your-chat-bots) section to add bots to your channels.

### Create Your Chat Bots
The Slack and Discord integrations does not come with a custom-made UI for adding bots to channels.

Until you've built your own UI (or maybe added support for slash commands?), you can use the API Explorer in Encore's Service Catalog to add bots to channels.

#### 1. **Open the Service Catalog**

Open your app in the [Cloud Dashboard](https://app.encore.dev) and click on `Service Catalog` in the main menu.

#### 2. **Create a Bot Profile:**

* Open the `bot` service and select the `bot.Create` endpoint.
* Give your bot a name, an engaging prompt, and enter `openai` as the LLM.
* Click `Call API`.
* Copy the bot ID in the response.

![Service catalog](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ige551n9jafpb2z4xgis.jpg)

#### 3. **Find a Chat Channel ID:**

* Now open the `chat` service and select the `chat.ListChannels` endpoint.
* Click `Call API`.
* Copy the `id` of the channel you want your bot to join.

![Channels](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pj24nuyvaixxihzxh3oo.jpg)


#### 4. **Add the Bot to a Channel:**

* Select the `chat.AddBotToChannel` endpoint.
* Enter the bot ID and the channel ID.
* Click `Call API`.

![Add bot to channel](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4g7yoqsbuxyrizg9p1ov.jpg)

#### 5. **Say hello to your Bot:**

**🎉 Success!**

Check your Slack/Discord channel; your bot should now be present and ready to chat!

![Slack bot](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/l2pwytodw1didts530oe.gif)

![Discord bot](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/euv5in3qflxkzcx8j5js.gif)

Now you can create bots that will make people laugh, think, or maybe even question the nature of reality (but no pressure!).

### Development: Make changes to the Chat Room UI
If you want to make changes to the app's built-in Chat Room interface you'll need to rebuild it before you deploy your changes.

#### 1. **Install npm***

If you don't have npm installed, you can download it from [https://www.npmjs.com/get-npm](https://www.npmjs.com/get-npm), or use your package manager, e.g.

```bash
brew install npm
```

#### 2. **Install Dependencies**

Navigate to the `chat/provider/local/static` directory and run:
```bash
npm install
```

#### 3. **Build the Interface**

Run the following command to build the web interface:
```bash
npm run build
```

#### 4. **Start the Interface**

Start the local chat service and open the interface in your browser:
```bash
encore run
```

#### 5. **Start the Interface in Dev Mode**
Alternatively, you can start the interface in development mode to get hot reloading and other goodies:
```bash
npm run dev
```
(You still need to start the local chat service with `encore run`.)

## Wrapping up

- ⭐️ Support the project by [starring Encore on GitHub](https://github.com/encoredev/encore).
- Learn more about building Go apps with Encore using these [Tutorials](https://encore.dev/docs/tutorials).👈
- Find inspiration on what to build with these Open Source [App Templates](https://encore.dev/templates).👈
- If you have questions or want to share your work, join the developers hangout in [Encore's community on Discord](https://encore.dev/discord).👈

## Appendix: Adding Gemini Credentials
To enable Gemini as an LLM provider, you'll need to set your Google Cloud credentials as an Encore secret. Here's how you can do it:
To enable Gemini as an LLM provider, you'll need to set your Google Cloud credentials as an Encore secret. Here's how you can do it:

#### 1. **Create a GCP Service Account:**

* Head over to the [Google Cloud Console](https://console.cloud.google.com/iam-admin/serviceaccounts).
* Click `Create Service Account` and give your new account a name and description.
* Grant your service account the `Vertex AI User` role.
* Click `Done`.

#### 2. **Create a JSON Key:**

* Click on your newly created service account and then on `Keys` -> `Add Key` -> `Create New`.
* Choose the `JSON` format and click `Create`.
* Download the JSON file.

####3. **Add JSON Key as an Encore Secret:**

```bash
encore secret set --type dev,local,pr,prod GeminiJSONCredentials < <downloaded json>.json
```