Build a LLM Chatbot with a Custom Knowledge Base
Published: 2024-02-01 21:01:30

With all the attention that these LLMs and ChatGPT have been receiving, a new principle emerges “Garbage in, garbage out.” When using GPT-4 or GPT-3.5, or any other large language model, this concept applies to two places [Prompt Engineering](https://en.wikipedia.org/wiki/Prompt_engineering#:~:text=Prompt%20engineering%20(also%20known%20as,of%20it%20being%20explicitly%20given.) and [Fine-tuning](https://en.wikipedia.org/wiki/Fine-tuning_(deep_learning)). We should also consider when these models were trained to verify how old or new the data they provide us is.

Most businesses today sit within corporate data sources – inside a firewall or outside and not in a public domain internet. If we can leverage LLMs on this data, then new possibilities emerge.

This article will discuss building a chatbot with a custom knowledge base in 30 minutes to have your AI answer questions about your data.

![](/cdn/3prze68gbwl1/3qYPBDJSpSd5oecj2fUDae/f2a980695856b249c645e7ea774b87fe/Screenshot_2023-07-20_at_10.17.38_PM.png)

Pre-Requisites for Creating a Knowledge Bot
-------------------------------------------

We must consider how much data we have to create a custom knowledge base. In this blog, we will create an assistant using PubNub documentation. There is a lot of documentation, so we should consider using a production service such as [Vectara](https://vectara.com/), [Pinecone](https://www.pinecone.io/), or [Weaviate](https://weaviate.io/) to manage our vector embeddings. You can use LangChain to create a local Vector Database for smaller amounts of data. LangChain also supports mapping your custom embeddings to your LLM models through a semantic or similarity search.

For this blog, we will go through setting up a Vector Database on Vectara, as it provides a simple drag-and-drop solution for all of your company data. The best way to interact with Vectara is to utilize [PubNub Functions](https://www.pubnub.com/docs/general/serverless/functions/overview#:~:text=PubNub%20provides%20Functions%20to%20capture,filter%2C%20and%20even%20aggregate%20data.), which provides a serverless JavaScript container that runs whenever a pre-defined event occurs. You will have customizability over when the function runs and gets fired, being able to adjust how your AI Knowledge Bot operates.

Vectara will use your dataset to index your data into multiple embeddings. When you pass a text input or user input Vectara will run a Semantic Search on your data and summarize the results it has found, providing an answer to your question using your custom data. This will give you the relevant information needed. Vectara supports multiple file types such as TXT, HTML, PDF, and Word Files. Gather all the documents you want to upload and add them to an individual folder.

### How to set up Vectara

Set up Vectara is as follows:

1.  [Sign up](https://console.vectara.com/login) or log in to Vectara
    
2.  Once you are on the dashboard, click `Create corpus`
    
3.  Once you give the Corpus a name and a description under the `Data Ingestion` header drag and drop the files you want your LLM to know about into the `Upload Files` section. After your files are uploaded, you can check your corpus ID at the top of the webpage as you will need it for the request we are about the write.
    
4.  Click on your email in the top right corner and save your `Customer ID` for later
    
5.  Select `API Keys` and create an API key for your corpus by selecting your corpus in the drop-down menu. Save your API key for later.
    

### High-Level Architecture

The architecture will be structured as follows:

1.  The chat application will use PubNub to send and receive messages
    
2.  A PubNub Function will listen to these messages on a specific channel
    
3.  A PubNub signal will be fired to let the user know when the AI is thinking and when it is done.
    
4.  The message will then be forwarded to Vectara using the Vectara Rest API
    
5.  The PubNub Function will then parse 1 of many results out of the response from Vectara 
    
6.  The response will then be published on a channel associated with your chatbot
    

Configuring your PubNub Function
--------------------------------

1.  Navigate to the [admin dashboard](https://admin.pubnub.com/)
    
2.  Select `functions` on the left-hand menu and click on the appropriate key set you would like to use
    
3.  Select `+ Create Module` and enter a module name and description
    
4.  Select the module you just created and click `+ Create a Function`
    
5.  Give the Function a name, such as `Vectara Query` and select `After Publish or Fire` in the drop-down menu. This function will fire after the message has been published to the relevant channel, in this case, `docs-pubnub-ai` 
    
6.  Set the channel name to `docs-pubnub-ai`
    
7.  Click on `My Secrets` and create a secret called `VECTARA_API_KEY` and `CUSTOMER_ID`
    

Here is the code for querying the Vectara Database from a PubNub Function
-------------------------------------------------------------------------

The code snippet for the PubNub Function is defined as follows:

```js
const Promise = require('promise');

//
// API Key For OpenAI
// **Add your Vectara API Key and Customer ID to MY SECRETS (Left Panel)**
//
let VECTARA_API_KEY = null;
let CUSTOMER_ID = null;
let channel = "docs-pubnub-ai-response";
function getVectaraKey() {
   // Use cached key
   if (VECTARA_API_KEY && CUSTOMER_ID) {
       return new Promise(resolve => resolve(VECTARA_API_KEY));
   }
   // Fetch key from vault
   return vault.get("VECTARA_API_KEY").then(apikey => {
       return vault.get("CUSTOMER_ID").then(customer_id => {
            VECTARA_API_KEY = apikey;
            CUSTOMER_ID = customer_id;
           return new Promise(resolve => resolve(VECTARA_API_KEY));
       });
   });
}

//
// Import Modules
//
const xhr = require('xhr');
const vault = require('vault');
const pubnub = require('pubnub');

//
// Main
//
export default (request) => {
   // Input data for Vectara
   let message = request.message.text;
   let signalMessage = { message: { type: "typing_on" }, channel: channel };
   return pubnub.signal(signalMessage).then((_) => {
       return getVectaraKey().then(_ => {
           return vectara(message).then(response => {
               return pubnub.publish({
                   channel: channel,
                   message: {
                       id: "PubNubAI",
                       type: "default",
                       text: response
                   },
               }).then((publishResponse) => {
                   console.log(publishResponse);
                   message = { message: { type: "typing_off" }, channel: channel };
                   return pubnub.signal(message).then((signalOffResponse) => {
                       console.log(signalOffResponse);
                       return request.ok();
                   });
               }).catch((err) => {
                   console.error(err);
               });
           });
       });
   });
};

//
// API Call to Vectara asking the AI a question
//
async function vectara(question) {
   console.log("SENDING REQUEST TO VECTARA");
   const url = "https://api.vectara.io/v1/query";
   const body = JSON.stringify({
       "query": [
           {
           "query": question,
           "start": 0,
           "numResults": 10,
           "contextConfig": {
               "charsBefore": 0,
               "charsAfter": 0,
               "sentencesBefore": 2,
               "sentencesAfter": 2,
               "startTag": "<b>",
               "endTag": "</b>"
           },
           "corpusKey": [
               {
               "customerId": CUSTOMER_ID,
               "corpusId": 1,
               "semantics": "DEFAULT",
               "dim": [
                   {
                   "name": "string",
                   "weight": 0
                   }
               ],
               "lexicalInterpolationConfig": {
                   "lambda": 0.025
               }
               }
           ],
           "rerankingConfig": {
               "rerankerId": 272725717
           },
           "summary": [
               {
               "summarizerPromptName": "",
               "maxSummarizedResults": 5,
               "responseLang": "eng",
               "summarizerPromptId": 1,
               "debug": false,
               "responseChars": 0
               }
           ]
           }
       ]
   });

   const http_options = {
       'method': 'POST',
       'timeout': 20000,
       'headers': {
           'Content-Type': 'application/json',
           'customer-id': CUSTOMER_ID,
           'x-api-key': VECTARA_API_KEY
        },
       'body': body
   };

   xhr.timeout = 100000;

   // Send API Request to OpenAI GPT Model
   const response = await xhr.fetch(url, http_options).then(res => res.json());

   // Parse the response and get the summary
   const summary = response.responseSet[0].summary[0].text;

   return summary;
}
```

Connecting the PubNub Function to your UI
-----------------------------------------

To connect the PubNub function to a UI following code defined above using one of the many [SDKs](https://www.pubnub.com/docs/sdks) that PubNub provides. [Publish/subscribe](https://www.pubnub.com/docs/sdks/javascript/api-reference/publish-and-subscribe) to the channel `pubnub-docs-ai` and wait for the Vectara query to finish running after utilizing the PubNub Function above. Connecting a [Typing Indicator](https://www.pubnub.com/docs/chat/sdks/messages/typing) to listen for PubNub signals on the channel `pubnub-docs-ai` will allow the user to see when the PubNub Knowledge bot is thinking, adding a smoother end-user experience.

The code for connecting the PubNub function in React:

```js
const pubnub = new PubNub({
   publishKey: process.env.NEXT_PUBLIC_PUBLISH_KEY!,
   subscribeKey: process.env.NEXT_PUBLIC_SUBSCRIBE_KEY!,
   userId: "You",
 });

 const publishChannel = "docs-pubnub-ai";

 const sendMessage = (message: string) => {
   if (message) {
     pubnub.publish({
         channel: publishChannel, message: {
           text: message
       }});
   }
 };
```

In Summary
----------

Using PubNub Functions along with any Vector Database or Vector Store is a very quick and production-ready way to create your own AI Knowledge Bot. Not only could you utilize Vectara this way but also Pinecone, Weaviate or any other production Vector Database. With PubNub Functions it is easy to host and control how the message is being sent and when it is sent to enhance your Vector Databases functionality. 

Sign up for our [admin dashboard](https://admin.pubnub.com/#/register) to start configuring your PubNub keyset. Also, check the number of [tutorials](https://www.pubnub.com/tutorials/) and [blogs](https://www.pubnub.com/blog/) we have for your specific [use case](https://www.pubnub.com/use-case/).

How can PubNub help you?
========================

This article was originally published on [PubNub.com](https://www.pubnub.com/blog/build-chatbot-with-custom-knowledge-base/)

Our platform helps developers build, deliver, and manage real-time interactivity for web apps, mobile apps, and IoT devices.

The foundation of our platform is the industry's largest and most scalable real-time edge messaging network. With over 15 points-of-presence worldwide supporting 800 million monthly active users, and 99.999% reliability, you'll never have to worry about outages, concurrency limits, or any latency issues caused by traffic spikes.

Experience PubNub
-----------------

Check out [Live Tour](https://www.pubnub.com/tour/introduction/) to understand the essential concepts behind every PubNub-powered app in less than 5 minutes

Get Setup
---------

Sign up for a [PubNub account](https://admin.pubnub.com/signup/) for immediate access to PubNub keys for free

Get Started
-----------

The [PubNub docs](https://www.pubnub.com/docs) will get you up and running, regardless of your use case or [SDK](https://www.pubnub.com/docs)