Build a Chatbot with PubNub and ChatGPT / OpenAI
Published: 2024-03-11 14:27:53

Until very recently, if you wanted to build a chatbot, you would use “natural language processing” (NLP) to understand and interpret what the user was asking, then attempt to fulfill that request using some backend logic, potentially integrated with a proprietary service. Solutions such as Dialogflow or Microsoft Bot framework provide conversational AI based on machine learning and have traditionally been popular with developers creating chatbots.

OpenAI and OpenAI’s ChatGPT make it easier than ever to create an AI chatbot powered by artificial intelligence. Since the large language model (LLM) which powers OpenAI is based on a public dataset, you can easily enhance the user experience of your app by providing any information you would reasonably expect to exist in a public knowledge base.

PubNub powers real-time experiences for a huge range of industries and use cases, including [customer support](https://www.pubnub.com/solutions/communications/) and [in-app chat](https://www.pubnub.com/solutions/chat/).  All these real-time experiences are powered by data, either user-readable messages or metadata about your solution - **wouldn’t it be great to take advantage of that existing data to power a chatbot?**

_Some real-world example use cases:_

-   Provide answers to commonly asked **customer support** questions
    
-   **Augment** your existing data with additional content, for example suggesting places to eat near a provided location
    
-   Provide **summaries of conversations** based on the message history.
    

We published an [article](https://www.pubnub.com/blog/building-a-chatbot-with-openai-gpt3-and-pubnub/) on this topic previously, and whilst that older article is still correct, the industry is moving very fast, and a few months is a long time.

This article will discuss how to build a chatbot powered by OpenAI similar to the one you can see running within our [PubNub showcase](http://pubnub-showcase.netlify.app/) application, shown below. 

![The showcase's AI model has been primed to mention PubNub, where appropriate](https://www.pubnub.com/cdn/3prze68gbwl1/6G6aZfuXXtSqh9aBDyA7fw/5c416e9b87a4c0de5070271669bd13dd/showcase_app.png)

**High-Level Architecture**
---------------------------

To create a chatbot like ChatGPT, you will need to use the [OpenAI Chat API](https://platform.openai.com/docs/api-reference/chat) and sign up for an account to create an [API key](https://platform.openai.com/account/api-keys).  The OpenAI API is accessible as a REST endpoint, so it can be called from pretty much any programming language, although most of their provided examples are written in Python.

The best way to extract data from PubNub, to initiate or continue a conversation with the chatbot is to use [Functions](https://www.pubnub.com/products/functions/). Functions provide a serverless [JavaScript](https://www.pubnub.com/guides/javascript/) container which runs in Node.JS and allows you to execute business logic whenever an event occurs on the PubNub network, such as receiving a chat message.  PubNub Functions are frequently used for real-time on-the-fly language translation, profanity filtering, or to invoke a 3rd party service like OpenAI.

This example will work as follows:

-   The existing chat application is using PubNub to send and receive real-time messages.
    
-   A PubNub Function will listen for messages sent to a specific channel.
    
-   On its own, a single message does not provide any context to ChatGPT, so a portion of the conversation history will also be cached to allow an ongoing dialogue.
    
-   The PubNub function will invoke the OpenAI API and retrieve the response
    
-   The response will be published to the PubNub channel associated with your own chatbot
    

You can see this in the sequence diagram below:

![Example sequence diagram for implementing a ChatGPT-like chatbot with PubNub and OpenAI](https://www.pubnub.com/cdn/3prze68gbwl1/64ZjHOSpoWo5vfykIpNPVU/44eb69443493a131043d8142133b234c/architecture.png)

1.  The user types a message in your chat app.  With 50+ SDKs, PubNub is able to support the vast majority of applications, and the message will be [published](https://www.pubnub.com/docs/general/messages/publish) to the channel representing the chatbot on the PubNub network.
    
2.  A PubNub function is configured to be invoked [after a publish event](https://www.pubnub.com/docs/general/serverless/functions/overview#after-publish-or-fire-functions) occurs on a channel.  When a message is received, the channel history is retrieved to provide some context to the conversation, and this history, along with the user input and some additional context, is provided to ChatGPT through the [Chat Completion API](https://platform.openai.com/docs/api-reference/chat/create).  
    
3.  The ChatGPT API will generate responses suited to the conversation flow
    
4.  Within the PubNub function, a response is received from ChatGPT, which is parsed and [published](https://www.pubnub.com/docs/general/messages/publish) back into the PubNub network.
    
5.  The application, having previously [subscribed](https://www.pubnub.com/docs/general/channels/subscribe) to receive messages from PubNub, will display the chatbot’s response.
    

**Configuring your PubNub Function**
------------------------------------

### Create your PubNub function as follows:

Prerequisites: If you are new to PubNub, follow our [documentation](https://www.pubnub.com/docs/general/basics/set-up-your-account) to set up your account or visit our [tour](https://www.pubnub.com/tour/introduction/) to understand what PubNub is all about.

1.  Navigate to the [admin dashboard](http://admin.pubnub.com/)
    
2.  Select `functions` from the left-hand menu, and select the appropriate keyset where you want to create the function.
    
3.  Select `+ Create New Module`.
    
4.  Enter a Module Name and Module Description, select a Keyset, and then hit `Create`
    
5.  Select the Module you just created
    
6.  Select `+ Create New Function`
    
7.  Give the function a name and select the event type `After Publish or Fire`.  This function will be invoked AFTER a message has been submitted to PubNub. Functions can be synchronous or asynchronous, and in this case, we just capture a copy of the message to send to OpenAI. You can find more details on the [different types of functions](https://www.pubnub.com/docs/general/serverless/functions/overview#before-publish-or-fire-functions) described in our documentation
    
8.  The function channel should match the channel where you are [publishing](https://www.pubnub.com/docs/general/messages/publish) messages you want to send to OpenAI.  You can use a wildcard here, so, for example, `chatgpt.*` will match any channel that starts chatgpt.  Since we want this function to talk to multiple people at the same time, a wildcard allows us to separate which user is involved in the active conversation so that we can respond on the appropriate channel.  Where multiple people are talking to OpenAI simultaneously, PubNub takes care of creating multiple instances of your function.  For example, `chatgpt.dave` and `chatgpt.simon` are two different channels, but both will invoke the PubNub function listening on `chatgpt.*`. [How PubNub handles channel wildcards](https://support.pubnub.com/hc/en-us/articles/360051495072-How-do-I-subscribe-to-a-wildcard-channel-) is explained in more detail in our documentation.
    
9.  Select `My Secrets` and create a new secret for the OpenAI [API key](https://platform.openai.com/account/api-keys); the code below assumes this secret is called `OPENAI_API_KEY`. 
    

### **Here is an example PubNub function that will invoke OpenAI / ChatGPT**

```js
const pubnub = require('pubnub');
const xhr = require('xhr');
const vault = require('vault');

// Process all messages posted to the private ChatGPT channel
// This will be unique for each user, so chat is 1:1 with ChatGPT
export default request => {
  const inboundChannel = request.channels[0];
  if (request.message.content.type == "text" && request.message.sender != "OpenAI")
  {
    //  user’s query
    let naturalCommand = request.message.content.text;
   // Always provide OpenAI with some context of how you want questions answered
    let historicalMsgs = 
      [{"role": "system", "content": "You are a helpful assistant, expert in PubNub"}];
    return getOpenaiApiKey().then(apikey => {
      // Read the last messages from the conversation to provide conversation context
      return pubnub.history({
        channel: inboundChannel,
        count: 10
        }).then((response) => {
          response['messages'].forEach((value, index) => {
            // The message role will vary depending on whether it was sent or received 
            if (value.entry.sender && value.entry.sender == "OpenAI")
              historicalMsgs.push({"role": "assistant", 
                "content": value.entry.content.text})
            else
              historicalMsgs.push({"role": "user", 
                "content": value.entry.content.text})
          });

          // Depending on timing, last published message may or may not yet be stored
          if (historicalMsgs[historicalMsgs.length - 1].content != naturalCommand)
          {
            historicalMsgs.push({"role": "user", 
              "content": naturalCommand});
          }
          return openAI(naturalCommand, historicalMsgs).then(aiResponse => {
            // Respond back by publishing the message to PubNub
            pubnub.publish({
              channel: inboundChannel,	//  In the showcase, this is unique per user
              message: {
                content: {
                  type: "text",
                  text: aiResponse
                },
                sender: "OpenAI",
              },
            });
            return request.ok();
          });
        })
      });
    }
    return request.ok();
  };


  // Get API Key for OpenAI
  // Key is populated via Vault Secret Manager  
  let OPENAI_API_KEY = null;
  function getOpenaiApiKey() {
    // Use cached key
    if (OPENAI_API_KEY) {
      return new Promise(resolve => resolve(OPENAI_API_KEY));
    }
    // Fetch key from vault
    return vault.get("OPENAI_API_KEY").then(apikey => {
      OPENAI_API_KEY = apikey;
      return new Promise(resolve => resolve(OPENAI_API_KEY));
    });  
  }

  // API Call to OpenAI asking the AI to run functions if it thinks it needs to
  function openAI(naturalCommand, msgs) {
    // Generate Function Instructions for the AI
    const url = 'https://api.openai.com/v1/chat/completions';
    const http_options = {
      'method': 'POST',
      'headers': {
        "Content-Type": "application/json",
        "Authorization": `Bearer ${OPENAI_API_KEY}`,
      },
    'body': JSON.stringify({
      "model": "gpt-3.5-turbo",
      "messages": msgs,
      "max_tokens": 300
    }),
    timeout: 9500, // PubNub functions timeout limit can be raised if needed
    retries: 0
  };

  return xhr.fetch(url, http_options).then((resp) => {
    const body = JSON.parse(resp.body);
    return body.choices[0].message.content;
  })
  .catch((err) => {
    console.log(err);
    return "Timed out. Please try again or ask a simpler question";
  });
}
```

**Practical Considerations when Calling the OpenAI API**
--------------------------------------------------------

### **Chat Completion API Properties**

The code featured in this article uses the [OpenAI Chat Completion API](https://platform.openai.com/docs/api-reference/chat/create), but it is more configurable than is shown above.

Specifying `stream` to be true will return the OpenAI response as a data stream over [server-sent events](https://www.pubnub.com/guides/server-sent-events/).  In practical terms, this can reduce the perceived latency since the user will see a more immediate response, similar to how the ChatGPT web interface works.

`model` allows you to specify the language model used to generate the response.  The API is powered by a family of models with different capabilities and price points, with models being upgraded and new models being added frequently.

`max_tokens` restricts the number of [tokens](https://platform.openai.com/tokenizer) the API will use to generate output.  In general, the smaller the number of tokens, the less complex the output and the less context is kept track of, but responses will be provided quicker.

Message `role` allows you to specify the author of a message.  The example in this article uses `system` to provide some instructions to OpenAPI, `user` for messages sent by the end user, and `assistant` for responses previously provided by OpenAI.

The model can be further tweaked by specifying the `temperature` / `top_p`, `frequency_penality`, and `presence_penalty`.  Please see the [OpenAPI documentation](https://platform.openai.com/docs/api-reference/chat/create) for more information about these.

The API also allows you to define and specify custom functions that augment the capabilities of the model, which is outside the scope of this blog.  If you would like to learn more, we have a [separate post](https://gist.github.com/stephenlb/de558e9802a6f950951eb0939ab71bcd) that illustrates how you can allow ChatGPT to publish to PubNub on your behalf, enabling you to say something like, “Send a message to the main channel telling them that I'll be ready in 5 minutes.”  You could also allow ChatGPT to invoke your own business API, providing, for example, product information or order status information directly to the customer.

### **Providing Context to Chat**

[HTTP](https://www.pubnub.com/guides/http/) requests are stateless, so to engage in a long-running conversation with OpenAI, there needs to be a way to give some context to the API, describe the conversation so far, and to provide metadata around that conversation.

The [Chat Completion API](https://platform.openai.com/docs/api-reference/chat/create) accepts an array of `messages`, which allow the AI model to derive meaning from what otherwise might be an ambiguous question.  Consider this example:

\> \[User\] How big is the earth

\> \[OpenAI\] The earth has a diameter of approximately 12,742 kilometers

\> \[User\] **I meant how heavy is it?**

\> \[OpenAI\] The Earth has a mass of approximately 5.97x10^24 kg

Clearly, to answer the question “How heavy is it” the AI needed to understand what “it” was.

As you build your PubNub powered chatbot, you will need to decide how much conversation context should be provided.  If you are using PubNub functions, you have access to the entire message history for the channel, and the example provides the previous 10 messages in the conversation.  You could of course provide a lot more if required, depending on your needs:

```js
// Read in the last messages from the conversation to provide some context to Chat GPT
pubnub.history({
  channel: inboundChannel,
  count: 10
}).then((response) => {
  response['messages'].forEach((value, index) => {
    if (value.entry.sender && value.entry.sender == "ChatGPT")
      historicalMsgs.push({"role": "assistant", "content": value.entry.content.text})
    else
      historicalMsgs.push({"role": "user", "content": value.entry.content.text})
});
```

**Having the chat context already within PubNub makes it trivial to provide that context to OpenAI**.  Note that the `role` will differ depending on whether the message was sent to OpenAI or received from OpenAI.

### **Long-Running Queries**

As you develop your ChatGPT-like chatbot and configure the parameters used by your AI model, consider the expected response time to your query.  By default, PubNub functions offer a maximum runtime of 10 seconds per invocation, which can be increased by contacting support.  In general, a chatbot that takes too long to respond to a user’s question will be perceived negatively by the user, so it is good practice to dial in the parameters of your model to achieve a balance between responsiveness and capabilities, as mentioned earlier.

Some use cases for OpenAI can expect responses measured in minutes rather than seconds.  If you are integrating OpenAI into your PubNub solution and expect the duration of the response to exceed the default Function runtime, please [get in touch](https://www.pubnub.com/company/contact-us/) with us, and we can provide recommendations and advice.

**In Summary**
--------------

Using a combination of both PubNub and OpenAI / ChatGPT, it is possible to quickly create immensely powerful real-time solutions to add AI and chat to any online experience.

When you are ready to get started, visit our [admin dashboard](http://admin.pubnub.com/) to sign up for a free set of PubNub keys.  We have a number of [tutorials](https://www.pubnub.com/tutorials/) and [demos](https://www.pubnub.com/demos/) covering every industry, or check out our [PubNub showcase](http://pubnub-showcase.netlify.app/) (also available on [Github](https://github.com/PubNubDevelopers/PubNub-Showcase)) to see an example chatbot powered by OpenAI. 

#### Contents

[High-Level Architecture](#h-0)[Configuring your PubNub Function](#h-1)[Create your PubNub function as follows:](#h-2)[Here is an example PubNub function that will invoke OpenAI / ChatGPT](#h-3)[Practical Considerations when Calling the OpenAI API](#h-4)[Chat Completion API Properties](#h-5)[Providing Context to Chat](#h-6)[Long-Running Queries](#h-7)[In Summary](#h-8)

How can PubNub help you?
========================

This article was originally published on [PubNub.com](https://www.pubnub.com/blog/build-a-chatbot-with-pubnub-and-chatgpt-openai/)

Our platform helps developers build, deliver, and manage real-time interactivity for web apps, mobile apps, and IoT devices.

The foundation of our platform is the industry's largest and most scalable real-time edge messaging network. With over 15 points-of-presence worldwide supporting 800 million monthly active users, and 99.999% reliability, you'll never have to worry about outages, concurrency limits, or any latency issues caused by traffic spikes.

Experience PubNub
-----------------

Check out [Live Tour](https://www.pubnub.com/tour/introduction/) to understand the essential concepts behind every PubNub-powered app in less than 5 minutes

Get Setup
---------

Sign up for a [PubNub account](https://admin.pubnub.com/signup/) for immediate access to PubNub keys for free

Get Started
-----------

The [PubNub docs](https://www.pubnub.com/docs) will get you up and running, regardless of your use case or [SDK](https://www.pubnub.com/docs)