Building a ChatGPT custom plugin for API Gateway
Published: 2023-07-14 06:12:21

[ChatGPT Plugins](https://openai.com/blog/chatgpt-plugins) serve as **bridges linking ChatGPT to external APIs** to use these APIs’ data intelligently. These plugins let ChatGPT undertake a range of tasks, such as retrieving up-to-date information from other APIs including sports results, stock market data, or breaking news, and assisting users in performing actions like flight booking or food ordering. Meanwhile, API Gateway is a powerful tool that allows developers to **build, deploy, and manage APIs** at scale. It acts as a gateway between ChatGPT and backend services, providing features such as authentication, rate limiting, and request/response transformations.

You can read from the [previous post](https://api7.ai/blog/api-gateway-for-chatgpt-plugins) where we explored **how API Gateway can be helpful for ChatGPT** plugin developers to **expose, secure, manage, and monitor** their API endpoints. This post guides you step-by-step through the uncomplicated and direct method of **developing a ChatGPT Plugin for API Gateway**. As a fun extra, you will also learn how to add your plugin to ChatGPT and try it out. So, make yourself comfortable, and let's embark on this journey!

## How to create a ChatGPT Plugin for API Gateway

As the [getting started](https://platform.openai.com/docs/plugins/getting-started) guide on the OpenAI website states, to build any new custom ChatGPT plugin we need to follow these general 3 steps:

1. Develop an API or use an existing one, that implements the OpenAPI specification.
2. Document the API using either the OpenAPI YAML or JSON format.
3. Generate a JSON plugin manifest file that contains essential information about the plugin.

By following the above similar steps, we are going to build a custom plugin for ChatGPT that acts as [API Gateway](https://docs.api7.ai/apisix/documentation) for backend API services. As an example, in the [ChatGPT user interface](https://chat.openai.com/), if a user wants to introduce an API Gateway in front of an existing [Conference API](https://conferenceapi.azurewebsites.net/) to obtain details about a speaker's sessions and topics, the plugin is capable of receiving commands in the chat and then forwards the user's request to the Apache APISIX [Admin API](https://apisix.apache.org/docs/apisix/admin-api/), which create a [Route](https://apisix.apache.org/docs/apisix/terminology/route/) with the user-specified input configuration. This can be another approach to **using the Chatbot to configure the API Gateway features.** See sample output below:

![Chatbot to configure the API Gateway features](https://static.apiseven.com/uploads/2023/07/09/BmW5Ce4m_Untitled%20%2810%29.png)

After the command runs successfully, APISIX creates the route and registers an [Upstream](https://apisix.apache.org/docs/apisix/terminology/upstream/) for our Conference backend API. So, you can access the API Gateway domain and URL path to get a response through the Gateway. For example, this GET request to [http://localhost:9080/speaker/1/sessions](http://localhost:9080/speaker/6/sessions) endpoint returns all speaker’s sessions from Conference API. You can also do basic operations like **get all routes, a route by Id, update an existing route or create a route with plugins and upstream** directly by asking ChatGPT.

To gain insight into the data flow between various components, you can refer to the provided architectural diagram:

### Prerequisites

- Before you start, it is good to have a basic understanding of APISIX. Familiarity with API gateway, and its key concepts such as [routes](https://docs.api7.ai/apisix/key-concepts/routes), [upstream](https://docs.api7.ai/apisix/key-concepts/upstreams), [Admin API](https://apisix.apache.org/docs/apisix/admin-api/), [plugins](https://docs.api7.ai/apisix/key-concepts/plugins), and HTTP protocol will also be beneficial.
- [Docker](https://docs.docker.com/get-docker/) is used to install the containerized etcd and APISIX.
- Download [Visual Studio Code](https://code.visualstudio.com/) compatible with your operating system, or use any other code editor like [IntelliJ IDEA](https://www.jetbrains.com/idea/), [PyCharm](https://www.jetbrains.com/pycharm/), etc.
- To develop custom **ChatGPT Plugins** you need to have a [ChatGPT Plus](https://openai.com/blog/chatgpt-plus) account and [join the plugins waitlist](https://openai.com/waitlist/plugins).

The entire code repository for the complete API Gateway Plugin is located [here](https://github.com/Boburmirzo/apisix-api-gateway-chatgpt-plugin). Let’s go through the steps one by one including code snippets, files and understand how to set up the API Gateway, document the API in the [OpenAPI definition](https://swagger.io/specification/), and create a [plugin manifest](https://platform.openai.com/docs/plugins/getting-started/plugin-manifest) file.

The **technology stack** employed for the development of this plugin includes the following:

1. An existing public [Conference API](https://conferenceapi.azurewebsites.net/) where we use two API endpoints to retrieve a speaker’s [sessions](https://conferenceapi.azurewebsites.net/speaker/1/sessions) and [topics](https://conferenceapi.azurewebsites.net/speaker/1/topics) information in the demo. You can discover other APIs on your browser.
2. [Apache APISIX](https://apisix.apache.org/) API Gateway to expose Admin API and manage the API traffic.
3. Python script to run the plugin, host the plugin manifest, OpenAPI specification, and plugin logo files on the local host URL http://localhost:5000/.
4. The project uses Docker for convenience to build, deploy and run APISIX, etcd, and our Python script with a single `docker compose up` command.

We will follow a series of steps to create the plugin. In **Part 1**, we will focus on setting up APISIX. **Part 2** will entail the development of the ChatGPT plugin itself, and **Part 3** involves connecting the plugin to ChatGPT and testing it. While we present the steps in sequential order, you are welcome to skip any of them based on your familiarity with the subject matter.

### Part 1: Set up Apache APISIX

#### Step 1: Define APISIX Config

First, we create an APISIX config file. APISIX configurations can be defined using various methods, including static configuration files typically written in YAML. In the [apisix.yml](https://github.com/Boburmirzo/apisix-api-gateway-chatgpt-plugin/blob/main/apisix_conf/config.yaml) file, we provided an example configuration:

```yaml
deployment:
  etcd:
    host:
      - "http://etcd:2397"
  admin:
    admin_key_required: false
    allow_admin:
      - 0.0.0.0/0
```

You can also consult with ChatGPT to understand what the above apisix.yml file means. Basically, this configuration defines the deployment mode for APISIX. Here, we use a simple [traditional option](https://apisix.apache.org/docs/apisix/deployment-modes/#traditional) where APISIX uses etcd storage to store configuration details as it gets them from Admin API. We also disable the Admin key `admin_key_required` for the sake of the demo.

#### Step 2: Register APISIX and etcd in Docker Compose

In the last step, we create a docker-compose.yml file and register APISIX and etcd there as services to build, deploy and run it. And APISIX Admin API will be available on port: `9180` and Gateway you can access it on port: `9080`

```yaml
version: "3"

services:
  apisix:
    image: apache/apisix:3.3.0-debian
    volumes:
      - ./apisix_conf/config.yaml:/usr/local/apisix/conf/config.yaml:ro
    restart: always
    ports:
      - "9080:9080"
      - "9180:9180"
    depends_on:
      - etcd
  etcd:
    image: bitnami/etcd:3.5.9
    environment:
      ETCD_ENABLE_V2: "true"
      ALLOW_NONE_AUTHENTICATION: "yes"
      ETCD_ADVERTISE_CLIENT_URLS: "http://0.0.0.0:2397"
      ETCD_LISTEN_CLIENT_URLS: "http://0.0.0.0:2397"
```

### Part 2:  ChatGPT plugin development

#### Step 1: Create a Plugin manifest

Every plugin requires an `ai-plugin.json` manifest file which provides important information about your plugin, like its name, description, logo assets, and so on. Read more about each manifest file field [here](https://platform.openai.com/docs/plugins/getting-started/plugin-manifest) and OpenAI includes some [best practices](https://platform.openai.com/docs/plugins/getting-started/writing-descriptions) for creating model descriptions (`description_for_model`). When you install the plugin on ChatGPT UI, the interface will look for the manifest file on your domain (http://localhost:5000/.well-known/ai-plugin.json). This helps ChatGPT understand how to interact with your plugin.

Create a new file named `ai-plugin.json` and paste the following code:

```yaml
{
    "schema_version": "v1",
    "name_for_human": "APISIX plugin for ChatGPT",
    "name_for_model": "apigatewayplugin",
    "description_for_human": "API Gateway plugin to manage your backend APIs.",
    "description_for_model": "Create, retrive, manage APISIX routes, upstreams, and plugins",
    "auth": {
        "type": "none"
    },
    "api": {
        "type": "openapi",
        "url": "http://localhost:5000/openapi.yaml"
    },
    "logo_url": "http://localhost:5000/logo.png",
    "contact_email": "support@example.com",
    "legal_info_url": "http://www.example.com/legal"
}
```

You may notice that the file also specifies the URL to http://localhost:5000/openapi.yaml file.

> When a user interacts with ChatGPT, it will refer to the `openapi.yaml` file to understand the descriptions of the endpoints. Based on this information, ChatGPT will determine the most suitable endpoint to utilize in response to the user's prompt.
>

#### Step 2: Create an OpenAPI specification

The OpenAPI specification is a standard for describing REST APIs. It is used to specify each API endpoint that the plugin will use to communicate with the model. [Read more here](https://platform.openai.com/docs/plugins/getting-started/openapi-definition). Here is an example of [openapi.yaml](https://github.com/Boburmirzo/apisix-api-gateway-chatgpt-plugin/blob/main/chatgpt-plugin-config/openapi.yaml) file:

```yaml
openapi: 3.1.0
info:
  title: APISIX Admin API
  description: >-
    APISIX Admin API is a RESTful API that allows you to create and manage
    APISIX resources.
  version: 3.3.0
servers:
  - url: http://localhost:5000
    description: Dev Environment
tags:
  - name: Route
    description: |-
      A route defines a path to one or more upstream services.  
        
      See [Routes](/apisix/key-concepts/routes) for more information.

paths:
  /apisix/admin/routes:
    get:
      operationId: getAllRoutes
      summary: Get All Routes
      deprecated: false
      description: Get all configured routes.
      tags:
        - Route
...
# See the full version on GitHub repo
```

The above OpenAPI specification is just an extraction from the real [Admin API schema](https://docs.api7.ai/apisix/reference/admin-api) and you can easily **add more API schemas** if needed. In our plugin demo, we used only Route related paths.

#### Step 3: Add endpoints for plugin static files (Optional for localhost)

Next, we create a [Flask app](https://flask.palletsprojects.com/en/2.3.x/) in Python to expose our statics file as endpoints for the [plugin's logo](https://github.com/Boburmirzo/apisix-api-gateway-chatgpt-plugin/blob/main/chatgpt-plugin-config/logo.png), [manifest](https://github.com/Boburmirzo/apisix-api-gateway-chatgpt-plugin/blob/main/chatgpt-plugin-config/ai-plugin.json), and [OpenAPI specification](https://github.com/Boburmirzo/apisix-api-gateway-chatgpt-plugin/blob/main/chatgpt-plugin-config/openapi.yaml). ChatGPT makes a call to the API endpoints to fetch all information needed for the custom plugin:

```yaml
import requests
import os

import yaml
from flask import Flask, jsonify, request, send_from_directory
from flask_cors import CORS

app = Flask(__name__)

PORT = 5000

# Note: Setting CORS to allow chat.openapi.com is required for ChatGPT to access your plugin
CORS(app, origins=[f"http://localhost:{PORT}", "https://chat.openai.com"])

api_url = 'http://apisix:9180'

@app.route('/.well-known/ai-plugin.json')
def serve_manifest():
    return send_from_directory(os.path.dirname(__file__), 'ai-plugin.json')

@app.route('/openapi.yaml')
def serve_openapi_yaml():
    with open(os.path.join(os.path.dirname(__file__), 'openapi.yaml'), 'r') as f:
        yaml_data = f.read()
    yaml_data = yaml.load(yaml_data, Loader=yaml.FullLoader)
    return jsonify(yaml_data)

@app.route('/logo.png')
def serve_openapi_json():
    return send_from_directory(os.path.dirname(__file__), 'logo.png')

# To proxy request from ChatGPT to the API Gateway
@app.route('/<path:path>', methods=['GET', 'POST'])
def wrapper(path):

    headers = {
    'Content-Type': 'application/json',
    }

    url = f'{api_url}/{path}'
    print(f'Forwarding call: {request.method} {path} -> {url}')

    if request.method == 'GET':
        response = requests.get(url, headers=headers, params=request.args)
    elif request.method in ['POST', 'DELETE', 'PATCH', 'PUT']:
        print(request.headers)
        response = requests.post(url, headers=headers, params=request.args, json=request.json)
    else:
        raise NotImplementedError(f'Method {request.method} not implemented in wrapper for {path=}')
    return response.content

if __name__ == '__main__':
    app.run(debug=True,host='0.0.0.0')
```

When the script runs, you can access files on the API domain:

1. `ai-plugin.json` will be available on the URI path http://localhost:5000/.well-known/ai-plugin.json
2. `openapi.yaml` will be accessible at the URI path http://localhost:5000/openapi.yaml
3. `logo.png` will be available on the URI path http://localhost:5000/logo.png

> Note that we enabled CORS in the above code only to **test the plugin locally** deployed with the ChatGPT interface. If the plugin is running on a remote server, you do not need a proxy part. Refer to [OpenAI docs](https://platform.openai.com/docs/plugins/getting-started/running-a-plugin) to run the plugin remotely.
>

#### Step 4: Dockerize the Python app

To run the Python app automatically with Docker, we create a Dockerfile and register it in [docker-compose.yml](https://github.com/Boburmirzo/apisix-api-gateway-chatgpt-plugin/blob/main/docker-compose.yml) file. Learn how to dockerize a Python app [here](https://www.docker.com/blog/how-to-dockerize-your-python-applications/).

```yaml
FROM python:3.9
WORKDIR /app
COPY requirements.txt requirements.txt
RUN pip3 install -r requirements.txt
COPY . .
CMD ["python","main.py"]
```

Finally, our `docker-compose.yml` file looks like this:

```yaml
version: "3"

services:
  apisix:
    image: apache/apisix:3.3.0-debian
    volumes:
      - ./apisix_conf/config.yaml:/usr/local/apisix/conf/config.yaml:ro
    restart: always
    ports:
      - "9080:9080"
      - "9180:9180"
    depends_on:
      - etcd
  
  etcd:
    image: bitnami/etcd:3.5.9
    environment:
      ETCD_ENABLE_V2: "true"
      ALLOW_NONE_AUTHENTICATION: "yes"
      ETCD_ADVERTISE_CLIENT_URLS: "http://0.0.0.0:2397"
      ETCD_LISTEN_CLIENT_URLS: "http://0.0.0.0:2397"
  
  chatgpt-config:
    build: chatgpt-plugin-config
    ports:
      - '5000:5000'
```

### Part 3: Integrate ChatGPT with the custom plugin

#### Step 1: Deploy the custom plugin

Once you have completed the development of your custom plugin, it's time to deploy and run it locally. To start the project run simply the following command from the project root directory:

```yaml
docker compose up
```

When you start the project, Docker downloads any images it needs to run. You can see that APISIX, etcd and Python app (`chatgpt-config`) services are running.

![Deploy the custom plugin](https://static.apiseven.com/uploads/2023/07/09/fjdeTtHh_Untitled%20%2811%29.png)

#### Step 2: Connect the custom plugin to the ChatGPT interface

So, you’ve deployed the ChatGPT plugin, plugin config files accessible through API and now you’re ready to test it out. If you have a **Plus account**, we must first enable the plugin in GPT-4 since it is disabled by default. We need to go to the settings and click the beta option and click “Enable plugins”. Then click the plugins pop bar on the top of ChatGPT, navigate to “Plugin Store” and select “Develop your own plugin”. Provide the local host URL for the plugin (`localhost:5000`).

![Connect the custom plugin to the ChatGPT interface](https://static.apiseven.com/uploads/2023/07/09/TXjNTZ4h_ChatGPT%20screenshot%203.png)

After you click on “Find a manifest file”, if everything is set it up correctly, you will see ChatGPT validates successfully both manifest and OpenAPI spec files.

![Find a manifest file](https://static.apiseven.com/uploads/2023/07/09/BXsrCUtY_ChatGPT%20screenshot%204.png)

#### Step 3: Test the custom plugin

Now the plugin is connected to the ChatGPT interface, we can write a simple command:

![Show all routes](https://static.apiseven.com/uploads/2023/07/12/ITTyORIN_252731767-a4e390c4-1cb1-4055-a5cb-f765d861d9bf.png)

Since we do not have any route in the APISIX, we can create a new one. Here depending on the language used in our prompt ChatGPT will choose to call the appropriate APISIX endpoints. Please, try to write a command with more specific details so that ChatGPT can create a Route correctly.

![Createa a route in the APISIX](https://static.apiseven.com/uploads/2023/07/11/5R8pO3sy_Untitled%20%289%29.png)

Now the route has been created, you can access the Gateway endpoint to fetch Conference session details: [http://localhost:9080/speaker/1/sessions](http://localhost:9080/speaker/6/sessions). You can replace this Conference API with your backend API too.

### API Gateway custom plugin other use cases

Now you can ask a question: *What else you can do with this plugin other than simply routing the requests?* There are many things you can **improve on this plugin or add extra features** that APISIX offers via the Admin API. Because we put the API Gateway in the front of the Conference API and API Gateway serves all requests coming from the plugin first, you can achieve at least the followings:

- **Security**: Assume that now you want to secure the Conference API, **without API Gateway** in place you can still secure the data exchange between ChatGPT UI and plugin by using [Plugin Authentication](https://platform.openai.com/docs/plugins/authentication) methods from OpenAI. However, the communication between the plugin and your backend service **still remains unsecured** until you implement some cross-cutting concerns in Python code, instead of spending time on this:
  - We can implement security measures like **authentication, authorization, and rate limiting** with the API Gateway to protect the Conference API from unauthorized access and potential attacks. ChatGPT can talk to API Gateway freely but the communication between API Gateway and backend service can be absolutely secure.
- **Caching:** It is possible to cache similar Conference API responses so that we can show data for ChatGPT quickly.
- **Versioning:** We can create the second version of Conference API to route ChatGPT plugin requests to the newest service without changing any config and downtime.
- **Load Balancing**: We can distribute incoming requests across multiple instances of the Conference API, ensuring high availability and efficient resource utilization.
- **Request Transformation:** We can modify the requests made to the Conference API, enabling data transformation, validation, or transform requests [from REST to GraphQL](https://api7.ai/blog/manage-graphql-api-with-api-gateway) or to [gRPC service](https://api7.ai/blog/what-is-grpc-and-how-to-work-with-apisix) calls.
- **Monitoring and Analytics:** The API Gateway provides robust monitoring and analytics [plugins](https://docs.api7.ai/apisix/how-to-guide/observability/monitor-apisix-with-prometheus), allowing you to gather valuable insights about API usage, performance, and potential issues.

> If you want to use [Apache APISIX](https://apisix.apache.org/) API Gateway as a front door for communication between ChatGPT custom plugins
and backend APIs, you can check this repo [API Gateway between ChatGPT custom plugin and backend APIs](https://github.com/Boburmirzo/apisix-chatgpt-gateway-plugin)

### Next steps

Throughout the post, you learned how to create the custom plugin for API Gateway with the basic functionalities. Now you can take the sample project as a foundation and improve functionalities by adding more APISIX [Admin API](https://docs.api7.ai/apisix/reference/admin-api) specifications to [openapi.yaml](https://github.com/Boburmirzo/apisix-api-gateway-chatgpt-plugin/blob/main/chatgpt-plugin-config/openapi.yaml) file to use and test other [plugins](https://docs.api7.ai/apisix/reference/admin-api#tag/Plugin) and add API [consumers](https://docs.api7.ai/apisix/reference/admin-api#tag/Consumer) and more. Feel free to contribute to the GitHub project by raising pull requests.

### Related resources

- [API Gateway For ChatGPT Plugins](https://api7.ai/blog/api-gateway-for-chatgpt-plugins)