Custom Plugin Development For APISIX With Lua And ChatGPT
Tags: webdev, chatgpt, ai, opensource
Published: 2023-06-14 10:37:31

One of the key features of Apache APISIX is its extensibility through [plugins](https://docs.api7.ai/apisix/key-concepts/plugins). APISIX allows you to build your own [custom plugin](https://apisix.apache.org/docs/apisix/plugin-develop/) to add extra functionalities and manage the API traffic more efficiently. Oftentimes, you use Lua programming language to implement new plugins or leverage [plugin runners](https://apisix.apache.org/docs/apisix/external-plugin/) to develop plugins in your favorite programming language. However, APISIX has the best support for [Lua](https://www.lua.org/). After writing a couple of plugins for APISIX in Lua, I understood that you do not need to know the fundamentals of **Lua programming** or to be an expert in this language at all when **your ChatGPT friend** is always with you. For example, with my background in **Java and C#**, I can understand the code and logic written in Lua and I believe that you can do the same. 

This article will guide you through the process of developing a new custom plugin called **file-proxy** for APISIX using Lua and ChatGPT (We use it to write some Lua code for us). This plugin will be used to **expose the static files through API** and fetch a file from a specified URL.

> APISIX was built to enhance the existing functionalities of [Nginx](https://www.nginx.com/). And Nginx provides a bunch of reusable [Lua modules](https://www.nginx.com/resources/wiki/modules/lua/) that APISIX makes use of them. 


## New file proxy plugin use-case

Before jumping into the actual plugin implementation, let’s understand first why we need this plugin.  At the time of writing this post, APISIX might not provide a built-in plugin for a similar case. That’s why we are going to build a new one. Often, we want to **expose a static file** (Yaml, JSON, JavaScript, CSS, or image files) through API. 

For example, APISIX API Gateway **acts as a front door** in your application to [route](https://docs.api7.ai/apisix/key-concepts/routes) incoming requests to multiple API endpoints, it is **the right place** to define all server URLs, paths, parameters, descriptions of each API endpoint and their inputs and outputs. And you build [OpenAPI specification](https://swagger.io/specification/)s to document the API. OpenAPI `.yaml` file is like a map that guides your API user in understanding and interacting with your API. By providing the path of `openapi.yaml` file (where it is stored in your server) to the plugin, you can fetch and serve the file directly through your API gateway, providing a consistent interface for API consumers. Then your API users can access .yaml file at the specified URL (`https://example.com/openapi.yaml`). 

There are other use cases as well, you might think of using this file-proxy plugin for a simple Content Delivery Network (CDN) replacement. If you have a small-scale application and don't want to use a full-fledged CDN, you can use the file-proxy plugin to serve static files from a specific location. The file-proxy plugin can be used as a caching layer for files. If you have files that are expensive to fetch or generate, you can use the plugin to fetch the file once and then serve the cached version for subsequent requests.

## Steps to develop the file-proxy plugin

We are going to run APISIX locally and our API Gateway will be hosted on 
[http://localhost:9080](http://localhost:9080/). When the development is ready, you can deploy it to your server or any cloud provider.  Basically, we want to place a file `openapi.yaml` to `http://localhost:9080/openapi.yaml` path. You will learn how to achieve this.

### Prerequisites

- Before you start, it is good to have a basic understanding of APISIX. Familiarity with API gateway, and its key concepts such as [routes](https://docs.api7.ai/apisix/key-concepts/routes), [upstream](https://docs.api7.ai/apisix/key-concepts/upstreams), [Admin API](https://apisix.apache.org/docs/apisix/admin-api/), [plugins](https://docs.api7.ai/apisix/key-concepts/plugins), and HTTP protocol will also be beneficial.
- [Docker](https://docs.docker.com/get-docker/) is used to installing the containerized etcd and APISIX.
- [curl](https://curl.se/) is used to send requests to APISIX Admin API. You can also use tools such as [Postman](https://www.postman.com/) to interact with the API.

### Understand the demo project and files

We will leverage the existing [file-proxy demo](https://github.com/Boburmirzo/apisix-file-proxy-plugin-demo) project on GitHub. It has a quite similar structure to the existing [Apisix docker example](https://github.com/apache/apisix-docker/tree/master/example) repo, only we removed unnecessary files to keep the demo simple. The project has 3 folders, [docker-compose.yml](https://github.com/Boburmirzo/apisix-file-proxy-plugin-demo/blob/main/docker-compose.yml), and sample [openapi.yaml](https://github.com/Boburmirzo/apisix-file-proxy-plugin-demo/blob/main/openapi.yaml) files.

- [docker-compose.yml](https://github.com/Boburmirzo/apisix-file-proxy-plugin-demo/blob/main/docker-compose.yml) defines two containers one for APISIX and another for etcd (which is configuration storage for APISIX).
- [custom-plugins](https://github.com/Boburmirzo/apisix-file-proxy-plugin-demo/tree/main/custom-plugins) folder has the implementation of the **file-proxy** plugin in Lua. We review it in the following sections.
- [openapi.yaml](https://github.com/Boburmirzo/apisix-file-proxy-plugin-demo/blob/main/openapi.yaml) is just a sample OpenAPI specification we expose.

### Implement file-proxy plugin

We start by asking ChatGPT, it gives us a guide almost close to the real implementation but the answer is too abstract and when you follow the process, you will end up with a non-working plugin. However, it helps us to extract useful Lua code. If we know the real process of developing plugins, it will be easier to combine both knowledge in practice.

![Implement file-proxy plugin with ChatGPT](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8270n0nfnaxg1pmm0wr3.png)

**1. Create a Lua File**

We create a new empty Lua file in the `/custom-plugins` directory of the project. The name of the file should be the name of our plugin. For example, if your plugin is named `file-proxy`, you should create a file named `file-proxy.lua`.

**2. Register the plugin in APISIX**

APISIX needs to know where this plugin file is located and is able to run the plugin accordingly. To do so, we should first define the file path where APISIX finds `file-proxy.lua` file by adding the file path to the `extra_lua_path` attribute of APISIX in the [config.yaml](https://github.com/Boburmirzo/apisix-file-proxy-plugin-demo/blob/main/apisix_conf/config.yaml)**.**

```yaml
apisix:
  extra_lua_path: "/opt/?.lua"
  node_listen: 9080              
```

Now you may ask why the file path is set to `/opt/?.lua`. Because we run APISIX using docker. You may notice this in the [docker-compose.yml](https://github.com/Boburmirzo/apisix-file-proxy-plugin-demo/blob/main/docker-compose.yml) file there are 3 volumes `./custom-plugins:/opt/apisix/plugins:ro`

```yaml
volumes:
      - ./apisix_conf/config.yaml:/usr/local/apisix/conf/config.yaml:ro
      - ./openapi.yaml:/usr/local/apisix/conf/openapi.yaml:ro
      - ./custom-plugins:/opt/apisix/plugins:ro
```

This mounts the local directory*`./custom-plugins`* where our `file-proxy.lua` file with the custom plugin implementation as a read-only volume in the docker container at the path **`/opt/apisix/plugins`**. This allows the custom plugin to be added to APISIX in the runtime to another path in Docker which is inside `/opt/?.lua`. Similarly, the other two files we copied to Docker folders.

Next step, we enable the plugin in the APISIX plugins list. This is done by adding the plugin name to the **`plugins`** list in the APISIX configuration file ([config.yaml](https://github.com/Boburmirzo/apisix-file-proxy-plugin-demo/blob/main/apisix_conf/config.yaml)):

```yaml
plugins:       
  - file-proxy
```

> Note that this action will override all existing default plugins specified in **[config-default.yaml](https://github.com/apache/apisix/blob/master/conf/config-default.yaml).** You need to add manually other plugins by their name if you want to use your custom plugin with the combination.
> 

**3. File proxy plugin Lua code breakdown**

Up to now, we only registered the plugin that simply does nothing. It is time to implement it. The plugin logic is implemented as Lua functions. You can check how it is done in **[file-proxy.lua](https://github.com/Boburmirzo/apisix-file-proxy-plugin-demo/blob/main/custom-plugins/file-proxy.lua).**

Let’s break down the **[file-proxy.lua](https://github.com/Boburmirzo/apisix-file-proxy-plugin-demo/blob/main/custom-plugins/file-proxy.lua)** file to better understand the structure of the code and flow that helps you to create new plugins on your own. You can simply ask ChatGPT to explain the Lua code:

![explain the Lua code with ChatGPT](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/q8aywuuntie5ptlabgqi.png)

Actually, we got quite a good explanation of the code (Because it was partially written by ChatGPT).


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/d3sq1m99k9ahom8yz60x.png)

I will only walk you through the important parts of this code so that you are not lost or fully rely on AI to write your plugins.

**4. Plugin file structure**

Every plugin Lua file should have the following structure:

**1. Modules:** You import the necessary modules/libraries we need for the plugin

```lua
local core     = require("apisix.core")
...
```

**2. Plugin name:** Every plugin has a unique name, it can be the same as our Lua file name.

```lua
local plugin_name = "file-proxy"
```

**3. Plugin schema:** Every plugin has a plugin schema, where we usually specify inputs to the plugin. The input we will pass from the APISIX route configuration, which you can see later when we test the plugin. For the `file-proxy` plugin, the plugin needs a file path to read the file and return a response so our parameter is the `path` which string type. You understand the schema like a method declaration with params in other programming languages.

```lua
local plugin_schema = {
    type = "object",
    properties = {
        path = {
            type = "string" -- The path of the file to be served
        },
    },
    required = {"path"} -- The path is a required field
}
```

**4. Plugin definition:** It is a really important part of plugin implementation that we define as a table with properties for the `version`, `priority`, `name`, and `schema`. The **`name`** and **`schema`** are the plugin's name and schema defined earlier. The **`version`** and **`priority`** are used by APISIX to manage the plugin. The version typically refers to the version that is currently in use like API versioning. If you publish and update your plugin logic, it is going to be `1.1` (You can set any version you wish). But you need to be very careful in choosing priority. The `priority` field defines in which order and phase your plugin should be executed. For example, the 'ip-restriction' plugin, with a priority of 3000, will be executed before the 'example-plugin', which has a priority of 0. This is due to **the higher priority** value of the 'ip-restriction' plugin. If you're developing your own plugin, make sure that you followed the order of plugins not to mess up the order of existing plugins. You can check the order of existing plugins in the [config-default.yaml](https://github.com/apache/apisix/blob/master/conf/config-default.yaml) file and open the [Apache APISIX Plugin Development Guide](https://apisix.apache.org/docs/apisix/plugin-develop/#name-priority-and-the-others) to determine. 

```lua
local _M = {
    version = 1.0,
    priority = 1000,
    name = plugin_name,
    schema = plugin_schema
}
```

**5. Schema check:** The **`check_schema`** Lua function is used to validate the plugin in a route configuration (You will see it soon in the test section) against the plugin schema we defined earlier.

```lua
-- Function to check if the plugin configuration is correct
function _M.check_schema(conf)
  -- Validate the configuration against the schema
  local ok, err = core.schema.check(plugin_schema, conf)
  -- If validation fails, return false, and the error
  if not ok then
      return false, err
  end
  -- If validation succeeds, return true
  return true
end
```

**6. Plugin logic**:  **`access`** function is the core function where we can write the major plugin logic. It is called during the access phase of the Nginx request processing pipeline and we control the traffic and write custom instructions. For **file-proxy**, we need to open the file specified in the plugin configuration, read its content, and return the content as the response. If the file cannot be opened, it logs an error and returns a 404 Not Found status. It is the exact place we give this work to ChatGPT:


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/cyvmvyeotzuo2o6jdp05.png)

After we structured and refactored the code, below is how it looks like:

```lua
function _M.access(conf, ctx)
  -- Open the file specified in the configuration
  local fd = io.open(conf.path, "rb")
  -- If the file is opened successfully, read its content and return it as the response
  if fd then
    local content = fd:read("*all")
    fd:close()
    ngx.header.content_length = #content
    ngx.say(content)
    ngx.exit(ngx.OK)
  else
    -- If the file cannot be opened, log an error and return a 404 Not Found status
    ngx.exit(ngx.HTTP_NOT_FOUND)
    core.log.error("File is not found: ", conf.path, ", error info: ", err)
  end
end
```

**7. Logging logic:** It is always preferable to log plugin configuration so that we can debug and check if the plugin is working as we expected. We can log requests to the plugin and responses.

```lua
-- Function to be called during the log phase
function _M.log(conf, ctx)
    -- Log the plugin configuration and the request context
    core.log.warn("conf: ", core.json.encode(conf))
    core.log.warn("ctx: ", core.json.encode(ctx, true))
end
```

### Install Apache APISIX

Once we learned how to develop our custom file-proxy plugin, registered in APISIX. Now it is time to test the plugin. You can easily install the [apisix-file-proxy-plugin-demo](https://github.com/Boburmirzo/apisix-file-proxy-plugin-demo) project by running `docker compose up` from the project root folder after you fork/clone [the project](https://github.com/Boburmirzo/apisix-file-proxy-plugin-demo). 

### Create a route with the file-proxy plugin

To use and test our new file-proxy plugin we need to create a route in APISIX that uses the plugin: 

```bash
curl "http://127.0.0.1:9180/apisix/admin/routes/open-api-definition" -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
   "name":"OpenAPI Definition",
   "desc":"Route for OpenAPI Definition file",
   "uri":"/openapi.yaml",
   "plugins":{
      "file-proxy":{
         "path":"/usr/local/apisix/conf/openapi.yaml"
      }
   }
}'
```

You can ask ChatGPT to explain the above configuration:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/e3u3tvs2njt2b1xbqo2k.png)

### Test the plugin

Then, you can send a cURL request to the route or open the link http://127.0.0.1:9080/openapi.yaml in your browser. The response should be the content of the file `openapi.yaml`at the specified URL.

```bash
curl -i http://127.0.0.1:9080/openapi.yaml
```

The plugin works as we expected. With this plugin configuration, you can now access any files using the specified route.

## Summary

Developing custom plugins for APISIX with Lua is a powerful way to extend the functionality of the API gateway. We demonstrated how to create the [file-proxy](https://github.com/Boburmirzo/apisix-file-proxy-plugin-demo) plugin in this post, defined the plugin definition and schema, validated the plugin configuration, and implemented custom logic during the access and log phases of the request processing pipeline in APISIX. ChatGPT helped us to write Lua code for the main functionality by filling our lacking knowledge of this programming language. Happy coding!

### Related resources

- [Custom plugin development in Lua](https://apisix.apache.org/docs/apisix/plugin-develop/)
- [How to Build an Apache APISIX Plugin From 0 to 1?](https://api7.ai/blog/how-to-build-an-apache-apisix-plugin-from-0-to-1)

### Community

🙋 [Join the Apache APISIX Community](https://apisix.apache.org/docs/general/join/)

🐦 [Follow us on Twitter](https://twitter.com/ApacheAPISIX)

📝 [Find us on Slack](https://join.slack.com/t/the-asf/shared_invite/zt-vlfbf7ch-HkbNHiU_uDlcH_RvaHv9gQ)

💁 [How to contribute page](https://apisix.apache.org/docs/general/how-to-contribute/)

### About the author

Follow me on Twitter: [@BoburUmurzokov](https://twitter.com/BoburUmurzokov)

Visit my blog: [www.iambobur.com](https://www.iambobur.com/)