Batch request processing with API Gateway
Tags: api, webdev, apigateway, tutorial
Published: 2023-04-28 09:26:19

*Batch request processing* is a powerful technique used in web development to improve the performance of APIs. It allows developers to group multiple API requests into a single HTTP request/response cycle. In other words, **a single API request** from a client can be turned into **multiple API requests** to a set of backend servers, and the **responses are aggregated** into a single response to the client. It can significantly reduce the number of round trips between the client and server.

In this article, we'll explore **how to implement batch request processing** in [Apache APISIX](https://apisix.apache.org/) and look at some use cases where it can be beneficial.

> It is similar to the [API Composition](https://microservices.io/patterns/data/api-composition.html) pattern which is widely applied in [Microservices architecture](https://microservices.io/patterns/microservices.html).

![Batch request processing vs multiple API calls](https://static.apiseven.com/uploads/2023/04/25/VfL2CwZI_Batch%20request%20processing%20with%20API%20Gateway%20%281%29.png)
Batch request processing vs multiple API calls.

## Why use batch request processing?

When a client sends multiple API requests to a server, each request requires a separate HTTP request/response cycle. This can result in increased latency, reduced performance, and increased server load. By grouping multiple requests into a single batch request, the number of HTTP request/response cycles can be reduced, resulting in improved performance and reduced latency.

Batch request processing can be particularly useful in scenarios where you need to retrieve or update multiple records in a single transaction, such as:

- Retrieving multiple records from a database
- Updating multiple records in a database
- Executing multiple API requests to complete a task

## Real-world examples for batch request processing

### Example 1:

Suppose you have a social media application that displays a user's feed, which includes posts from their friends and pages they follow. To populate this feed, you need to make multiple API requests to retrieve the necessary data, such as:

- Retrieve a list of the user's friends and the pages they follow.
- For each friend or page, retrieve their recent posts.

![social media application that displays a user's feed](https://static.apiseven.com/uploads/2023/04/25/jhCOSXJf_Batch%20request%20processing%20with%20API%20Gateway%20%283%29.png)

In a traditional approach, you would perform each of these API requests separately. First, you retrieve a list of users’ friends and in the second request, you get recent posts for each user’s friend. This can result in increased latency, especially when the user has a large number of friends and follows many pages.

### Example 2:

Another example, you have a mobile application that displays a list of products for users to browse. To populate this list, you need to make multiple API requests to retrieve product data from a remote server, such as:

- Retrieve a list of product IDs.
- For each product ID, retrieve the product details (name, description, image, etc.)

### Example 3:

Imagine that you have a web app for conference management where there are multiple speakers in the system and you want to display a speaker’s sessions and related topics on a single web page. A backend [Conference API](https://conferenceapi.azurewebsites.net/) service has two different endpoints `/speaker/{speakerId}/sessions` and `/speaker/{speakerId}/`topics to expose this information. To show both sessions and topics belong to a single speaker, you can send two requests from the frontend app which is not an ideal solution. Instead, you can use an [API Gateway](https://apisix.apache.org/docs/apisix/terminology/api-gateway/) to group all of these requests into a single HTTP request, as it is explained in the next section.

![Batch request processing with API Gateway](https://static.apiseven.com/uploads/2023/04/25/OiyxMkqE_Batch%20request%20processing%20with%20API%20Gateway%20%282%29.png)

## Batch request processing with Apache APISIX API Gateway

To implement batch request processing in APISIX, you can use the [batch-requests](https://apisix.apache.org/docs/apisix/plugins/batch-requests/)
plugin. This plugin allows you to define a set of API requests in a single `HTTP POST` request payload. Each request can have its own HTTP method, URL path, set of headers, and payload. See a curl request command below for example 3 (Conference API requests):

```bash

curl -i http://{API_GATEWAY_HOST_ADDRESS}/speaker  -X POST -d \
'{
    "pipeline": [
        {
            "method": "GET",
            "path": "/speaker/1/topics"
  },
        {
            "method": "GET",
            "path": "/speaker/1/sessions"
  }
    ]
}'
```

When a batch request is received by APISIX, the [batch-requests](https://apisix.apache.org/docs/apisix/plugins/batch-requests/) plugin will parse the payload and execute each request in the batch in parallel. The plugin will also aggregate the responses from each request and return them in a single HTTP response to the client. See the next demo section to learn how to achieve this step-by-step.

## Batch requests plugin demo

Before you can use the [batch-requests](https://apisix.apache.org/docs/apisix/plugins/batch-requests/) plugin, you'll need to install Apache APISIX.

### Prerequisites

- [Docker](https://docs.docker.com/get-docker/) is used to install the containerized etcd and APISIX.
- [curl](https://curl.se/) is used to send requests to APISIX Admin API. You can also use easy tools such as [Postman](https://www.postman.com/) to interact with the API.

APISIX can be easily installed and started with the following quickstart script:

```bash
curl -sL <https://run.api7.ai/apisix/quickstart> | sh
```

### Configure the backend service (upstream)

You will need to configure the backend service for Conference API that you want to route requests to. This can be done by adding an upstream server in the Apache APISIX through the [Admin API](https://apisix.apache.org/docs/apisix/admin-api/).

```bash
curl http://127.0.0.1:9180/apisix/admin/upstreams/1 -X PUT -d '
{
  "name": "Conferences API upstream",
  "desc": "Register Conferences API as the upstream",
  "type": "roundrobin",
  "scheme": "https",
  "nodes": {
    "conferenceapi.azurewebsites.net:443": 1
  }
}'
```

### Create a Route for batch processing API

We need to create a new route that intercepts requests to `/speaker` and exposes a public virtual endpoint for batch processing using `[public-api](https://apisix.apache.org/docs/apisix/plugins/public-api/)` plugin.

```bash
curl http://127.0.0.1:9180/apisix/admin/routes/a -X PUT -d '
{
    "uri": "/speaker",
    "plugins": {
        "public-api": {
      "uri": "/apisix/batch-requests"
    }
    }
}'
```

### Create a Route for the speaker’s topics and sessions endpoints

Next, we create another route for the speaker’s topics and sessions endpoints (`/speaker/*/topics` and `/speaker/*/topics` paths matching) so that individual requests extracted by the API Gateway from batch requests to retrieve the speaker’s topics or sessions are forwarded to the responsible Conference API endpoints and we referenced to the existing upstream service.

```bash
curl http://127.0.0.1:9180/apisix/admin/routes/b -X PUT -d '
{
  "methods": ["GET"],
 "uris": ["/speaker/*/topics","/speaker/*/sessions"],
    "plugins": {
      "proxy-rewrite":{
      "host":"conferenceapi.azurewebsites.net"
  }
  },
 "upstream_id":"1"
}'
```

You may notice that we are using another [proxy-rewrite](https://apisix.apache.org/docs/apisix/plugins/proxy-rewrite/) plugin and specifying implicitly host address for Conference API. Otherwise, API Gateway can make DNS conversion and request the Conference API by its IP address.

### Test batch request processing

Here's an example of how to use the batch-requests plugin in APISIX:

```bash
curl -i http://127.0.0.1:9080/speaker  -X POST -d \
'{
    "pipeline": [
        {
            "method": "GET",
            "path": "/speaker/1/topics"
  },
        {
            "method": "GET",
            "path": "/speaker/1/sessions"
  }
   ]
}'
```

In this example, the route is defined for the `/speaker` endpoint, which supports batch request processing via the batch-requests plugin. The plugin is configured with a set of two requests, each retrieving a speaker record by ID with topics and sessions. If you run this command, you will get a merged response back from the API Gateway:

```bash
[
   {
      "body":"{\r\n  \"collection\": {\r\n    \"version\": \"1.0\",\r\n    \"links\": [],\r\n    \"items\": [\r\n      {\r\n        \"href\": \"https://conferenceapi.azurewebsites.net/topic/8\",\r\n        \"data\": [\r\n          {\r\n            \"name\": \"Title\",\r\n            \"value\": \"Microsoft\"\r\n          }\r\n        ],\r\n        \"links\": [\r\n          {\r\n            \"rel\": \"http://tavis.net/rels/sessions\",\r\n            \"href\": \"https://conferenceapi.azurewebsites.net/topic/8/sessions\"\r\n          }\r\n        ]\r\n      },\r\n      {\r\n        \"href\": \"https://conferenceapi.azurewebsites.net/topic/10\",\r\n        \"data\": [\r\n          {\r\n            \"name\": \"Title\",\r\n            \"value\": \"Mobile\"\r\n          }\r\n        ],\r\n        \"links\": [\r\n          {\r\n            \"rel\": \"http://tavis.net/rels/sessions\",\r\n            \"href\": \"https://conferenceapi.azurewebsites.net/topic/10/sessions\"\r\n          }\r\n        ]\r\n      }\r\n    ],\r\n    \"queries\": [],\r\n    \"template\": {\r\n      \"data\": []\r\n    }\r\n  }\r\n}",
      "status":200,
      "headers":{
         "Expires":"-1",
         "Connection":"keep-alive",
         "Pragma":"no-cache",
         "Content-Length":"953",
         "Server":"APISIX/3.2.0",
         "Content-Type":"application/vnd.collection+json",
         "X-AspNet-Version":"4.0.30319",
         "Cache-Control":"no-cache",
         "X-Powered-By":"ASP.NET"
      },
      "reason":"OK"
   },
   {
      "body":"{\r\n  \"collection\": {\r\n    \"version\": \"1.0\",\r\n    \"links\": [],\r\n    \"items\": [\r\n      {\r\n        \"href\": \"https://conferenceapi.azurewebsites.net/session/206\",\r\n        \"data\": [\r\n          {\r\n            \"name\": \"Title\",\r\n            \"value\": \"\\r\\n\\t\\t\\tjQuery Mobile and ASP.NET MVC\\r\\n\\t\\t\"\r\n          },\r\n          {\r\n            \"name\": \"Timeslot\",\r\n            \"value\": \"05 December 2013 09:00 - 10:00\"\r\n          },\r\n          {\r\n            \"name\": \"Speaker\",\r\n            \"value\": \"Scott Allen\"\r\n          }\r\n        ],\r\n        \"links\": [\r\n          {\r\n            \"rel\": \"http://tavis.net/rels/speaker\",\r\n            \"href\": \"https://conferenceapi.azurewebsites.net/speaker/16\"\r\n          },\r\n          {\r\n            \"rel\": \"http://tavis.net/rels/topics\",\r\n            \"href\": \"https://conferenceapi.azurewebsites.net/session/206/topics\"\r\n          }\r\n        ]\r\n      }\r\n    ],\r\n    \"queries\": [],\r\n    \"template\": {\r\n      \"data\": []\r\n    }\r\n  }\r\n}",
      "status":200,
      "headers":{
         "Expires":"-1",
         "Connection":"keep-alive",
         "Pragma":"no-cache",
         "Content-Length":"961",
         "Server":"APISIX/3.2.0",
         "Content-Type":"application/vnd.collection+json",
         "X-AspNet-Version":"4.0.30319",
         "Cache-Control":"no-cache",
         "X-Powered-By":"ASP.NET"
      },
      "reason":"OK"
   }
]
```

> The [maximum size](https://apisix.apache.org/docs/apisix/plugins/batch-requests/#configuration) of a batch request is limited by API Gateway. You can check the API Gateway documentation for the current limits and request timeout configuration.

## Takeaways

- Batch request processing with API Gateway can be a useful technique for improving the performance of your API.
- Apache APISIX provides a plugin called **`batch-requests`** that allows developers to implement batch request processing easily.

## Next steps

With API Gateway, it is also possible to provide some form of custom aggregation in the response data to your users. You can use the [serverless-function](https://apisix.apache.org/docs/apisix/plugins/serverless/) plugin to execute custom code and merge the response from backend services and return it to the API consumer in a different structure.

### Recommended content

- [Why Is Apache APISIX the Best API Gateway?](https://api7.ai/blog/why-is-apache-apisix-the-best-api-gateway)
- [What are API Gateway Policies?](https://api7.ai/blog/api-gateway-policies)

### About the author

Visit my blog: [www.iambobur.com](https://www.iambobur.com/)