Implementing resilient applications with API Gateway (Circuit breaker)
Tags: api, microservices, dotnet, apigateway
Published: 2022-08-01 10:25:00

## Everything fails, accept it!🤦

In a **distributed system**, a client service usually calls other services to retrieve data and there is the possibility that one of those services can fail due to some unexpected scenarios if there is a tough issue with a particular microservice, then it will be unavailable for a long time that's unlikely to succeed and it is not recoverable after it tries again and again with some delay (that’s how traditional [Retry strategy](https://docs.microsoft.com/en-us/azure/architecture/patterns/retry) works) because the caller service has no idea about its health and waiting for the other service to respond continuously. As a result, the network resources will be spent by delivering low performance and a bad user experience.

## Circuit breaker is a rescuer🦸

In such a situation, the service should be coded to accept that the operation has failed and handle the failure accordingly. The [Circuit Breaker](https://docs.microsoft.com/en-us/azure/architecture/patterns/circuit-breaker) pattern comes as a savior that can prevent a service from repeatedly trying to execute an operation that's likely to fail.  

![Circuit breaker on, off](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nhoy00j8uvibk6u7j05i.gif) 

> “_Your code can not just wait forever for a response that might never come, sooner or later, it needs to give up. Hope is not a design method._” -Michael T. Nygard, from the book Release It!

One solution can be implementing the circuit breaker pattern for each microservice by using proven **.NET libraries** like [Polly](https://github.com/App-vNext/Polly) which is a very time-consuming and sometimes even challenging task.

Another recommended approach is to take advantage of an [API Gateway pattern](https://microservices.io/patterns/apigateway.html) that can be considered as a single entry point means all the external requests will go through it only as a [Circuit breaker mechanism](https://microservices.io/patterns/reliability/circuit-breaker.html) should be also implemented as a proxy for operations that might fail. The proxy should monitor the number of recent failures that have occurred, and use this information to decide whether to allow the operation to proceed, or simply return an exception immediately.

## Circuit breaker in an API Gateway

By now, **circuit breaker pattern** is pretty well-known, it has many benefits in a Microservices architecture and it can improve the stability and resiliency of an application. You can learn this design pattern from many other resources like [this one](https://docs.microsoft.com/en-us/azure/architecture/patterns/circuit-breaker). 

In this post, let's examine primarily how to apply the circuit breaker for [ASP.NET Core WEB API](https://docs.microsoft.com/en-us/aspnet/core/?view=aspnetcore-6.0)s with the help of [**Apache APISIX API Gateway**](https://apisix.apache.org/).  

## Apache APISIX API Breaker Plugin

**Apache APISIX** offers 💁 a special [api-breaker](https://apisix.apache.org/docs/apisix/plugins/api-breaker/) Plugin 🔌 that implements and uses circuit breaker functionality to invoke, handle failures and prevent upstream services from constant retry attempts if the service fails or performs slowly. It determines the health of targets based on their behavior responding to requests. From the usage point of view, it is very easy to enable the plugin (there is no need to add anything in code) and you can make use of it with combination of other [built-in plugins](https://apisix.apache.org/plugins/) as well.

![Apache APISIX API Breaker Plugin](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/uivstp7944t1czvi1wtc.jpg) 

## Circuit breaker states

As you may already know, the [circuit breaker](https://docs.microsoft.com/en-us/azure/architecture/patterns/circuit-breaker) works with three different internal states. Let's go through each state and understand how **APISIX API Breaker plugin** addresses them accordingly. 

### Closed State

In the initial state, we can assume that the system is healthy and sending connections to the upstream service (backend). The `api-breaker` plugin routes the requests to the backend and counts the number of failures in each period as an interceptor. You can also define it with `unhealthy.failures` plugin’s attribute (it counts the number of consecutive failures for the upstream service to be considered unhealthy).

☝️ Note that it is also possible to configure HTTP status codes with `unhealthy.http_statuses` property for the upstream to be considered unhealthy. See all available [plugin's properties](https://apisix.apache.org/docs/apisix/plugins/api-breaker/). 

![APISIX API Breaker Closed State](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3k9gho7ilec21mply47d.jpg)

If the number of recent failures exceeds a specified threshold within a given time period (`max_breaker_sec` attribute to set the value in seconds), the proxy state is placed into the **Open state** by `api-breaker` plugin.
 
### Open State

When `api-breaker` plugin moves to the “Open” state, requests from the upstream service will fail immediately, and an exception will be returned directly from the API gateway as its shown in the below diagram:

![APISIX API Breaker Open State](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/91f01hzlnwu68kxt10zf.jpg)
 
### Half-Open State
 
In this state, `api-breaker` plugin only allows fixed number of requests pass through the upstream service.  If these requests are successful (with specified HTTP status codes in the configuration field `healthy.http_statuses` for the number of successes defined in the `healthy.successes` field), it's assumed that the fault that was previously causing the failure has been fixed and the circuit breaker switches to the healthy state (Closed state) and the number of failures counter is reset. 

![APISIX API Breaker Plugin Half-Open State](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1o7b1owsipxfc4v75nyi.jpg)

### API Breaker Plugin Demo

🙋🏼 With all the theoretical knowledge in mind, let's jump into the demo of using `api-breaker` plugin offered by [Apache APISIX](https://apisix.apache.org/) and see the result in action for our the existing sample project [ASP.NET Core WEB API](https://github.com/Boburmirzo/apisix-dotnet-docker) with a single endpoint (retrieves all products list).

#### Prerequisites

☝️ If you followed the previous blog post about [Manage .NET Microservices APIs with Apache APISIX API Gateway](https://dev.to/apisix/manage-net-microservices-apis-with-apache-apisix-api-gateway-2cbk), make sure you have read it and completed steps (To run APISIX, etcd, and ASP.NET WEB API) before continuing with a demo session.

#### Demo scenario

Take a very simple scenario of an online marketplace website where there is the main webpage we offer all/some products available in the shop with their corresponding name, price, and all other meta info (clients can search for products or sort them). While we are exchanging information between UI and backend API Server there can be a case the upstream service is failing and we obviously should show an error message indicating that service is temporarily unavailable to the user. You can handle it directly on UI code but it is not an ideal solution. Especially, it can be tough in case UI collects and aggregates data from different microservices. At this moment, API Gateway comes into the picture.

#### Create an Upstream

The first step we need to create a new upstream for our backend product service. The following command creates a sample upstream (that's our API Server):

``` bash
curl "http://127.0.0.1:9080/apisix/admin/upstreams/1" -H "X-API-KEY: edd1c9f034335f136f87ad84b625c8f1" -X PUT -d '
{
 "type": "roundrobin",
 "nodes": {
 "productapi:80": 1
 }
}
```

#### Configure the API Breaker Plugin

Now let's start with adding `api-breaker` plugin to Apache APISIX declarative configuration file `config.yaml` under `/apisix_conf` folder in the project or you can simple check-out the [api-circuit-breaker](https://github.com/Boburmirzo/apisix-dotnet-docker/tree/api-circuit-breaker) branch in the repo (it is already configured and you can find example commands we show in this demo under [/cmds](https://github.com/Boburmirzo/apisix-dotnet-docker/tree/api-circuit-breaker/cmds) folder) and run the solution from that branch. Because in the current project, we have not registered yet the plugin we are going to use for this demo. We appended `api-breaker` plugin's name to the end of plugins list:

``` yaml
plugins:
 - http-logger
 - ip-restriction
 …
 - api-breaker
```

Next, we can directly request [Admin API](https://apisix.apache.org/docs/apisix/admin-api/) to reload the latest plugin code without restarting Apache APISIX. See the command to reload the newly added plugin:

``` bash
curl http://127.0.0.1:9080/apisix/admin/plugins/reload -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT
```

#### Enable the API Breaker Plugin

Next, we configure the `api-breaker` plugin on a specific Route. The following configuration is an example of how to add circuit breaker capabilities to the upstream we created in the previous step:

``` bash
curl "http://127.0.0.1:9080/apisix/admin/routes/1" -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
    "name": "Route for circuit breaker",
    "methods": ["GET"],
    "uri": "/api/products",
    "upstream_id": "1",
    "plugins": {
        "api-breaker": {
            "break_response_code": 502,
			      "max_breaker_sec": 30,
            "unhealthy": {
                "http_statuses": [500, 503],
                "failures": 3
            },
            "healthy": {
                "http_statuses": [200],
                "successes": 1
            }
        }
    }
}'
```

In the plugin configuration example above ⤴️, the circuit breaker policy is configured so it breaks or opens the circuit when there have been three consecutive faults (500, 503 HTTP statuses return) when retrying the HTTP requests. When that happens, the upstream service moved to unhealthy state and the circuit will break for 30 seconds. In that period, calls will be failed immediately by the `api-breaker` rather than actually be placed. Any single HTTP response code of 200 will restore its healthy status again. 

#### How to validate API Breaker🙎

It is very straightforward to validate this scenario. One stupid option can be just stopping `productapi` container from **docker** 🤪 which causes the _503 Service Temporarily Unavailable_ error and we can see `api-breaker` takes the control😏 or another option update the code in the `ProductController.cs` file so that the endpoint returns _500 Internal Server error_. 

For example:

``` c#
        [HttpGet]
        public IActionResult GetAll()
        {
            throw new Exception();
            
            return Ok(_productsService.GetAll());
        }
```

We request 3 times the API gateway and starting from the 4th call, you can expect to get _502 Bad Gateway_ error immediately from the gateway as we set `break_response_code` to 502 in the `api-breaker` plugin configuration. 

Example curl command:

``` bash
curl http://127.0.0.1:9080/api/products -i
```

## Related resources

➔ [Manage .NET Microservices APIs with Apache APISIX API Gateway](https://medium.com/@ApacheAPISIX/manage-net-microservices-apis-with-apache-apisix-api-gateway-13e85853876f).

➔ [API Gateway Caching for ASP.NET Core WEB API](https://medium.com/apache-apisix/api-gateway-caching-for-asp-net-core-web-api-e7ce01ecb9ed)

## Recommended content 💁

➔ Watch Video Tutorial [Getting Started with Apache APISIX](https://youtu.be/dUOjJkb61so).

➔ Watch Video Tutorial [Manage .NET Microservice API with Apache APISIX API Gateway](https://youtu.be/nU8-uQkAHA4).

➔ Read the blog post [Overview of Apache APISIX API Gateway Plugins](https://dev.to/apisix/overview-of-apache-apisix-api-gateway-plugins-2m8o).

➔ Read the blog post [Run Apache APISIX on Microsoft Azure Container Instance](https://dev.to/apisix/run-apache-apisix-on-microsoft-azure-container-instance-1gdk).

➔ Read the blog post [API Security with OIDC by using Apache APISIX and Microsoft Azure AD](https://dev.to/apisix/api-security-with-oidc-by-using-apache-apisix-and-microsoft-azure-ad-50h3).

➔ Read the blog post [API Observability with Apache APISIX Plugins](https://dev.to/apisix/apis-observability-with-apache-apisix-plugins-1bnm).


## Community⤵️

🙋 [Join the Apache APISIX Community](https://apisix.apache.org/docs/general/join/) 
🐦 [Follow us on Twitter](https://twitter.com/ApacheAPISIX)
📝 [Find us on Slack](https://join.slack.com/t/the-asf/shared_invite/zt-vlfbf7ch-HkbNHiU_uDlcH_RvaHv9gQ)
📧 [Mail to us](dev@apisix.apache.org) with your questions.

### About the author

Visit my personal blog: [www.iambobur.com](https://www.iambobur.com/)