Can we implement asynchronous circuit breakers with serverless architecture?
Tags: todayilearned, aws, serverless
Published: 2023-12-22 14:38:42

## Scenario
We are attempting to make a post request that involves calling a third-party **API**, the application is serverless and an [EDA](https://www.youtube.com/watch?v=KXR17uwLEC8&ab_channel=AWSEvents), so everything. 
Nevertheless, this API occasionally experiences unexpected outages. We are currently exploring possible solutions to address this issue. 
A typical "trust no one" problem can arise when integrating with external partners or suppliers, and the first thing that occurred in my mind was whether I should use a [circuit breaker](https://martinfowler.com/bliki/CircuitBreaker.html), However, this is only used in **[synchronous](https://stackoverflow.com/questions/748175/asynchronous-vs-synchronous-execution-what-is-the-difference)** applications, and my application is entirely **[asynchronous](https://stackoverflow.com/questions/748175/asynchronous-vs-synchronous-execution-what-is-the-difference)**. How can I achieve it? First of all, let's recap.

## What is a [circuit breaker](https://martinfowler.com/bliki/CircuitBreaker.html)?
Visualise your app as a rock star holding an arena concert, and the Circuit Breaker is that guy backstage who knows when you've just gone too far. You'll fry the stage wiring if you keep up like that.
![a rock concert](https://i.giphy.com/cqwHhVm0BAyyc.webp)

So, your application is at the top of its lungs, each note is a service call. But then suddenly, the guitar amp (your database) wants to rest. Rather than having your app crying for more solos, like a cool roadie, the Circuit Breaker watches out after things, saying:

> "Calm down there. The amp needs time to breathe."

It's like making sure your concert doesn't descend into a riotous frenzy, allowing the guitar amp (your company) to cool down. 

> Who wants to hear the tale of a concert fiasco?

In the curtain call, the Circuit Breaker is your backup plan for staying gigable even when glitches strike you out midair. So, you can rock on, but with a safety net.

### Backing to the tech buzzwords

> In a nutshell, a circuit breaker is designed for **synchronous** applications because it can interrupt the sequence of operations when one fails swiftly. In **synchronous** settings, operations occur linearly, allowing the circuit breaker to prevent cascading failures effectively.
 
_In contrast, **asynchronous** applications, with concurrent and independent operations, often rely on different error-handling mechanisms better suited to their non-linear and non-blocking nature_

## Is it possible to accomplish an asynchronous circuit breaker?
![someone thinking](https://i.giphy.com/1jl173guBKkbvC03rQ.webp)
I've been brainstorming and coming up with some out-of-the-box ways to achieve our goal. 
I'm excited to share these two approaches with you!

## 1 - [A self-healing Lambda function that adapts its throughput based on performance](https://theburningmonk.com/2019/05/a-self-healing-kinesis-function-that-adapts-its-throughput-based-on-performance/)
This unbeatable article goes through a way how to handle third parties in an approach using _kinesis_ and a _ventilator lambda function_, below it's a small piece of this treasure written by the marvellous _serverless hero_ [Yan Cui](https://theburningmonk.com/hire-me/) at the OG [**BurningMonk** Blog](https://theburningmonk.com).
> The ventilator function can, therefore, self-adjust its batch size by updating the Kinesis event source mapping.
> As the provider API’s response time goes up, the ventilator can respond by reducing its batch size. This gives the API a chance to catch its breath and recover. When the response time returns to acceptable levels, then the ventilator can gradually increase its batch size back to previous levels

## 2 - Take a break for your downstream 😅

After a quick catch-up with a 🇵🇱 Polish Wizard a.k.a [Łukasz Kiedrowski ](https://www.linkedin.com/in/%C5%82ukasz-kiedrowski-6b389871/)🧙‍♂️, giving me some wisdom, he sent me a quote that basically blew my mind. This quote, written by him, clarified the perspective I was wondering about, a way to achieve the wanted async circuit ~~breaker~~ using basically only a [DLQ](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-dead-letter-queues.html) 

![luke's quote](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4gq2gr6w6mb1e9fqx3ct.png)

> In EDA architectures DLQ (maxReceiveCount + dynamic visibility timeout) will act as a circuit breaker in Synchronous architectures

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/k7fc7v1guwkxokgj0sdp.png)

**This approach will protect you from the third party but won't act, in fact, as a "breaker". It will reduce the invocations instead of breaking it, protecting you from third-party failure**.


This post presents some alternatives for implementing a circuit breaker within an asynchronous serverless environment. 
Please let me know if you would like to see an implementation of this. 

Of course, I would highly appreciate constructive discussion. If you don't agree with anything mentioned here, please feel free to leave your comments below.


Cheers!