Microsoft Releases New .NET Rate Limiter in .NET 7 - And It's Amazing!🤯
Tags: dotnet, news, beginners, csharp
Published: 2022-07-20 15:11:00

It’s official — Microsoft has released the [**new .NET Rate Limiter in .NET 7**](https://devblogs.microsoft.com/dotnet/announcing-rate-limiting-for-dotnet/)! It’s amazing, and it will avoid any problems you have had with rate limiting in the past! Check out this blog post to learn more about it!

As far as performance goes, this new .NET rate limiter has been designed to handle **thousands of concurrent requests** with minimal overhead — making it perfect for high traffic sites with millions of active users.

First of all, before we start talking about the amazing features that Microsoft’s rate limiting brings to .NET 7, we need to understand what rate limiting is, what it is for and how it works.

---

## What is Rate Limiting?

Before we are building our project (.NET _API, Blazor application or_whatever…) we must make sure that we **don’t have any security problem** and the app throttling has already a long time. That’s why this _rate limiting_ concept is not new. Rate limiting is about (_as its name suggests_) limiting or **restricting access to certain resources in a specific time window** by setting maximum access rates.

To explain this concept in a more practical way, imagine that you have an application that connects to a database through an API. Let’s assume that this database and API is not hosted on a quantum super server and we know that it can handle about 100,000 requests daily. At this point many developers will be wondering things like:

-   Will the API and database be able to handle more than 100,000 requests daily?
-   What would happen if it exceeded that number of requests?
-   If it can’t handle any more requests, will it go down?

All these questions raise doubts as to whether the database and API have the capacity to handle more requests.

This is why the concept of rate limiting has been created. A rate limiter (_using different types of algorithms_) would accept, in this case, 100,000 requests each day and deny or block any request that exceeds that number. In this way we could ensure that the service is never saturated.

If rate limiting is not implemented in our developments we create a **nice gateway for any attacker** who wants to harm our application or service (_or even unintentionally by excessive requests!_).

Implementing rate limiting within .NET services and controlling the maximum number of requests will **prevent possible API and database saturation** (_API limit rate_), but this is not the only problem it solves.

---

## Why do we need and use Rate Limiting?

As I said before, the problem of saturation of a service or functionality can not only be due to an involuntary action. **It can also be intentionally** thought and performed by a hacker or attacker causing a DoS (_Denial_ of _Service_) attack.

If we assume the case where we have a public API, which can be accessed by many clients at the same time. Even if most of them are legitimate users, just by having the API or public service opens the possibility of attacks from the outside.

This **depends on the requirements that the API may need**, as the throttling may affect a specific endpoint or all endpoints.

Using API rate limit will also help **protect us from possible Dos attacks** by human attackers or even bot attacks, which are multiple infected computers controlled by the attacker (_botnets_).

These controls can limit by IP address (_commonly_), by user or by any identifier.

Another (_more commercial_) use of Rate Limiting is to offer a charge or **subscription for using a service or API (**_called_ **_pay as you go_**). In this way, many IaaS companies and cloud providers that offer public APIs ensure that their users and customers do not use the service more than they have paid for.

Now that the concept of Rate Limiting is clear, let’s see and explain the new Rate Limiter that Microsoft has released in .NET 7.

---

## .NET Rate Limiting

First of all, let’s remember that we have [AspNetCoreRateLimit](https://www.nuget.org/packages/AspNetCoreRateLimit/) nuget package for a MVC app or web API and it’s also in [Github](https://github.com/stefanprodan/AspNetCoreRateLimit) by [Stefan Prodan](https://github.com/stefanprodan). This new Rate Limiter is integrated in .NET 7 and [according to Microsoft](https://devblogs.microsoft.com/dotnet/announcing-rate-limiting-for-dotnet/) it will help us to keep the traffic of our application on a safe level and will prevent us from overwhelming the application:

> “Rate limiting provides a way to protect a resource in order to avoid overwhelming your app and keep traffic at a safe level.”

There are actually multiple combination of algorithms and different ways to control the flow of all the requests that a .NET application may have and [Microsoft has decided to present the 4 main algorithms](https://devblogs.microsoft.com/dotnet/announcing-rate-limiting-for-dotnet/) for .NET applications provided in .NET 7.

---

### Concurrency limit

[Concurrency limiter](https://devblogs.microsoft.com/dotnet/announcing-rate-limiting-for-dotnet/#concurrency-limit) is the “_simplest_” solution for rate limiting. This limiter presented by Microsoft is in charge of **limiting the maximum number of concurrent requests.** By specifying a limit number, the limiter will deny the next request because it has exceeded the maximum number of allowed requests.

Imagine you set the limit to **50 concurrent requests**, which would mean that **only 50 requests would be allowed** at a time.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ce3a54k2rl5tv0zpjbtq.png)
 

If for any reason a **51st request is generated**, it would be denied for exceeding the specified limit.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/sjvp6op3qti7duiipwjr.png)
 

Using [RateLimitLease](https://devblogs.microsoft.com/dotnet/announcing-rate-limiting-for-dotnet/#ratelimiter-apis) with `RateLimiter` class, we can increase the number of requests allowed each time a request is completed. Let’s check this code example:

```csharp
public abstract class RateLimiter : IAsyncDisposable, IDisposable
{
    public abstract int GetAvailablePermits();
    public abstract TimeSpan? IdleDuration { get; }

    public RateLimitLease Acquire(int permitCount = 1);
    public ValueTask<RateLimitLease> WaitAsync(int permitCount = 1, CancellationToken cancellationToken = default);

    public void Dispose();
    public ValueTask DisposeAsync();
}
```

RateLimitLease is part of `[System.Threading.RateLimiting](https://www.nuget.org/packages/System.Threading.RateLimiting)`, the new nuget package that provides the [built-in “_primitives_” and algorithms](https://devblogs.microsoft.com/dotnet/announcing-rate-limiting-for-dotnet/#ratelimiter-apis) for creating and configuring rate limiters and is included in .NET 7.

---

### Token bucket limit

The [**token bucket**](https://devblogs.microsoft.com/dotnet/announcing-rate-limiting-for-dotnet/#token-bucket-limit) is the second algorithm released by Microsoft. This algorithm **limits the number of requests based on a defined amount** of allowed requests. [As Microsoft says](https://devblogs.microsoft.com/dotnet/announcing-rate-limiting-for-dotnet/#token-bucket-limit), its name describes how it works (_a bit abstactly_) but yes, let’s understand it with a simple example.

Suppose we have an application and this application has an imaginary bucket. This bucket has a limit of tokens (_requests to the application_) and only **50** **tokens** can fit in it, for example.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/leio3szppxf4tb6rezew.png)
 

If a user comes and makes a request (_1 token_), this will be consumed from the bucket and **49 tokens will remain.**

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/58zgocsdl6vwzaom0maz.png)
 

Now let’s imagine that an attacker comes with a botnet and generates **100 tokens** (_100 requests_). As there were **49 free tokens left** before, only the first **49 of the 100 requests** made by the attacker will be processed and the remaining **51 will be denied.**

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/cr7mckatckl2mhe9isqr.png)
 

In this current scenario, no request will be processed until some free token is generated in the bucket. This process happens every minute.

---

### Fixed window limit

The third algorithm released by Microsoft in the new .NET rate limiter is [**fixed window limit**](https://devblogs.microsoft.com/dotnet/announcing-rate-limiting-for-dotnet/#fixed-window-limit). This algorithm is somewhat similar in some respects to token bucket limit but it does have its differences.

Fixed window limit is in charge of allowing a certain amount of requests through a fixed window in a given time.

To explain it in a more practical way I will reuse [Microsoft’s example](https://devblogs.microsoft.com/dotnet/announcing-rate-limiting-for-dotnet/#fixed-window-limit) of the cinema.

Let’s imagine that we have in front of us a cinema with **50 seats** (_maximum capacity of 50_). Now, in this cinema is projected the Fast and Furious 9 movie that lasts 2h 15m.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jguhu8gxq81agfa0jbr3.png)
 

**50 people enter** and watch the movie, but at the same time **another 50 people can queue up** to watch the next session.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/79cmyz82u8vat9cw9wuo.png)
 

Once the 2h 15m of the first session is over, the 50 people who were watching the movie will leave and leave the **50 seats free for the next 50 people** and so on.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xteykpyx79yo4vfm5pxs.png)
 

Let’s say it goes in cycles of 50 at a time in this example.

---

### Sliding window limit

The fourth algorithm for rate limiting in .NET is [**sliding window**](https://devblogs.microsoft.com/dotnet/announcing-rate-limiting-for-dotnet/#sliding-window-limit). The sliding window limit is similar to fixed window limit but has an addition of divisions (_also known as_ **_segments_**).

Let’s understand it based on the [example provided by Microsoft](https://devblogs.microsoft.com/dotnet/announcing-rate-limiting-for-dotnet/#sliding-window-limit) again.

Let’s imagine that we have a **window of 2 hours**, this window is divided into **2 segments of 1 hour** each and can accept a **maximum of 40 requests** simultaneously. Now, we also have an index that will point to the current segment of the window (_the most recent or new one_).

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/g6w8937uprgh0z1ihc0e.png)
 

Now in the first hour we receive **20 requests**. These 20 requests will go directly to the current segment pointed to by the index.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wfryb6780hh3guu4a2lq.png)
 

When the first hour has passed, the window index will go to the next segment with a total **capacity of 20 requests** (_since the other 20 are occupied in the previous segment_).

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gccbxr2iosbc4fnfk7jz.png)
 

In the second hour **10 new requests** come in, which go back to the segment indicated by the index and the number of available requests would **drop to 10 requests.**

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/j4no0ha6odo7yakgctn9.png)
 

Again, when this second hour has passed, the window index will move to the next position and, as the first 20 requests that entered in the first hour have been left out of the window, those **20 requests are recovered** leaving a total of **30 free requests** at the moment.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8oxfeoldcg57fr6n9xrp.png)
 

Having explained this, we also have a couple of abstractions inside the nuget package [System.Threading.RateLimiting](https://www.nuget.org/packages/System.Threading.RateLimiting) like `PartitionedRateLimiter` or `RateLimiting middleware` and let’s explain them to understand them and be able to move our .NET applications and APIs away from throttle.

---

## PartitionedRateLimiter

As I said a moment ago, `PartitionedRateLimiter` belongs to the popular nuget package `System.Threading.RateLimiting`. It is similar in some aspects to `RateLimiter` that I explained above, but the main difference is that `PartitionedRateLimiter` allows to have arguments for methods on [TResource](https://docs.microsoft.com/en-us/dotnet/api/microsoft.visualstudio.utilities.reusableresourcestore-1?view=visualstudiosdk-2022) instances.

As Microsoft explains, [Acquire](https://docs.microsoft.com/en-us/dotnet/api/microsoft.visualstudio.utilities.reusableresourcestore-2.acquire?view=visualstudiosdk-2022) becomes:

```csharp
Acquire(TResource resourceID, int permitCount = 1)
```

Let’s see how [Microsoft provides the example](https://devblogs.microsoft.com/dotnet/announcing-rate-limiting-for-dotnet/#partitionedratelimiter):

```csharp
enum MyPolicyEnum
{
    One,
    Two,
    Admin,
    Default
}

PartitionedRateLimiter<string> limiter = PartitionedRateLimiter.Create<string, MyPolicyEnum>(resource =>
{
    if (resource == "Policy1")
    {
        return RateLimitPartition.Create(MyPolicyEnum.One, key => new MyCustomLimiter());
    }
    else if (resource == "Policy2")
    {
        return RateLimitPartition.CreateConcurrencyLimiter(MyPolicyEnum.Two, key =>
            new ConcurrencyLimiterOptions(permitLimit: 2, queueProcessingOrder: QueueProcessingOrder.OldestFirst, queueLimit: 2));
    }
    else if (resource == "Admin")
    {
        return RateLimitPartition.CreateNoLimiter(MyPolicyEnum.Admin);
    }
    else
    {
        return RateLimitPartition.CreateTokenBucketLimiter(MyPolicyEnum.Default, key =>
            new TokenBucketRateLimiterOptions(tokenLimit: 5, queueProcessingOrder: QueueProcessingOrder.OldestFirst,
                queueLimit: 1, replenishmentPeriod: TimeSpan.FromSeconds(5), tokensPerPeriod: 1, autoReplenishment: true));
    }
});
RateLimitLease lease = limiter.Acquire(resourceID: "Policy1", permitCount: 1);

// ...

RateLimitLease lease = limiter.Acquire(resourceID: "Policy2", permitCount: 1);

// ...

RateLimitLease lease = limiter.Acquire(resourceID: "Admin", permitCount: 12345678);

// ...

RateLimitLease lease = limiter.Acquire(resourceID: "other value", permitCount: 1);
```

If you want to know more, please check the original source: [**Microsoft PartitionedRateLimiter**](https://devblogs.microsoft.com/dotnet/announcing-rate-limiting-for-dotnet/#partitionedratelimiter)**.**

---

## RateLimiting middleware

[RateLimiting middleware](https://devblogs.microsoft.com/dotnet/announcing-rate-limiting-for-dotnet/#ratelimiting-middleware) is also part of `System.Threading.RateLimiting` and its main function is to be able to configure and add custom rate limiting policies to endpoints.

Going back to the [Microsoft examples](https://devblogs.microsoft.com/dotnet/announcing-rate-limiting-for-dotnet/#ratelimiting-middleware), in this case we can find that:

```csharp
Func<HttpContext, RateLimitPartition<TPartitionKey>>
```

Has the same functionality as `PartionedRateLimiter`.

It also has the option `RateLimiterOptions`, which includes `RejectionStatusCode` with which we can **return a status code** (_default is 503_).

```csharp
new RateLimiterOptions()
{
    OnRejected = (context, cancellationToken) =>
    {
        context.HttpContext.StatusCode = StatusCodes.Status429TooManyRequests;
        return new ValueTask();
    }
};
```

If you want to know more, please check the original source: [**Microsoft RateLimiting middleware**](https://devblogs.microsoft.com/dotnet/announcing-rate-limiting-for-dotnet/#ratelimiting-middleware)**.**

I hope these explanations have cleared up a lot of doubts about .NET rate limiting. I would like to know if any developer is going to implement or has already implemented the .NET rate limiter for API limiting in API.