Per user rate limiting with OpenID connect and Istio in Kubernetes
Tags: architecture, kubernetes, istio
Published: 2018-07-09 14:52:26

---
title: Per user rate limiting with OpenID connect and Istio in Kubernetes
published: true
tags: Architecture,Kubernetes,Istio
description: How to do rate limiting with Istio on Kubernetes a per user basis, using OpenID Connect to identify it. 
canonical_url: https://akvo.org/blog/per-user-rate-limiting-with-openid-connect-and-istio-in-kubernetes/
cover_image: https://danlebrero.com/images/blog/istio-rate-limit.jpg
---

*This article originally appeared on [Akvo's blog](https://akvo.org/blog/per-user-rate-limiting-with-openid-connect-and-istio-in-kubernetes/)*


To make sure that each of our partners is able to use [Akvo’s API](https://github.com/akvo/akvo-flow-api/wiki/Akvo-Flow-REST-API), we need to ensure that nobody is able to abuse it. We want to ensure that each partner has access to a fair share of the servers’ resources.

In the case of HTTP APIs, this usually means limiting the rate at which partners can make requests. A system that performs rate limiting needs to:

* Identify who is making the HTTP request.
* Count how many requests each user has made.
* Reject any user request once that user has depleted their allotment.

There are plenty of open source products and libraries out there that you can choose from, but we decided to give [Istio](https://istio.io/) a try.

For such a task, Istio is a little bit heavy-handed. However, since Istio is a [service mesh](http://philcalcado.com/2017/08/03/pattern_service_mesh.html), it also provides routing, load balancing, blue/green deployment, canary releases, traffic forking, circuit breakers, timeouts, network fault injection and telemetry. What’s more, it also offers [internal TLS encryption](https://istio.io/docs/concepts/security/mutual-tls.html) and [Role-Based access control](https://istio.io/docs/concepts/security/rbac.html), which is very important for us given our commitment to the [upcoming GDPR legislation](https://akvo.org/blog/gdpr-becoming-compliant/).

## Identifying the user

Akvo’s API already uses the [OpenID connect standard](https://github.com/akvo/akvo-flow-api/wiki/Authentication) and Istio comes with a handy [JWT-auth filter](https://github.com/istio/proxy/tree/master/src/envoy/http/jwt_auth), so we just need to configure the filter to point to our OpenID provider:

```yaml
apiVersion: config.istio.io/v1alpha2
kind: EndUserAuthenticationPolicySpec
metadata:
  name: flow-api-auth-policy
  namespace: default
spec:
  jwts:
    - issuer: https://kc.akvotest.org/auth/realms/akvo
      jwks_uri: https://kc.akvotest.org/auth/realms/akvo/protocol/openid-connect/certs
```

And then we need to tell Istio to apply the authentication spec to our backend service:

```yaml
apiVersion: config.istio.io/v1alpha2
kind: EndUserAuthenticationPolicySpecBinding
metadata:
  name: flow-api-auth-policy-binding
  namespace: default
spec:
  policies:
    - name: flow-api-auth-policy
      namespace: default
  services:
    - name: flow-api
      namespace: default
```

With this, if there is a JWT access token present in the request, Istio will validate it and will add the principal to the request, but if there is no token, the requests will still go through.

## Enforcing a user
   
Given that any access to the API must be done with an access token, we can add a policy rule to enforce it. To configure a policy we will need:
   
A handler, which in this particular case is a [Denier adapter](https://archive.istio.io/v0.6/docs/reference/config/adapters/denier) that will return a 401:

```yaml
apiVersion: "config.istio.io/v1alpha2"
kind: denier
metadata:
  name: flow-api-handler
  namespace: default
spec:
  status:
    code: 16
    message: You are not authorized to access the service
```
    
An instance, which in this case is a [Check Nothing](https://archive.istio.io/v0.6/docs/reference/config/template/checknothing) template as the handler requires no data:

```yaml
apiVersion: "config.istio.io/v1alpha2"
kind: rule
metadata:
  name: flow-api-deny
  namespace: default
spec:
  match: destination.labels["run"] == "flow-api" && (request.auth.principal|"unauthorized") == "unauthorized"
  actions:
  - handler: flow-api-handler.denier.default
    instances: [flow-api-denyrequest.checknothing.default]
```
 
See the [Istio documentation](https://archive.istio.io/v0.6/docs/concepts/policy-and-control/mixer-config) if you are not familiar with the handler, instance or rule concepts.

## Counting usage

First, we need to define what we want to count:

```yaml
apiVersion: config.istio.io/v1alpha2
kind: quota
metadata:
  name: requestcount
  namespace: istio-system
spec:
  dimensions:
    destination: destination.labels["run"] | destination.service | "unknown"
    user: request.auth.principal|"unauthorized"
```

We are using two dimensions, the user and the destination service so that we can have different limits for different backend services.

To do the actual counting:

```yaml
apiVersion: config.istio.io/v1alpha2
kind: QuotaSpec
metadata:
  name: flow-api-quota
  namespace: default
spec:
  rules:
    - quotas:
        - quota: requestcount.quota.istio-system
          charge: 1
```

Istio rate limiting gives you the flexibility to "charge" more for requests that could be more expensive to execute, but in our case, we’ve decided to treat all the requests the same.

And last, we need to wire the counting with the backend service:

```yaml
apiVersion: config.istio.io/v1alpha2
kind: QuotaSpecBinding
metadata:
  name: flow-api-quota-binding
  namespace: default
spec:
  quotaSpecs:
    - name: flow-api-quota
      namespace: default
  services:
    - name: flow-api
      namespace: default
```

## Enforcing usage quotas

Now that we know who you are and how to count, we need to define what is a reasonable usage. We do this through a [Memory Quota adapter](https://archive.istio.io/v0.6/docs/reference/config/adapters/memquota):

```yaml
apiVersion: config.istio.io/v1alpha2
kind: memquota
metadata:
  name: handler
  namespace: istio-system
spec:
  quotas:
  - name: requestcount.quota.istio-system
    maxAmount: 60
    validDuration: 10s
    overrides:
    - dimensions:
        destination: flow-api
      maxAmount: 20
      validDuration: 10s
```

So we allow up to ten requests per second for each user, except if the requests go to the Flow API, in which case we allow up to two requests per second.

Note that in production you will want to use a [Redis Quota](https://archive.istio.io/v0.6/docs/reference/config/adapters/redisquota) instead of a Memory Quota, as the Memory Quota is ephemeral and local to the Mixer instance.

Finally, we create a policy rule to wire up the quota with the counters:   
   
   
```yaml
apiVersion: config.istio.io/v1alpha2
kind: rule
metadata:
  name: quota
  namespace: istio-system
spec:
  actions:
  - handler: handler.memquota
    instances:
    - requestcount.quota
```
   
## Testing

Now, we can check that everything is working as expected and that no user is able to abuse the system. For the testing, we changed the quota to one request every three seconds. Here is the result:

<iframe src="https://player.vimeo.com/video/260455239?title=0&byline=0&portrait=0" width="640" height="360" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>
   
You can find a version of the test script [here](https://github.com/akvo/akvo-platform/blob/issue-57/rate-limit-istio/istio/simple-check.sh) and all the code above [here](https://github.com/akvo/akvo-platform/tree/b17ed526fa48d22e7ef7a2f37195302119309dca/istio/k8s). 

## Are we done?

Istio allows us to ensure that all of our partners get a fair share of the resources, with a little bit of configuration and without having to modify or change any of our existing code, which is a big plus.

But rate limiting is just one part of making Akvo’s platforms more stable. Istio also comes with a lot more goodies to add to that stability, and to make it more secure, which for sure we will investigate in the near future.