AWS Elasticsearch Access with Serverless Lambda
Tags: aws, elasticsearch, serverless, cognito
Published: 2020-11-08 18:39:02

Elasticsearch is a well known search solution and AWS offers a [fully-managed service](https://aws.amazon.com/elasticsearch-service/) for it. The managed service has the exact same API to interact with just like an unmanaged cluster which is great because you can use all the available tooling as it is.

There are a couple of different options when it comes to deploying a business service backed by [Elasticsearch](http://elastic.co/). For example,
1. You can expose the ES endpoint directly so that clients can invoke the ES rest API.
2. Create a business centric API around ES and expose it as the only way to consume ES service.

Which approach is better than the other depends on your needs. Additionally, AWS ES offers Kibana to visualize your search data. To protect Kibana from unauthorized access, you can integrate it with Cognito. Once you do that, only authenticated users from your Cognito User Pool will be able to access the Kibana Dashboard.

In my case, I wanted to deploy an ES domain on AWS with lambda and API Gateway in front. And I am using [Serverless Framework](https://www.serverless.com/) to manage my lambda functions. Additionally, I wanted to secure the access to Kibana using Cognito User Pools. When I first started with this, I face a lot of challenges to set up my ES domain in AWS with proper access policies. Merely defining an [Identity Based Policy](https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-ac.html#es-ac-types-identity) did not work.  There is a great article by [Jon Handler](https://aws.amazon.com/blogs/database/author/handler/) on [Getting started with Elasticsearch and Cognito](https://aws.amazon.com/blogs/database/get-started-with-amazon-elasticsearch-service-use-amazon-cognito-for-kibana-access-control/) which talks about a lot of things in great detail. However, there were still some missing pieces given the limited knowledge I had at the time. This post outlines some of the concepts which may help you understand what makes Elasticsearch access control different than other [IAM policies](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies.html) you are used to defining in your `serverless.yml`.

## Resource vs Identity Based Policy
This is one of the most important concepts I think, which you’d have to understand before you configure Elasticsearch Access Policy.

A [*Resource Based Policy*](https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-ac.html#es-ac-types-resource) is attached to the resource which you are trying to secure. That means, you have to know upfront **who** can access the resource. The *who* may mean many different things depending on what you want to do. For example, you can allow everything under one [AWS Account ID](https://docs.aws.amazon.com/IAM/latest/UserGuide/console_account-alias.html) to have access to ES.

An [*Identity Based Policy*](https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-ac.html#es-ac-types-identity) is attached to the entity accessing your resource. For example an indexing lambda function to insert data in Elasticsearch. 

> The main difference between the two is that for the *Resource Based Policy*, you have to plan ahead.

One of the many ways you can do this is by [creating an IAM Role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create.html), assigning it to your lambda and granting it access in the ES Access Policy. So the policy document may look like the following:

```
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:sts::400176810461:...ROLE ARN"
      },
      "Action": "es:*",
      "Resource": "arn:aws:es:us-east-1:400176810461:domain/my-es-domain/*"
    }
  ]
}
```

Notice the [￼`Principal`￼](https://docs.aws.amazon.com/IAM/latest/UserGuide/intro-structure.html#intro-structure-principal) key here. It means that we are specifically telling Elasticsearch to allow all access to everyone who is assigned this role.

## Initial plan
In my first iteration, I planned this:
1. Create two separate roles (one for read and other for read and write). The actions in the role don’t matter (I think) because ES will override those.
2. Add these additional roles to my `serverless.yml` and attach it to functions.

But I couldn’t find a way to keep the default roles which *Serverless* creates and add this additional role. So that plan failed.

## Second Iteration
Then I created an access policy in Elasticsearch which allowed access to everything under my account. 
```
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::123456789012:root"
      },
      "Action": "es:*",
      "Resource": "arn:aws:es:us-east-1:400176810461:domain/my-es-domain/*"
    }
  ]
}
```

But this was still not enough as I kept getting error that the lambda didn’t have access to invoke `es:HttpGet`.

## Making it work with Serverless
I finally found the reason why my second attempt failed in the [this section of Identity and Access Management document for Elasticsearch](https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-ac.html#es-ac-conflict). Essentially, if you have a *Resource Based Policy* which allows access to everything under the account, you must also create an *Identity Based Policy* for the entity accessing the service. That means, in addition to my second attempt, I also had to configure the `serverless.yml` with the following block under `iamRoleStatements`

```
- Effect: Allow
  Action:
    - es:*
  Resource:
    - arn:aws:es:us-east-1:400176810461:domain/my-es-domain/*
```

## Signing the requests
As you know, a client (lambda in this case) talks to Elasticsearch via REST APIs. That means, the requests don't go internally in AWS (like calling a lambda via ARN). So how does ES differentiate between a request coming from your lambda vs a request from some other source (because they are both HTTP).

Well, once you allow your lambda to access the Elasticsearch instance, you must sign the HTTP requests with AWS V4 signing as well. Otherwise the request will appear as if it is coming from an unauthorized user. To sign a request in NodeJS, checkout the following two libraries.

1. [aws-elasticsearch-connector](https://github.com/compwright/aws-elasticsearch-connector)
2. [aws4](https://github.com/mhart/aws4)

## Configuring Cognito
The [Getting started with Elasticsearch and Cognito](https://aws.amazon.com/blogs/database/get-started-with-amazon-elasticsearch-service-use-amazon-cognito-for-kibana-access-control/) article mentioned before has a step by step guide to configure Cognito for Kibana. However, I found somethings which didn’t work for me:

1. I didn’t see the need for checking “Enable access to unauthenticated identities”. Because I don’t fully understand the [Cognito Identity Pools](https://docs.aws.amazon.com/cognito/latest/developerguide/identity-pools.html) yet, this seemed dangerous. The permissions that you assign to the `Unauth_Role` don’t matter to ES because the ES access policy supersedes. This is only needed (I think) if you want to create a dashboard in Kibana which is accessible without authentication.
2. If you follow the steps in “Change the Principal to the ARN for the assumed Auth role.”, it will allow Kibana access to only the Identity Pool users. You can always add additional role here but then you may run into the same problem as my attempt 1 about how to assign this extra role to your lambda.

## Summary
- Create an access policy with a *Principal* value which allows everything under your account to access Elasticsearch.
```
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::123456789012:root"
      },
      "Action": "es:*",
      "Resource": "arn:aws:es:us-east-1:400176810461:domain/my-es-domain/*"
    }
  ]
}
```
- Assign necessary permissions to your lambda in the `serverless.yml`.
```
- Effect: Allow
  Action:
    - es:*
  Resource:
    - arn:aws:es:us-east-1:400176810461:domain/my-es-domain/*
```
- Sign your request with V4 signing.