Private serverless APIs in AWS
Published: 2021-02-17 14:29:39

Our application landscaping is growing to more interfacing applications; A website, mobile apps and customer service tooling just to give a few examples. We've been using the [AWS Serverless Application Model](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/what-is-sam.html) to quickly create stacks that are consumable by multiple applications. One downside of using AWS SAM is that the API Gateway endpoints are public by default. We protect these endpoints against abuse via an authentication, but for some endpoints we would like to have an additional layer of protected by making these endpoints only available from inside our [VPC](https://aws.amazon.com/vpc/). In other words make our endpoints private.

**NB** This article specifically outlines making an _AWS Serverless Application Model_ stack private. A serverless infrastructure has a great amount of similarities with `AWS::ApiGateway::RestApi`, but is still subtly different. I like to look at it as a simplification of `AWS::ApiGateway::RestApi` and `AWS::Lambda::Function`.

## A serverless stack
A typical serverless stack consists of two parts: an [AWS Lambda](https://aws.amazon.com/lambda/) function and a trigger. This trigger can be anything from an upload on an S3 bucket, a message being pushed on a SQS or a call to an HTTP endpoint. In this last scenario the endpoint is defined as an [API Gateway](https://aws.amazon.com/api-gateway/) resource.

We use [Cloudformation](https://aws.amazon.com/cloudformation/) to define our stacks. Let's consider a minimal function that allows retrieval of entities of the type `entity` via the call `GET /entities`:

```yaml
Parameters:
  Environment:
    Type : "String"
    Default: "development"
    Description: "Environment in which resources are deployed."

Resources:
  # Lambdas
  GetEntitiesFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: index.getEntities
      FunctionName: !Sub "${AWS::StackName}-get-entities"
      CodeUri: .
      Role: "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
      Events:
        GetResource:
          Type: Api
          Properties:
            Path: /entities
            Method: get
            RestApiId: !Ref "ServerlessApi"

  # Api gateway
  ServerlessApi:
    Type: "AWS::Serverless::Api"
    Properties:
      StageName: !Ref "Environment"
```

When deployed this should result in a serverless stack with a single endpoint that is publicly available for everyone that knows the url. 

## VPC endpoint
Making an API Gateway private requires a VPC. Every stack that requires access to the private API must be part of the VPC. Benefit of having a VPC is that access control is centralized in the VPC configuration. 

In order to make the API private the traffic will have to pass through the VPC. This is done via a VPC endpoint:

```yaml
Resources:
  APIGatewayVpcEndpoint:
    Type: "AWS::EC2::VPCEndpoint"
    Properties:
      SubnetIds:
        - !ImportValue "exported-subnet-id"
      SecurityGroupIds:
        - !Ref "resource of type AWS::EC2::SecurityGroup"
      ServiceName: "com.amazonaws.eu-west-1.execute-api"
      VpcId: !ImportValue "exported-vpc-id"
      VpcEndpointType: "Interface"
      PrivateDnsEnabled: false
```

Disabling the private DNS is a deliberate choice. The reason for that is explained later in this article.

## Marking an API private
By default API Gateway resources are publicly available. In 2018 [AWS introduced the possibility to mark API Gateway resources as private](https://aws.amazon.com/blogs/compute/introducing-amazon-api-gateway-private-endpoints/). Marking an API Gateway as private is easily done in the Cloudformation definition of our stack:

```yaml
Resources:
  # ...
  
  # Api gateway
  ServerlessApi:
    Type: "AWS::Serverless::Api"
    Properties:
      StageName: !Ref "Environment"
      EndpointConfiguration:
        Type: "PRIVATE"
        VPCEndpointIds:
          - !ImportValue "infrastructure-api-gateway-vpc-endpoint"
      Auth:
        ResourcePolicy:
          IntrinsicVpceWhitelist:
            - !ImportValue "infrastructure-api-gateway-vpc-endpoint"
```

**NB** This example imports a VPC endpoint. We try to separate stacks as much as possible for ownership and maintainability. In a separate _infrastructure_ stack the VPC endpoint(s) are defined and the required identifiers are exported from those stacks. That way every other stack can use them while the hosting team still has the possibility to make changes to the endpoint. 

The [`EndpointConfiguration`](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-resource-api.html#sam-api-endpointconfiguration) tells AWS that the API should no longer be publicly available. A required property of `EndPointConfiguration` is a list of VPC endpoints.

Next to marking the API private we also need to tell the API it is allowed to be called from the endpoint. This is done via a resource policy. In a `AWS::Serverless::Api` this is done via the `Auth` property. 

When we now deploy this stack, the API will no longer be publicly available.

## Consuming the private API
The downside of making an API Gateway resource is that consuming becomes a little bit more complicated. There are two methods that work out-of-the-box. Those methods use the VPC endpoint as entry for the request: 

```
curl -v https://{public-dns-hostname}.execute-api.{region}.vpce.amazonaws.com/{stage}/{path}
```

The `{public-dns-hostname}` of the VPC endpoint is visible in the AWS Console. For the following examples I will call the `/entities` endpoint defined in the Cloudformation for region `eu-west-1` and stage `testing`.

The first method uses the `Host` header:

```
curl -v https://vpce-01234567abcdef012-01234567.execute-api.eu-west-1.vpce.amazonaws.com/testing/entities \
-H 'Host: 01234567ab.execute-api.eu-west-1.amazonaws.com'
```

The second method uses the `x-apigw-api-id` header:

```
curl -v https://vpce-01234567abcdef012-01234567.execute-api.eu-west-1.vpce.amazonaws.com/testing/entities \
-H 'x-apigw-api-id: 01234567ab'
```

Both the host and the API id can be found in the AWS Console after deploying your stack.

There are other options to call your private endpoints that a bit more user-friendly, but that have their own caveats. You can enable the "Private DNS Name" while creating an interface VPC endpoint for API Gateway, but this also means that the VPC where the VPC endpoint is present will no longer be able to access public APIs:

> When you select the Enable Private DNS Name option while creating an interface VPC endpoint for API Gateway, the VPC where the VPC Endpoint is present won't be able to access public (edge-optimized and regional) APIs. For more information, see [Why can't I connect to my public API from an API Gateway VPC endpoint?](https://aws.amazon.com/premiumsupport/knowledge-center/api-gateway-vpc-connections/).

Source: [Invoking your private API using private DNS names](https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-private-api-test-invoke-url.html#w7aac17b9c33c26c11)

As we have both public and private APIs and we also consume public APIs from within our VPC we are not able to use this approach.

Another option is to use a `Route53` alias. We have not yet tried this, so I cannot make any comments on this option.

## Conclusion
The SAM model offered by AWS allows for fast development of applications as you don't have to worry too much about your infrastructure choices. AWS will scale if needed. Building a modest sized API can be achieved using the tandem Lambda and API Gateway, but this will result in a public endpoint by default.

If you want to utilize the benefits of the SAM model, but you don't want to expose your API to the public you can make your API private. This will increase security on your endpoint, but it will also introduce some downsides like the necessity to have a VPC and a VPC endpoint.