In Depth Guide to Serverless APIs with AWS Lambda and AWS API Gateway (Part 1)
Tags: aws, serverless, api
Published: 2019-07-15 17:25:37

---
title: "In Depth Guide to Serverless APIs with AWS Lambda and AWS API Gateway (Part 1)"
published: true
description: A two-part article about building an API with serverless AWS technology
tags: aws, serverless, api
series: "Serverless API Guide"
cover_image: https://thepracticaldev.s3.amazonaws.com/i/u12himxkgwbj3nx20co7.jpg
canonical_url: https://www.moesif.com/blog/serverless/aws-lambda/In-Depth-Guide-To-Serverless-APIs-With-AWS-Lambda-And-AWS-API-Gateway-Part1/
---

_TL;DR The repository with an example project can be found on [GitHub](https://github.com/kay-is/serverless-api-example)_

Building your software products around an API is THE thing for years now and doing it with serverless technology right from the start seems rather intriguing for many reasons — on-demand pricing, auto-scaling, and less operational overhead.

## Why

When I read what people are talking about serverless technology, I have the feeling there are still many questions open.

How to

1. [use a query string or route parameters?](/blog/serverless/aws-lambda/In-Depth-Guide-To-Serverless-APIs-With-AWS-Lambda-And-AWS-API-Gateway-Part2.md/#how-use-a-query-string-or-route-parameters)
2. [use the params or body of a POST request?](#how-to-access-the-request-body)
3. [set HTTP status codes?](#how-to-set-http-status-codes)
4. [set a response header?](#how-to-set-an-http-response-header)
5. [call a Lambda from a Lambda?](/blog/serverless/aws-lambda/In-Depth-Guide-To-Serverless-APIs-With-AWS-Lambda-And-AWS-API-Gateway-Part2.md/#how-use-a-query-string-or-route-parameters#how-to-call-a-lambda-from-a-lambda)
6. [store keys for third-party APIs?](/blog/serverless/aws-lambda/In-Depth-Guide-To-Serverless-APIs-With-AWS-Lambda-And-AWS-API-Gateway-Part2.md/#how-use-a-query-string-or-route-parameters#how-store-credentials-for-third-party-apis)

So I decided to write an article about how to build an API with serverless technology, specifically AWS Lambda and API-Gateway.

This article is split into two parts.

This one, the first, is about the architecture, setup, and authentication.

The second one is about _actual_ work, uploading images, tagging them with the third-party API, and retrieving the tags and the images later.

## What

We will build a simple RESTful image tagging API. It lets us upload images, tags them automatically with a third-party API called [Clarifai](https://www.clarifai.com/), and stores the tags and image names into Elasticsearch.

The work-flows I have in mind are:

- Signing up and in
- Uploading an image
- Getting all tags of all images
- Getting all images and their tags
- Getting an image by tag

## How

For the API, we use API-Gateway, which is Amazons all-round serverless HTTP solution. It's optimized for RESTful APIs and works as the entry-point for our system.

**Amazon Cognito** handles the authentication. Cognito is a managed serverless authentication, authorization, and data synchronization solution. We use it to sign our users up, and in so we don't have to reinvent the wheel here.

The actual computing work of our API is done by **AWS Lambda**, a **function as a service** solution. Lambda is a serverless event-based system that allows triggering functions when something happens, for example, an HTTP request hit our API, or someone uploaded a file directly to S3.

The images are stored in an **Amazon S3** bucket. S3 is a serverless object-based storage solution. It allows direct access and uploads of files via HTTP and can, as **API-Gateway**, be an event source for **Lambda**.

The tag data and the corresponding image-names are stored in **Amazon Elasticsearch Service**; an AWS managed version of Elasticsearch. Sadly this is not serverless but built on EC2 instances, but there is a free tier for the first 12 months. Elasticsearch is a very flexible document storage and comes with a powerful query language.

A non-AWS service called Clarifai provides image recognition magic. AWS has its service for this, called Rekognition, but by using Clarifai, we can learn how to store third-party API keys.

The whole infrastructure we build is managed by **AWS SAM**, the **Serverless Application Model**. SAM is an extension for AWS CloudFormation that reduces some boilerplate code needed to set up AWS Lambda and API-Gateway resources.

We use **AWS Cloud9** as an IDE because it comes with all the tools and permissions pre-installed to use AWS resources.

## Prerequisites

- A browser
- An AWS account with a Cloud9 environment. _(Setup can be found [here](https://dev.to/kayis/10-easy-steps-to-create-aws-lambda-functions-with-the-serverless-framework--reason-in-aws-cloud9-8d1) in the first 5 steps.)_
- Basic JavaScript knowledge
- Basic Lambda, API-Gateway, and SAM knowledge

## Setup

Let's start by getting a basic serverless API going that just implements a login with the help of AWS SAM, API-Gateway and Cognito.

    mkdir serverless-api
    cd serverless-api
    mkdir functions
    touch template.yaml

First, we create the folder structure and a `template.yaml` file that holds the definition of the infrastructure we create with SAM.

## Implementing the SAM Template

The content of our SAM template is as follows:

```yaml
AWSTemplateFormatVersion: "2010-09-09"
Transform: "AWS::Serverless-2016-10-31"
Description: "An example REST API build with serverless technology"

Globals:
  Function:
    Runtime: nodejs8.10
    Handler: index.handler
    Timeout: 30
    Tags:
      Application: Serverless API

Resources:
  ServerlessApi:
    Type: AWS::Serverless::Api
    Properties:
      StageName: Prod
      Cors: "'*'"
      Auth:
        DefaultAuthorizer: CognitoAuthorizer
        Authorizers:
          CognitoAuthorizer:
            UserPoolArn: !GetAtt UserPool.Arn
      GatewayResponses:
        UNAUTHORIZED:
          StatusCode: 401
          ResponseParameters:
            Headers:
              Access-Control-Expose-Headers: "'WWW-Authenticate'"
              Access-Control-Allow-Origin: "'*'"
              WWW-Authenticate: >-
                'Bearer realm="admin"'

  # ============================== Auth =============================
  UserPool:
    Type: AWS::Cognito::UserPool
    Properties:
      UserPoolName: ApiUserPool
      LambdaConfig:
        PreSignUp: !GetAtt PreSignupFunction.Arn
      Policies:
        PasswordPolicy:
          MinimumLength: 6

  UserPoolClient:
    Type: AWS::Cognito::UserPoolClient
    Properties:
      UserPoolId: !Ref UserPool
      ClientName: ApiUserPoolClient
      GenerateSecret: no

  PreSignupFunction:
    Type: AWS::Serverless::Function
    Properties:
      InlineCode: |
        exports.handler = async event => {
          event.response = { autoConfirmUser: true };
          return event;
        };

  LambdaCognitoUserPoolExecutionPermission:
    Type: AWS::Lambda::Permission
    Properties:
      Action: lambda:InvokeFunction
      FunctionName: !GetAtt PreSignupFunction.Arn
      Principal: cognito-idp.amazonaws.com
      SourceArn: !Sub "arn:${AWS::Partition}:cognito-idp:${AWS::Region}:${AWS::AccountId}:userpool/${UserPool}"

  AuthFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: functions/auth/
      Environment:
        Variables:
          USER_POOL_ID: !Ref UserPool
          USER_POOL_CLIENT_ID: !Ref UserPoolClient
      Events:
        Signup:
          Type: Api
          Properties:
            RestApiId: !Ref ServerlessApi
            Path: /signup
            Method: POST
            Auth:
              Authorizer: NONE
        Signin:
          Type: Api
          Properties:
            RestApiId: !Ref ServerlessApi
            Path: /signin
            Method: POST
            Auth:
              Authorizer: NONE

Outputs:
  ApiUrl:
    Description: The target URL of the created API
    Value: !Sub "https://${ServerlessApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/"
    Export:
      Name: ApiUrl
```

First, we define a ServerlessApi resource. In a SAM template, this resource is usually created implicitly for us, but when we want to enable CORS or use an authorizer, we need to define it explicitly.

We also set the default authorizer for all routes to be the CognitoAuthorizer and then configure it right away with a Cognito user pool we define later in the template.

Then we define an `UNAUTHORIZED` gateway response because API-Gateway won't add CORS headers to our responses on its own. In our Lambda backed routes, we can do this via JavaScript code, but when we dont't have permissions to call them, this doesn't help. Gateway responses are a way to add headers directly with API-Gateway.

Next, we define the Cognito related resources:

- `UserPool` is the part of Cognito that holds our users' accounts.
- `UserPoolClient` is the part of Cognito that allows programmatic interaction with a user pool.
- `PreSignupFunction` is a Lambda function that is called before the actual signup with the user pool happens. It allows us to activate users without the need to send an email to them. It only has a few lines of code, so we inline it.
- `LambdaCognitoUserPoolExecutionPermission` grant the manged Cognito service the permission to execute our `PreSignupFunction`.

Finally, we define a Lambda function that is called by API-Gateway routes. Specifically `POST /signup` and `POST /signin`. This `AuthFunction` also gets the user pool ID and the user pool client ID as environment variables. These values are dynamically generated at deployment time, but environment variables are a way to pass such values from SAM/Cloudformation into a function.

With the `Global/Function/Handler` we set at the top and the `CodeUri` in our `AuthFunction` Properties, we can determine where our JavaScript file has to be and how it has to export a handler function for Lambda.

    ...
    Handler: index.handler
    ...
    CodeUri: functions/auth/

The file has to be called `index.js`, it has to export a function called `handler` and it has to be in the `functions/auth/` directory.

The `Auth` property of our endpoint definitions are set to `Authorizer: NONE` so API-Gateway lets us request the endpoints without the need of a token.

At the end of the file, we have one output called `ApiUrl`; we use it after the deployment to fetch the actual API URL from CloudFormation.

## Implementing the Auth Lambda Function

We created the `functions` directory at the beginning, so we only need to add an `auth` directory with an `index.js`

The `index.js` holds the following code:

```javascript
const users = require("./user-management");

exports.handler = async event => {
  const body = JSON.parse(event.body);
  if (event.path === "/signup") return signUp(body);
  return signIn(body);
};

const signUp = async ({ username, password }) => {
  try {
    await users.signUp(username, password);
    return createResponse({ message: "Created" }, 201);
  } catch (e) {
    console.log(e);
    return createResponse({ message: e.message }, 400);
  }
};

const signIn = async ({ username, password }) => {
  try {
    const token = await users.signIn(username, password);
    return createResponse({ token }, 201);
  } catch (e) {
    console.log(e);
    return createResponse({ message: e.message }, 400);
  }
};

const createResponse = (
  data = { message: "OK" },
  statusCode = 200
) => ({
  statusCode,
  body: JSON.stringify(data),
  headers: { "Access-Control-Allow-Origin": "*" }
});
```

It requires a `user-management.js` to do its work, but we talk about this later. First, let us look at what this file does.

As we said in the `template.yaml`, It exports a `handler` function that receives the HTTP request event from API-Gateway when someone sends a POST request to the `/signup` or `/signin` endpoints.

Here we can see an answer to one of the most frequent questions.

### How to access the request body?

```javascript
exports.handler = async event => {
  const body = JSON.parse(event.body);
  ...
};
```

The first parameter of a JavaScript Lambda function holds an event object. When the function is called with an API-Gateway event, this object has a `body` attribute that holds a string of the request body if the client sent one.

In our case, we expect a JSON with a `username` and `password` so we can create new user accounts or sign users into their accounts, so we need to `JSON.parse()` the body first.

Next, we have two functions, `signUp` and `signIn` that use the required `user-management` module, here called `users` to do their work. They are both passed `username` and `password` as arguments.

`createResponse` is a utility function that builds a response object. This object is what a Lambda function has to return.

Here, we have an answer to other questions from the beginning.

### How to set HTTP status codes?

```javascript
exports.handler = async event => {
  ...
  return { statusCode: 404 };
};
```

Every Lambda function needs to return a response object. This object has to have at least a `statusCode` attribute. Otherwise, API-Gateway considers the request failed.

### How to set an HTTP response header?

```javascript
exports.handler = async event => {
  ...
  return {
    statusCode: 200,
    headers: {
      "Access-Control-Allow-Origin": "*"
    }
  };
};
```

The response object our Lambda function has to return can also have a `headers` attribute, it has to be an object with header names as the objects keys and the header values as the values of the object.

Here, we can see that we need to set CORS headers manually. Otherwise, browsers won't accept the response.

Now, let's look at the `user-management.js` file we required.

```javascript
global.fetch = require("node-fetch");
const Cognito = require("amazon-cognito-identity-js");

const userPool = new Cognito.CognitoUserPool({
  UserPoolId: process.env.USER_POOL_ID,
  ClientId: process.env.USER_POOL_CLIENT_ID
});

exports.signUp = (username, password) =>
  new Promise((resolve, reject) =>
    userPool.signUp(username, password, null, null, (error, result) =>
      error ? reject(error) : resolve(result)
    )
  );

exports.signIn = (username, password) =>
  new Promise((resolve, reject) => {
    const authenticationDetails = new Cognito.AuthenticationDetails({
      Username: username,
      Password: password
    });

    const cognitoUser = new Cognito.CognitoUser({
      Username: username,
      Pool: userPool
    });

    cognitoUser.authenticateUser(authenticationDetails, {
      onSuccess: result => resolve(result.getIdToken().getJwtToken()),
      onFailure: reject
    });
  });
```

First, we use the `node-fetch` package to polyfill the `fetch` browser API. This polyfill is needed for the `amazon-cognito-identity-js` package we require next, which does the heavy lifting of talking to the Cognito service for us.

We create a `CognitoUserPool` object with the help of the environment variables we got from the SAM template and use this object inside the `signIn` and `signUp` functions we export.

The most exciting part is in the `signIn` function that fetches an ID token from the Cognito user pool and returns it.

This token needs to be passed inside an `Authorization` request header with a `Bearer` prefix on every request to our API, and it needs to be re-fetched when it expires. The CognitoAuthorizer in the API configuration of our SAM template told API-Gateway how to handle everything else with Cognito.

Now we need to install the packages we used. The `node-fetch` polyfill package and the `amazon-cognito-identity-js` package.

For this, we need to go into the `functions/auth` directory and init an NPM project, then install the packages.

    npm init -y
    npm i node-fetch amazon-cognito-identity-js

All files inside the `functions/auth` directory are uploaded to S3 when we deploy the function to Lambda with the rest of our API.

## Deploying the API

To deploy our project to AWS, we use the `sam` CLI tool.

First, we need to package our Lambda function source and upload it to an S3 deployment bucket. We can create this bucket with the `aws` CLI.

    aws s3 mb s3://<DEPLOYMENT_BUCKET_NAME>

The bucket name has to be globally unique, so we need to invent one.

When the creation worked, we need the `DEPLOYMENT_BUCKET_NAME` to `package` our Lambda source.

    sam package --template-file template.yaml \
    --s3-bucket <DEPLOYMENT_BUCKET_NAME> \
    --output-template-file packaged.yaml

This command creates a `packaged.yaml` file that holds URLs to our packaged Lambda sources on S3.

Next, we need to do the actual deployment with CloudFormation.

    sam deploy --template-file packaged.yaml \
    --stack-name serverless-api \
    --capabilities CAPABILITY_IAM

If everything went well, weuse the following command to get the APIs base URL.

    aws cloudformation describe-stacks \
    --stack-name serverless-api \
    --query 'Stacks[0].Outputs[?OutputKey==`ApiUrl`].OutputValue' \
    --output text

The URL should look something like this:

    https://<API_ID>.execute-api.<REGION>.amazonaws.com/Prod/

This URL can now be used to issue POST requests to the `/signup` and `/signin` endpoints we created.

## Conclusion

This article was the first of two articles about creating serverless APIs on AWS. We talked about the motivations to do so, the AWS services we need to get things done and implemented token-based authentication with the help of AWS Cognito.

We also answered a few of the most pressing questions that arise when building an API with API-Gateway and Lambda.

In the next part, we implement two Lambda functions that allow us to work with the API.

- `ImagesFunction` is responsible for creating upload links for an S3 bucket.
- `TagsFunction` handles the S3 uploads, third-party API integration, and the listing of created tags.

---

*Originally published on the [Moesif Blog](https://moesif.com/blog)*