AWS project - Building a serverless microservice with Lambda
Tags: aws, cloud, serverless, microservices
Published: 2021-01-28 21:14:05

## **Overview**

In this article, we are going to build a simple serverless microservice on AWS that enables users to create and manage movies data. We will set up an entire backend using API Gateway, Lambda and DynamoDB. Then we will use SwaggerHub as an integrated design environment for our APIs:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/iepg75jn316dyt8yj31a.png) 

The goal of this project is to learn how to create an application composed of small, easily deployable, loosely coupled, independently scalable, serverless components.

As I’m strongly against managing environments manually and take Infrastructure as Code for granted (if we are not on the same page, I’d suggest you to read [this](https://www.simplethread.com/why-infrastructure-as-code/) article first), AWS SAM will be a great fit for our serverless application. As a result, the entire application should be deployed in any AWS account with a single CloudFormation template.

## **Architecture**

:one: A user requests to the server by calling APIs from SwaggerHub UI. User's request which includes all necessary information is sent to Amazon API Gateway restful service.
:two: API Gateway transfers the collected user information to a particular AWS Lambda function based on user's request.
:three: AWS Lambda function executes event-based logic calling DynamoDB database.
:four: DynamoDB provides a persistence layer where data can be stored/retrieved by the API's Lambda function.

The high-level architecture for the serverless microservice is illustrated in the diagram below:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/jhs39tv5ocybjdckhjus.png)

## **Source code**

All source code for this project is available on GitHub in a public [AwsServerlessMicroserviceWithLambda](https://github.com/Tiamatt/AwsServerlessMicroserviceWithLambda) repository.

To clone the GitHub repository, execute the following command:

```YAML
cd my-folder
git clone https://github.com/Tiamatt/AwsServerlessMicroserviceWithLambda.git
```

## **Initial Setup**

To codify, build, package, deploy, and manage our AWS resources in a fully automated fashion, we will use:

:point_right: AWS SAM
:point_right: AWS Cloud​Formation
:point_right: AWS CLI
:point_right: AWS SDK for Python (boto3)
:point_right: Docker

If you don't want to install or maintain a local IDE, use AWS Cloud9 instead (you can find how to set up AWS Cloud9 [here](https://dev.to/tiamatt/serverless-create-debug-and-deploy-lambda-and-api-gateway-via-aws-sam-and-aws-cloud9-5158)). Otherwise you might need to install the latest [AWS CLI](https://aws.amazon.com/cli/), [AWS SAM](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html) and [Python](https://www.python.org/downloads/) on your development machine.

## **AWS Resources**

Here is the list of AWS resources that we are going to create:

:heavy_check_mark: AWS Lambda
:heavy_check_mark: Amazon DynamoDB
:heavy_check_mark: Amazon API Gateway
:heavy_check_mark: AWS IAM
:heavy_check_mark: Amazon S3 (that is where your CloudFormation template will be stored)

In this project, we will be building Lambda functions with Python 3.8.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/mony7f66essycq77cvtv.jpg)


## **Step 1. Generate an AWS SAM template**

Download **AWS SAM Hello World** template which implements a basic API backend based on Amazon API Gateway endpoint and AWS Lambda function:

```YAML
cd my-folder
sam init
```

If you're feeling lost, please, follow **Step 2. Create a SAM application** described in my [previous](https://dev.to/tiamatt/serverless-create-debug-and-deploy-lambda-and-api-gateway-via-aws-sam-and-aws-cloud9-5158) article. 

AWS SAM should create a directory with all necessary files and folders:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/o1wtmpx7f2fe4pjezshp.png)

## **Step 2. Create a DynamoDb table**

Rewrite pre-generated "template.yaml" with the following code:

```YAML
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: 'SAM Template for ServerlessMicroserviceApp'

# ======================== RESOURCES ======================== #
Resources:
  MoviesTable:
    Type: AWS::Serverless::SimpleTable
    Properties:
      TableName: 'movies-table'
      PrimaryKey:
        Name: uuid
        Type: String
```

`AWS::Serverless::SimpleTable` is AWS SAM syntax for DynamoDB table creation with a single attribute primary key named `uuid`. Note, if you want to add *sort key* or use other advanced functionality of DynamoDB, then you may want to consider using AWS CloudFormations's `AWS::DynamoDB::Table` resource instead (yes, you can use AWS CloudFormation syntax in AWS SAM template, combining the best of both worlds :blue_heart:). 

## **Step 3. Create an API Gateway**

Next step is to define API Gateway in "template.yaml":

```YAML
MoviesApi: 
    Type: AWS::Serverless::Api
    Properties:
      StageName: Prod
      # DefinitionUri: ./swagger.json
```

`AWS::Serverless::Api` is AWS SAM syntax for a collection of Amazon API Gateway resources. FYI, there is no need to explicitly add `Api` to the template - AWS SAM can implicitly create it based on `fromRestApiId` attribute defined on `AWS::Serverless::Function` resources (see Step 4 below). However I prefer an explicit definition as `AWS::Serverless::Api` has a lot of useful [properties](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-resource-api.html) I might use in the future (without actially refactoring the template).

Amazon API Gateway acts as the interface layer between the frontend (SwaggerHub) and AWS Lambda, which calls the backend (DynamoDB database). Below are the different APIs that we are about to define in `Events` attribute of each Lambda function in Step 4:
:point_right: POST /movie (CreateMovie)
:point_right: GET /movie/{uuid} (GetMovie)
:point_right: GET /movie/list (GetMovies)
:point_right: PUT /movie (UpdateMovie)
:point_right: DELETE /movie/{uuid} (DeleteMovie)


## **Step 4. Create Lambda functions**

We are going to create five lambda functions:
* **CreateMovieFunction** (create_movie/app.py)
    * triggered by *POST /movie* API
* **GetMovieFunction** (get_movie/app.py)
    * triggered by *GET /movie/{uuid}* API
* **GetMoviesFunction** (get_movies/app.py)
    * triggered by *GET /movie/list* API
* **UpdateMovieFunction** (update_movie/app.py)
    * triggered by *PUT /movie* API
* **DeleteMovieFunction** (delete_movie/app.py)
    * triggered by *DELETE /movie/{uuid}* API

Let's start with **CreateMovieFunction**:

:one: Rename "hello_world" folder to "create_movie".

:two: Clear the content of "create_movie/requirements.txt" file (no need in any Python dependency).

:three: Open "create_movie/app.py" file and add the following code:

```Python
import json
import boto3
import uuid
from botocore.exceptions import ClientError

# Extract movie object from request body and insert it into DynamoDB table
def lambda_handler(event, context):

    if ('body' not in event or event['httpMethod'] != 'POST' or 'title' not in event['body']):
        return {
            'statusCode': 400,
            'body': json.dumps({'message': 'Bad Request'})
        }
    
    new_movie = json.loads(event['body'])
    response = add_movie_to_db(new_movie)
    
    return {
        'statusCode': 200,
        'body': json.dumps({'message': 'A new movie was saved successfully in database'})
    }

# Insert a new item into DynamoDB table
def add_movie_to_db(new_movie):
    
    new_movie['uuid'] = str(uuid.uuid1())

    dynamodb = boto3.resource('dynamodb')
    table = dynamodb.Table('movies-table')
    
    try:
        response = table.put_item(Item=new_movie)
    except ClientError as e:
        print(e.response['Error']['Message'])
    else:
        return response
```
This function helps to add a new item to the DynamoDB's `movies-table`. `title` field is the only required property of `body` object passed by a user. `uuid` field, as a primary key that uniquely identifies each movie, should be automatically generated by `uuid.uuid1()` Python function.  

If you want to debug the code, please follow **Step 3. Debug your function locally** described in my [previous](https://dev.to/tiamatt/serverless-create-debug-and-deploy-lambda-and-api-gateway-via-aws-sam-and-aws-cloud9-5158) article. 


:four: Add lambda function to "template.yaml":

```YAML
# ======================== GLOBAL ======================== #
Globals:
  Function:
    Runtime: python3.8
    Handler: app.lambda_handler
    Timeout: 60 # default is 3 seconds the function can run before it is stopped
    Environment:
      Variables:
        TABLE_NAME: !Ref MoviesTable

# ======================== RESOURCES ======================== #
Resources:
  # ... other resources ...
  CreateMovieFunction:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: 'CreateMovieFunction'
      CodeUri: create_movie/
      Policies: # follow the principle of least privilege
        - DynamoDBCrudPolicy: # AWS SAM policy
            TableName: !Ref MoviesTable
        - AWSLambdaBasicExecutionRole
      Events:
        CreateMovieApi:
          Type: Api
          Properties:
            RestApiId: !Ref MoviesApi
            Path: /movie
            Method: post
```
* **Globals** - resources that have common configurations (think of it as a *variable* in programming). As we are going to create four more functions with the same `Runtime`, `Handler`, `Timeout` and `Environment` values, we can put all these attributes into `Global` section following a good old DRY (Don't Repeat Yourself) principle.

* **CodeUri** - the Lambda function's path (might be Amazon S3 URI, local file path, or FunctionCode object). Here we are adding a reference to a *create_movie* folder. Note, in `Global` section's `Handler` attribute we have already specified `app` file name (without *py* file extension) and `lambda_handler` function name.

* **Policies** - one or more policies that will be appended to the default role (a new IAM role will be created) for a Lambda function. Here we are giving our function permission to access our DynamoDB table.

* **Events** - events that trigger the Lambda function. In our case it is *POST /movie* API event.

In the same way as above, we need to create four more Lambda functions:

* GetMovieFunction - get the code from GitHub [here](https://github.com/Tiamatt/AwsServerlessMicroserviceWithLambda/blob/main/get_movie/app.py)
* GetMoviesFunction - get the code from GitHub [here](https://github.com/Tiamatt/AwsServerlessMicroserviceWithLambda/blob/main/get_movies/app.py)
* UpdateMovieFunction - get the code from GitHub [here](https://github.com/Tiamatt/AwsServerlessMicroserviceWithLambda/blob/main/update_movie/app.py)
* DeleteMovieFunction - get the code from GitHub [here](https://github.com/Tiamatt/AwsServerlessMicroserviceWithLambda/blob/main/delete_movie/app.py)
* The final version of the CloudFormation  template - get the code from GitHub [here](https://github.com/Tiamatt/AwsServerlessMicroserviceWithLambda/blob/main/template.yaml)

At the end we should get the following files and folders:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/50ogyb79ep3izw03dxfr.png)

## **Step 5. Build and deploy the project to AWS Cloud**

Build the project inside a Docker container:

```YAML
cd my-folder
sam build --use-container
```

AWS SAM builds any dependencies that your application has, and copies your application source code to folders under `.aws-sam/build` to be zipped and uploaded to Lambda.

Then deploy your application using the following AWS SAM command:

```YAML
sam deploy --guided
```
Please follow **Step 5. Deploy the project to the AWS Cloud** described in my [previous](https://dev.to/tiamatt/serverless-create-debug-and-deploy-lambda-and-api-gateway-via-aws-sam-and-aws-cloud9-5158) article for a detailed explanation.

AWS SAM deploys the application using AWS CloudFormation. You can navigate to AWS Management Console -> CloudFormation to view the list of newly generated resources:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/oflblwduws6ruwxlszed.png)

## **Step 6. SwaggerHub integration**

Instead of logging into AWS Management Console and navigating from one API Gateway endpoint to another, we will use SwaggerHub to manage all your APIs in one place and to document them all.

**SwaggerHub** is an integrated API development platform that brings together all the core capabilities of the open source Swagger framework, along with additional advanced capabilities to build, document, manage, and deploy your APIs. 

All you need is to create a free account on [Swagger](https://swagger.io/tools/swaggerhub/), then navigate to `Create New` -> `Create New API`:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/8kx85nm7i5qjm9ssd3q3.png)

Give a name to your template:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/bfa0auc0lc03mcsfseop.png)

Next, modify information of Petstore APIs with our own APIs. For example, for *POST /movie/{uuid}* API the template should look like this one below:

```YAML
swagger: '2.0'
info:
  description: |
    This is  a swagger for an AWS serverless microservice built with API Gateway, Lambda and DynamoDB.
  version: 1.0.0
  title: Swagger for AWS serverless microservice
  termsOfService: http://swagger.io/terms/
  license:
    name: Apache 2.0
    url: http://www.apache.org/licenses/LICENSE-2.0.html
tags:
- name: movie
  description: CRUD operations for movie domain
  externalDocs:
    description: Find out about the project
    url: https://github.com/Tiamatt/AWSServerlessMicroserviceWithLambda
paths:
  /movie/{uuid}:
    get:
      tags:
      - movie
      summary: Find a movie by uuid
      description: Returns a single movie
      operationId: getMovieByUuid
      produces:
      - application/json
      - application/xml
      parameters:
      - name: uuid
        in: path
        description: UUID of movie to return
        required: true
        type: string
      responses:
        200:
          description: successful operation
          schema:
            $ref: '#/definitions/Movie'
        400:
          description: Invalid UUID supplied
        404:
          description: Movie not found
definitions:
  Director:
    type: object
    properties:
      firstname:
        type: string
        example: Francis
      lastname:
        type: string
        example: Coppola
    xml:
      name: Director
  Movie:
    type: object
    required:
    - title
    - year
    properties:
      title:
        type: string
        example: Godfather
      year:
        type: integer
        format: int32
        example: 1972
      director:
        $ref: '#/definitions/Director'
      country:
        type: string
        example: United States
    xml:
      name: Movie   
externalDocs:
  description: Find out more about Swagger
  url: http://swagger.io
schemes:
 - https
# Added by API Auto Mocking Plugin
host: {my-id}.execute-api.us-east-1.amazonaws.com
basePath: /Prod
```

Get the final version of the Swagger template from GitHub [here](https://github.com/Tiamatt/AwsServerlessMicroserviceWithLambda/blob/main/swagger.yaml).


You need to replace `host` value with your **API Gateway invoke url**, which you can find in *Outputs* section of AWS CloudFormation Clonsole:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/srvvrzpfq4905dzityad.png)


And don't forget to save your changes! Then click on `View Documentation` icon: 

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/xnjirxgifreywu3wzelm.png)


## **Step 7. Results**

Finally, it's time to play with our serverless microservice using beautiful SwaggerHub UI:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/mopu3jut4wm44n101br5.png)

:tada: Now, let's create a new movie:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/miqjzzxddy3jqptkhll3.png)

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/gp6uvocc3y4vsneg85sv.png)

Feel free to add some more movies.

:tada: Let's get a list of all movie:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/tbpvprl3ah9twmws2p4m.png)

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/tnfwu2pdy7dg8zwiokbt.png)

Copy `uuid` of 'Godfather' movie for our next steps!

:tada: Get a movie by uuid:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/lxchosh4877whsa7oe7x.png)

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/m0v371pqmo75gxyxbk4u.png)


:tada: Delete a movie by uuid:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/ulmzch4apwpe63caqgnu.png)

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/q5b3sbw547l6ki6dokmq.png)

:tada: And finally, let's update an existing movie:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/i5infrb625vsozaesysc.png)

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/efkkmeoe3ei0e28jdqvy.png)

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/v44i2po70p7j6pikbqmc.png)

## **Step 8. Cleanup**

Use the AWS CloudFormation command to delete the stack along with all the resources it created:

```YAML
aws cloudformation delete-stack --stack-name aws-serverless-microservice-app-stack
```

## **Conclusion**

Congratulations on getting this far!

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/2ihewytdl7m1cdevy90m.png)

Now you know how to create a simple CRUD (create, read, update, delete) app, and to set the foundational services, components, and plumbing needed to get a basic AWS serverless microservice up and running!