Local Serverless Application Development with AWS SAM and Localstack
Published: 2023-10-18 08:32:31

Are you looking for cost-efficient and streamlined local development and testing of serverless applications built with AWS services like Amazon API Gateway, AWS Lambda, and Amazon DynamoDB? 

With AWS SAM and Localstack, you can emulate AWS services on a local machine and allow developers to work offline, saving costs and enabling rapid iteration. 

In this article, we will explore on how to build a serverless API service and test it locally with AWS SAM and Localstack.

## Follow me for more
{% embed https://linktr.ee/fullstacksaiyan %}

## AWS Serverless Application Model (SAM)
![aws-sam](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/os532ki492j27zxpdg9r.png)
[AWS SAM (Serverless Application Model)](https://aws.amazon.com/serverless/sam/) is an open-source framework for building serverless applications. It extends AWS CloudFormation to provide a simplified way of defining the Amazon API Gateway APIs, AWS Lambda functions, and Amazon DynamoDB tables needed by your serverless application.

> To install AWS SAM you can refer to this AWS documentation page for detailed installation. [Install AWS SAM](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/install-sam-cli.html)

> We will also need to install AWS CLI. You can refer to this documentation [Install AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html).

## Localstack
![localstack](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nfnsvo9tnr83oqyo6ztj.png)
[Localstack](https://www.localstack.cloud/) is an open-source project that provides a fully functional local AWS cloud stack for development and testing purposes. It allows developers to emulate several AWS services on their local machines, enabling them to develop and test their applications without accessing the actual AWS cloud.

LocalStack aims to replicate the behavior of various AWS services, such as Amazon S3, Amazon DynamoDB, AWS Lambda, Amazon SQS, and many others. Developers can interact with these emulated services through the AWS SDKs or command-line tools, just as they would with the real AWS services.

> To install Localstack you can refer to this AWS documentation page for detailed installation. [Install Localstack](https://docs.localstack.cloud/getting-started/installation/)

## Docker 
> Both AWS SAM and Localstack utilize Docker to run simulate AWS Service locally, so we will be using Docker also.


![docker](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jh41mjjfe8daavaw34gm.png)



Docker is a platform and a set of tools designed to simplify the creation, deployment, and running of applications using containers. Containers are lightweight, portable, and consistent environments that encapsulate an application and its dependencies. Docker allows developers to package an application and its dependencies into a container image, which can then be easily distributed and run consistently across different environments.

> To install Docker, you can also check Docker documentation page for detailed installation. [Install Docker Engine](https://docs.docker.com/engine/install/)

## Create project with AWS SAM.
- We can start creating and initializing our AWS SAM project using this command.
```sh
sam init
```
- Then we can follow this configuration wizard to start creating our Todo API service.
--- For the template source we can choose `1- AWS Quick Start Templates`
```sh
You can preselect a particular runtime or package type when using the `sam init` experience.
Call `sam init --help` to learn more.

Which template source would you like to use?
        1 - AWS Quick Start Templates
        2 - Custom Template Location
Choice: 1
```
- For the AWS Quick Start application template, we can choose `7 - Serverless API`
```sh
Choose an AWS Quick Start application template
        1 - Hello World Example
        2 - Data processing
        3 - Hello World Example with Powertools for AWS Lambda
        4 - Multi-step workflow
        5 - Scheduled task
        6 - Standalone function
        7 - Serverless API
        8 - Infrastructure event management
        9 - Lambda Response Streaming
        10 - Serverless Connector Hello World Example
        11 - Multi-step workflow with Connectors
        12 - GraphQLApi Hello World Example
        13 - Full Stack
        14 - Lambda EFS example
        15 - Hello World Example With Powertools for AWS Lambda
        16 - DynamoDB Example
        17 - Machine Learning
Template: 7
```
- For runtime we can choose `2 - nodejs18.x`
```sh
Which runtime would you like to use?
        1 - dotnet6
        2 - nodejs18.x
        3 - nodejs16.x
        4 - nodejs14.x
Runtime: 2
```
- For enable X-Ray tracing, choose `N`
```sh
Would you like to enable X-Ray tracing on the function(s) in your application?  [y/N]: N
```
- Choose `N` for enable monitoring using CloudWatch Application Insights
```sh
Would you like to enable monitoring using CloudWatch Application Insights?
For more info, please view https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch-application-insights.html [y/N]: N
```
- Project name, type `serverless-api`
```sh
Project name [sam-app]: serverless-api
```
- The complete SAM CLI configuration will look like this.
```sh
You can preselect a particular runtime or package type when using the `sam init` experience.
Call `sam init --help` to learn more.

Which template source would you like to use?
        1 - AWS Quick Start Templates
        2 - Custom Template Location
Choice: 1

Choose an AWS Quick Start application template
        1 - Hello World Example
        2 - Data processing
        3 - Hello World Example with Powertools for AWS Lambda
        4 - Multi-step workflow
        5 - Scheduled task
        6 - Standalone function
        7 - Serverless API
        8 - Infrastructure event management
        9 - Lambda Response Streaming
        10 - Serverless Connector Hello World Example
        11 - Multi-step workflow with Connectors
        12 - GraphQLApi Hello World Example
        13 - Full Stack
        14 - Lambda EFS example
        15 - Hello World Example With Powertools for AWS Lambda
        16 - DynamoDB Example
        17 - Machine Learning
Template: 7

Which runtime would you like to use?
        1 - dotnet6
        2 - nodejs18.x
        3 - nodejs16.x
        4 - nodejs14.x
Runtime: 2

Based on your selections, the only Package type available is Zip.
We will proceed to selecting the Package type as Zip.

Based on your selections, the only dependency manager available is npm.
We will proceed copying the template using npm.

Would you like to enable X-Ray tracing on the function(s) in your application?  [y/N]: N

Would you like to enable monitoring using CloudWatch Application Insights?
For more info, please view https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch-application-insights.html [y/N]: N

Project name [sam-app]: serverless-api

    -----------------------
    Generating application:
    -----------------------
    Name: serverless-api
    Runtime: nodejs18.x
    Architectures: x86_64
    Dependency Manager: npm
    Application Template: quick-start-web
    Output Directory: .
    Configuration file: serverless-api\samconfig.toml

    Next steps can be found in the README file at serverless-api\README.md


Commands you can use next
=========================
[*] Create pipeline: cd serverless-api && sam pipeline init --bootstrap
[*] Validate SAM template: cd serverless-api && sam validate
[*] Test Function in the Cloud: cd serverless-api && sam sync --stack-name {stack-name} --watch
```

AWS SAM will then generate our `serverless-api` project according to the configuration we chose above like this. 

```sh
.
|-- README.md
|-- __tests__
|   `-- unit
|       `-- handlers
|-- buildspec.yml
|-- env.json
|-- events
|   |-- event-get-all-items.json
|   |-- event-get-by-id.json
|   `-- event-post-item.json
|-- package.json
|-- samconfig.toml
|-- src
|   `-- handlers
|       |-- get-all-items.mjs
|       |-- get-by-id.mjs
|       `-- put-item.mjs
`-- template.yaml
```
`src/handlers` is where our Lambda function will be, and `template.yaml` is our AWS SAM template.

Our project architecture above will look like this

![architecture](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bcdcr4hju9rbprsoecdt.png)

In this project, we are using several AWS Services like
- API Gateway
- Lambda
- DynamoDB

So we need a way to emulate these services locally.

Next, we will explore how to run our project and emulate AWS services locally using AWS SAM.


## Run and Test Locally with AWS SAM
### Run DynamoDB locally
AWS SAM doesn't emulate DynamoDB service that we are going to use in our project. So we need to run DynamoDB locally.

Amazon DynamoDB provides a way to run DynamoDB locally on our computer. [Deploying DynamoDB locally on your computer](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.DownloadingAndRunning.html)

So we will run our DynamoDB locally using Docker.
- Create `docker-compose.yml`
- Type the following code

```yaml
version: '3.8'
services:
 dynamodb-local:
   command: "-jar DynamoDBLocal.jar -sharedDb -dbPath ./data"
   image: "amazon/dynamodb-local:latest"
   container_name: dynamodb-local
   ports:
     - "8000:8000"
   volumes:
     - "./docker/dynamodb:/home/dynamodblocal/data"
   working_dir: /home/dynamodblocal
```
- Run the following command in your terminal
```sh
docker-compose up
```
- Check whether our Dynamodb container is successfully running
```sh
$ docker ps
CONTAINER ID   IMAGE                                          COMMAND                  CREATED          STATUS          PORTS                      NAMES
e7d0b58896e5   amazon/dynamodb-local:latest                   "java -jar DynamoDBL…"   2 months ago     Up 2 hours      0.0.0.0:8000->8000/tcp     dynamodb-local
```
> Take note on our dynamodb container name which is `dynamodb-local` and the dynamodb network  named `dynamodb_default`. We will be using this information to configure our application to run locally depending on the `ENVIRONMENT` variable.

### Create our DynamoDB table inside our local DynamoDB
To create our DynamoDB table inside our local DynamoDB, we can run this command
```sh
aws dynamodb create-table \
    --table-name SampleTable \
    --attribute-definitions \
        AttributeName=id,AttributeType=S \
    --key-schema \
        AttributeName=id,KeyType=HASH \
    --provisioned-throughput \
        ReadCapacityUnits=2,WriteCapacityUnits=2 \
    --table-class STANDARD \
    --endpoint-url http://localhost:8000
```
As you notice, we will run above command into our local DynamoDB endpoint using this flag `--endpoint-url HTTP://localhost:8000`


### Modify our `template.yaml` and function handlers
- First we will modify our `template.yaml` by adding this configuration
```yaml
Globals:
  Function:
    Environment:
      Variables:
        ENVIRONMENT: "local"
```
So that our complete `template.yaml` configuration will looks like this.
```yaml
AWSTemplateFormatVersion: 2010-09-09
Description: >-
  serverless-api

Transform:
  - AWS::Serverless-2016-10-31

Globals:
  Function:
    Environment:
      Variables:
        ENVIRONMENT: "local"

Resources:
  getAllItemsFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: src/handlers/get-all-items.getAllItemsHandler
      Runtime: nodejs18.x
      Architectures:
        - x86_64
      MemorySize: 128
      Timeout: 100
      Description: A simple example includes a HTTP get method to get all items from a DynamoDB table.
      Policies:
        - DynamoDBCrudPolicy:
            TableName: !Ref SampleTable
      Environment:
        Variables:
          SAMPLE_TABLE: !Ref SampleTable
      Events:
        Api:
          Type: Api
          Properties:
            Path: /
            Method: GET

  getByIdFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: src/handlers/get-by-id.getByIdHandler
      Runtime: nodejs18.x
      Architectures:
        - x86_64
      MemorySize: 128
      Timeout: 100
      Description: A simple example includes a HTTP get method to get one item by id from a DynamoDB table.
      Policies:
        - DynamoDBCrudPolicy:
            TableName: !Ref SampleTable
      Environment:
        Variables:
          SAMPLE_TABLE: !Ref SampleTable
      Events:
        Api:
          Type: Api
          Properties:
            Path: /{id}
            Method: GET

  putItemFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: src/handlers/put-item.putItemHandler
      Runtime: nodejs18.x
      Architectures:
        - x86_64
      MemorySize: 128
      Timeout: 100
      Description: A simple example includes a HTTP post method to add one item to a DynamoDB table.
      Policies:
        - DynamoDBCrudPolicy:
            TableName: !Ref SampleTable
      Environment:
        Variables:
          SAMPLE_TABLE: !Ref SampleTable
      Events:
        Api:
          Type: Api
          Properties:
            Path: /
            Method: POST

  SampleTable:
    Type: AWS::Serverless::SimpleTable
    Properties:
      PrimaryKey:
        Name: id
        Type: String
      ProvisionedThroughput:
        ReadCapacityUnits: 2
        WriteCapacityUnits: 2

Outputs:
  WebEndpoint:
    Description: "API Gateway endpoint URL for Prod stage"
    Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/"

```
- Next, we will modify all our function handler `get-all-items.mjs`, `get-by-id.mjs` and `put-item-mjs`. We will modify this part of the code.
```js
const client = new DynamoDBClient();
```
into
```js
const client = new DynamoDBClient(
  process.env.ENVIRONMENT === "local"
    ? {
        endpoint: "http://dynamodb-local:8000",
      }
    : {}
);
```
### Run and Test Locally
Finally, we can start running and testing our project locally using AWS SAM.

AWS SAM provides several commands for us to run our function locally like:

- `sam local invoke`

This command is used to initiate a one-time invocation of an AWS Lambda function locally. 
```sh
$ sam local invoke
Invoking app.lambda_handler (python3.9)
Local image is out of date and will be updated to the latest runtime. To skip this, pass in the parameter --skip-pull-image
Building image....................................................................................................................
Using local image: public.ecr.aws/lambda/python:3.9-rapid-x86_64.

Mounting /Users/.../sam-app/.aws-sam/build/HelloWorldFunction as /var/task:ro,delegated, inside runtime container
START RequestId: 64bf7e54-5509-4762-a97c-3d740498d3df Version: $LATEST
END RequestId: 64bf7e54-5509-4762-a97c-3d740498d3df
REPORT RequestId: 64bf7e54-5509-4762-a97c-3d740498d3df  Init Duration: 1.09 ms  Duration: 608.42 ms       Billed Duration: 609 ms Memory Size: 128 MB     Max Memory Used: 128 MB
{"statusCode": 200, "body": "{\"message\": \"hello world\"}"}% 
```
- `sam local start-api`

This command is used to run your AWS Lambda functions locally and test through a local HTTP server host. This type of test is helpful for Lambda functions that are invoked by an Amazon API Gateway endpoint. 
```sh
$ sam local start-api

Initializing the lambda functions containers.
Local image is up-to-date
Using local image: public.ecr.aws/lambda/python:3.9-rapid-x86_64.

 * Running on http://127.0.0.1:3000
```
- `sam local start-lambda`

This command is used to invoke your AWS Lambda function through the AWS Command Line Interface (AWS CLI) or SDKs. This command starts a local endpoint that emulates AWS Lambda.
```sh
$ sam local start-lambda
Initializing the lambda functions containers.
Local image is up-to-date
Using local image: public.ecr.aws/lambda/python:3.9-rapid-x86_64.

 * Running on http://127.0.0.1:3001
2023-04-13 07:25:43 Press CTRL+C to quit
```
Because we want to emulate Amazon API Gateway and AWS Lambda services, we will use `sam local start-api` and using our dynamodb container network `dynamodb_default`
```sh
sam local start-api --docker-network dynamodb_default
```
```sh
Initializing the lambda functions containers.
Local image is up-to-date
Local image is up-to-date
Local image is up-to-date
Using local image: public.ecr.aws/lambda/nodejs:18-rapid-x86_64.

Containers Initialization is done.
Mounting getAllItemsFunction at http://127.0.0.1:3000/ [GET]
Mounting putItemFunction at http://127.0.0.1:3000/ [POST]
Mounting getByIdFunction at http://127.0.0.1:3000/{id} [GET]
You can now browse to the above endpoints to invoke your functions. You do not need to restart/reload SAM CLI while working on your functions, changes will be reflected   
instantly/automatically. If you used sam build before running local commands, you will need to re-run sam build for the changes to be picked up. You only need to restart  
SAM CLI if you update your AWS SAM template
2023-11-17 00:32:55 WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on http://127.0.0.1:3000
2023-11-17 00:32:55 Press CTRL+C to quit
```
`sam local start-api` will emulate our function through this endpoint
```sh
Mounting getAllItemsFunction at http://127.0.0.1:3000/ [GET]
Mounting putItemFunction at http://127.0.0.1:3000/ [POST]
Mounting getByIdFunction at http://127.0.0.1:3000/{id} [GET]
```
So we can test POST item by running this command in new terminal.
```sh
$ curl -X POST -H "Content-Type: application/json" -d '{"id":"3","name":"test"}' http://localhost:3000
{"id":"3","name":"test"}
```
And, test GET all items using this command
```sh
$ curl http://localhost:3000
[{"name":"test","id":"1"},{"name":"test","id":"2"}]
```

## Run and Test Locally with Localstack
With Localstack the step to run and test locally will be more easier.

But first we need to configure our `template.yaml` again.
```yaml
Globals:
  Function:
    Environment:
      Variables:
        ENVIRONMENT: "localstack"
```
we are changing our `ENVIRONMENT` variable into `localstack` so that our DynamoDBCLient will connect directly inside Localstack DynamoDB.

Then we will modify the output of our endpoint into this so that it will output Localstack endpoint that we can use to test our application.
```yaml
Outputs:
  WebEndpoint:
    Description: "API Gateway endpoint URL for Prod stage"  
    Value: !Sub "http://${ServerlessRestApi}.execute-api.localhost.localstack.cloud:4566/Prod/"
```

### Install `samlocal`
To emulate AWS services locally with Localstack, we can install and use `samlocal`, a wrapper for the AWS SAM CLI, so that every `aws sam` commands will be executed against the Localstack endpoints (http://localhost:4566 by default) instead of real AWS endpoints.

You can install it using these commands.
```sh
pip install aws-sam-cli-local
```

### Deploy project to Localstack
To deploy our project to Localstack, we can run this command
```sh
samlocal deploy
```
`samlocal` will deploy your project but not into AWS but into Localstack.
```sh
                Managed S3 bucket: aws-sam-cli-managed-default-samclisourcebucket-b1dc8c32
                A different default S3 bucket can be set in samconfig.toml
                Or by specifying --s3-bucket explicitly.
        Uploading to 9a5628197c0aebb840bb53c26bf4e03c  11801 / 11801  (100.00%)
File with same data already exists at 9a5628197c0aebb840bb53c26bf4e03c, skipping upload
File with same data already exists at 9a5628197c0aebb840bb53c26bf4e03c, skipping upload

        Deploying with following values
        ===============================
        Stack name                   : serverless-api
        Region                       : ap-southeast-1
        Confirm changeset            : True
        Disable rollback             : False
        Deployment s3 bucket         : aws-sam-cli-managed-default-samclisourcebucket-b1dc8c32
        Capabilities                 : ["CAPABILITY_IAM"]
        Parameter overrides          : {}
        Signing Profiles             : {}

Initiating deployment
=====================

        Uploading to 1245f569e29a8187dfbd04b64f50e277.template  3166 / 3166  (100.00%)


Waiting for changeset to be created..

CloudFormation stack changeset
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------     
Operation                                         LogicalResourceId                                 ResourceType                                      Replacement
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------     
* Modify                                          putItemFunctionApiPermissionProd                  AWS::Lambda::Permission                           False
* Modify                                          putItemFunction                                   AWS::Lambda::Function                             False
* Modify                                          getAllItemsFunction                               AWS::Lambda::Function                             False
* Modify                                          getByIdFunctionRole                               AWS::IAM::Role                                    False
* Modify                                          getAllItemsFunctionApiPermissionProd              AWS::Lambda::Permission                           False
* Modify                                          getAllItemsFunctionRole                           AWS::IAM::Role                                    False
* Modify                                          putItemFunctionRole                               AWS::IAM::Role                                    False
* Modify                                          getByIdFunction                                   AWS::Lambda::Function                             False
* Modify                                          ServerlessRestApiProdStage                        AWS::ApiGateway::Stage                            False
* Modify                                          ServerlessRestApiDeploymenta4d359a69a             AWS::ApiGateway::Deployment                       False
* Modify                                          ServerlessRestApi                                 AWS::ApiGateway::RestApi                          False
* Modify                                          getByIdFunctionApiPermissionProd                  AWS::Lambda::Permission                           False
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------     


Changeset created successfully. arn:aws:cloudformation:ap-southeast-1:000000000000:changeSet/samcli-deploy1700159060/1d7348f1
```
Once your project is deployed into Localstack locally, `samlocal` will return this output
```sh
CloudFormation outputs from deployed stack
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------   
Outputs
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------   
Key                 WebEndpoint
Description         API Gateway endpoint URL for Prod stage
Value               http://a4uowlkucn.execute-api.localhost.localstack.cloud:4566/Prod/
```
Finally, we can test our service through Localstack endpoint `http://a4uowlkucn.execute-api.localhost.localstack.cloud:4566/Prod/` by running this command
So we can test POST item by running this command in new terminal.
```sh
$ curl -X POST -H "Content-Type: application/json" -d '{"id":"3","name":"test"}' http://a4uowlkucn.execute-api.localhost.localstack.cloud:4566/Prod/
{"id":"3","name":"test"}
```
And, test GET all items using this command
```sh
$ curl http://a4uowlkucn.execute-api.localhost.localstack.cloud:4566/Prod/
[{"name":"test","id":"1"},{"name":"test","id":"2"}]
```

## Conclusions
While AWS SAM provides a means to run and test serverless services locally, the process is significantly streamlined with Localstack. Localstack not only facilitates the local execution and testing of serverless applications but also extends its capabilities to emulate various other services, thereby enriching our local development environment.

![localstack-services](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/a7aoryxdzu6x4qs986id.png)

## Check out my previous post
{% embed https://dev.to/aws-builders/code-anywhere-configure-your-personal-web-based-code-editor-with-aws-cli-or-aws-cdk-2og3 %}