Create and deploy a To-do CRUD service using Node.js, AWS and Serverless Framework
Tags: javascript, node, serverless, aws
Published: 2020-08-06 16:22:33

## Introduction

In this post, we will go through creating a simple CRUD (Create, Read, Update and Delete) service by creating a To-do service using Node.js, AWS and [Serverless Framework](https://www.serverless.com/). We will be creating and deploying [AWS Lambda](https://aws.amazon.com/lambda/) functions and expose them as RESTful APIs using [Amazon API Gateway](https://aws.amazon.com/api-gateway/). We will also make use of a powerful Node.js middleware [middy](https://github.com/middyjs/middy) to make our development even simpler.


## Architecture

Below is a high-level architecture of what we are going to build.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/wvvjfbyh6bzfqs41dyd3.png)

## Setup

Before we get started, we will require some setup.

### Create AWS Account

We must create an AWS account. For this set of instructions, it will not cost us anything. The [AWS Free Tier](https://aws.amazon.com/free/?all-free-tier.sort-by=item.additionalFields.SortRank&all-free-tier.sort-order=asc) should be plenty for our use case.

### Serverless Framework Installation

We will install the Serverless Framework on our machines as a standalone binary.
There are multiple ways of doing this in the [Serverless docs](https://www.serverless.com/framework/docs/getting-started/). In this post, we will be installing through NPM:

```bash
npm install -g serverless
```

To verify installation we will execute:

```bash
sls --version
```

### AWS CLI Installation

In order to use the Serverless Framework efficiently in our machine, we will make use of the AWS CLI. Instructions specific to your machine can be found [here](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html). For macOS users like me, the instructions will be:

```bash
curl "https://awscli.amazonaws.com/AWSCLIV2.pkg" -o "AWSCLIV2.pkg"

sudo installer -pkg ./AWSCLIV2.pkg -target /
```

We can then verify the installation as follows:

```bash
aws --version
```

### Configuring AWS CLI

At the moment AWS CLI does not know who we are. We will need to provide some information about this. The purpose of this is to link our local machine CLI with AWS.

Going back to our AWS console. We go into the Identity and Access Management (IAM) service. This service manages who can access our AWS resources.

Click on the "Users" tab.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/yuy9wr7d59yan0g41yeu.png)

Then, we can create a user.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/hfc0o9igrm2idmgvcjjq.png)

Select "Attach existing policies directly". For the purpose of this post, we will grant this user with `AdministratorAccess`. Ideally, we should only grant users the level of access that is required.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/k6i4pom3iqr9swxa7mxi.png)

The step to add tags can be skipped for this tutorial and we can proceed with creating the user.

Take note of your AWS Management Console access sign-in link. Note that the prefix on the link is our created user ID. 

Also, take note of your `Access Key ID` and `Secret Access Key`.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/jquio27s9jrulurd80oc.png)

Back in our terminal, we will execute the following command then enter the credentials we created. We will then select the location appropriate for us. In my case, I chose Europe as it is closest to me and that is where I would like my data to be stored. 

```bash
aws configure
```

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/6y64hq648grooe74is75.png)

Now, AWS is configured and linked to our local machine.


## Create Project

Now, we will create a project, which we will call `todo-service`. We will use a fork of a base project from [Ariel Weinberger](https://github.com/arielweinberger) at [codingly.io](codingly.io).

```bash
sls create --name todo-service --template-url https://github.com/jericopingul/sls-base
```
This will create a starter project for us. We have called it `todo-service` because all operations we will be doing for this to-do API will be defined within this service.

In our `serverless.yml`, we will add our region within the `provider` property. In my case it will be: 
```yaml
provider:
  stage: ${opt:stage, 'dev'}
  region: eu-west-1
```
You may be curious what the `stage` property is. In this case, this will define the stage to which where we will deploy our service. In real-life there will be multiple stages that include the likes of production or any other stage, depending on the development requirements. In this tutorial, we will just use one stage `dev`. 
In terms of the syntax, the `opt.stage` can be used to reference variable, while the second parameter is a default ('dev') if `opt.stage` is not set.

We also use two plugins:
```yaml
plugins:
  - serverless-bundle
  - serverless-pseudo-parameters
```
`serverless-bundle` provides us with a number of benefits including allowing us to bundle our JavaScript using webpack, reduce our bundle size, allow the use of modern JavaScript (ES6+) with minimal configuration.

`serverless-pseudo-parameters` allows us to easily interpolate AWS parameters which will make our life easier later on. More information on this plugin can be found [here](https://www.serverless.com/plugins/serverless-pseudo-parameters).

## Create a Database

We will need to store our to-do items in a database. We will make use of a NoSQL [DynamoDB](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Introduction.html) provided by AWS. The AWS free tier gives us a generous amount of storage.

In order to create the database, we will add the following statement to our `serverless.yml` so that we can instruct CloudFormation to create it in AWS. We define an attribute that is going to be our primary key, in this case, it is `id`.

```yaml
provider:
  ...

resources:
  Resources:
    TodoTable:
      Type: AWS::DynamoDB::Table
      Properties:
        TableName: TodoTable-${self:provider.stage}
        BillingMode: PAY_PER_REQUEST
        AttributeDefinitions:
          - AttributeName: id
            AttributeType: S
        KeySchema:
          - AttributeName: id
            KeyType: HASH
```
In the above statement, we are instructing [AWS CloudFormation](https://aws.amazon.com/cloudformation/) to create a table named `TodoTable-dev` with a primary key `id` with a [PAY_PER_REQUEST](https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_BillingModeSummary.html) billing.

With the changes above, we can create the database on AWS and deploy our project by using the command:

```bash
sls deploy -v
```

We are using an optional `-v` option which means verbose just to see more information on the logs.

In AWS CloudFormation we should see the `todo-service-dev` stack. In the resource tab, we can verify that our table has been created:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/hp6kzbmm5hvpt5leoea5.png)

## Third-party libraries

### AWS SDK
We will require the `aws-sdk` library to create the DynamoDB client instance. More information [here](https://aws.amazon.com/sdk-for-node-js/).

### Middy
We will be using the `middy` middleware library to simplify our AWS Lambda code. We will be using middy, middy body-parser and middy HTTP error handler. So we will install the following:

```bash
yarn add @middy/core @middy/http-event-normalizer @middy/http-error-handler @middy/http-json-body-parser
```

The purpose of each library are as follows:
- `@middy/core` is the core middy library.
- `@middy/http-event-normalizer` simplifies accessing query string or path parameters.
- `@middy/http-error-handler` handles uncaught errors and generates a proper HTTP response for them. See more info [here](https://www.npmjs.com/package/@middy/http-error-handler).
- `@middy/http-json-body-parser` parses HTTP requests with a JSON body and converts them into an object for use within our Lambdas.


### Error handler
`@middy/http-error-handler` recommends using `http-errors` library to be used together with their library to simplify creating errors so we will also install the following:

```bash
yarn add http-errors
```

### UUID
We will require to generate a unique identifier for each of our to-dos in the database so we will use the `uuid` library.


```bash
yarn add uuid
```
## Creating our AWS Lambdas

Now, we will move onto creating our AWS Lambdas that we will expose via HTTP.

### Create a To-do

Now we will create our create to-do Lambda function. In our `serverless.yml` we will add the following entry in the functions property:

```yaml
functions:
  createTodo:
    handler: src/handlers/createTodo.handler
    events:
      - http:
          method: POST
          path: /todo
```

This means that we will have a `createTodo.js` file that exports a function `handler` in the `src/handlers` directory.

Here, we will use the middleware we installed. We will define a common middleware for all of Lambdas we will use in our project in `common/middlware.js` with the contents:

```js
import middy from '@middy/core';
import jsonBodyParser from '@middy/http-json-body-parser';
import httpEventNormalizer from '@middy/http-event-normalizer';
import httpErrorHandler from '@middy/http-error-handler';

export default (handler) =>
  middy(handler).use([
    jsonBodyParser(),
    httpEventNormalizer(),
    httpErrorHandler(),
  ]);

```

This exported function will execute the listed middlewares in the array on the passed handler function.

Below, we add the `custom` property in our `serverless.yml` file. The purpose of this is to make it easier to make change our tables down the line. We make use of AWS CloudFormation intrinsic functions [Ref](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-ref.html) and [GetAtt](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-getatt.html) so that when our stack is deployed then these values will be dynamically evaluated. For the purpose of this post, we will turn off linting on our JavaScript code but I would recommend this to be turned on in production code. 

```yaml
custom:
  TodoTable:
    name: !Ref TodoTable
    arn: !GetAtt TodoTable.Arn
  bundle:
    linting: false
```

We will also require to add permissions to our Lambda in `serverless.yml` to create entries in our database table:

```yaml
provider:
  ...
  iamRoleStatements:
    - Effect: Allow
      Action:
        - dynamodb:PutItem
```

Below will be the code for our Lambda function in our `createTodo.js` file. We create a to-do item with the description from the request body and we set the `done` status as false by default. Note that we execute our common middleware in the last line.

```js
const dynamoDB = new AWS.DynamoDB.DocumentClient();

async function createTodo(event, context) {
  const { description } = event.body;
  const now = new Date();

  const todo = {
    id: uuid(),
    description,
    created: now.toISOString(),
    updated: now.toISOString(),
    done: false,
  };

  try {
    await dynamoDB
      .put({
        TableName: process.env.TODO_TABLE_NAME,
        Item: todo,
      })
      .promise(); // to return a promise instead
  } catch (error) {
    console.error(error);
    throw new createError.InternalServerError(error);
  }

  return {
    statusCode: 201,
    body: JSON.stringify(todo),
  };
}

export const handler = middleware(createTodo);
```

We can deploy our changes with the same deploy command:

```bash
sls deploy -v
```

We should find our API URL/endpoint that we created in our terminal and we can verify using a REST client, here I'm using postman:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/r09zylbh52s7dfk0agtc.png)

### Retrieve To-dos

We create a new entry in `serverless.yml` to add the new `getTodos` function:

```yaml
functions:
  ...     
  getTodos:
    handler: src/handlers/getTodos.handler
    events:
      - http:
          method: GET
          path: /todo
```

We are also required to add `Scan` action permissions.

```yaml
provider:
  ...
  iamRoleStatements:
    - Effect: Allow
      Action:
        - dynamodb:Scan
```

Below is the code to retrieve all entries in the database table then returns it.

```js
async function getTodos(event, context) {
  let todos;

  try {
    const result = await dynamoDB
      .scan({
        TableName: process.env.TODO_TABLE_NAME,
      })
      .promise();
    todos = result.Items;
  } catch (error) {
    console.error(error);
    throw new createError.InternalServerError(error);
  }

  return {
    statusCode: 200,
    body: JSON.stringify(todos),
  };
}

export const handler = middleware(getTodos);
```

### Update a To-do

We will require to add the `UpdateItem` permissions.

```yaml
provider:
  ...
  iamRoleStatements:
    - Effect: Allow
      Action:
        - dynamodb:UpdateItem
```

We create the following new function in our `functions` property. Note that we are using `PATCH` as we are going apply a [partial update](https://rapidapi.com/blog/put-vs-patch/) to the resource. 

```yaml
functions:
  ...
  updateTodo:
    handler: src/handlers/updateTodo.handler
    events:
      - http:
          method: PATCH
          path: /todo/{id}

```

Below we have the code for our update function. We will only allow the `description` and `done` fields to be updated. In the implementation below, we require at least one of `description` and `done` to be part of the request body, updates the data accordingly and finally returns the updated resource.

```js
async function updateTodo(event, context) {
  const { id } = event.pathParameters;
  const { description, done } = event.body;
  const now = new Date();

  if (!description && done === undefined) {
    throw new createError.BadRequest(
      'You must update either description or done status!'
    );
  }

  const updatedAttributes = [];
  const expressionAttributeValues = {};

  if (description) {
    updatedAttributes.push(`description = :description`);
    expressionAttributeValues[':description'] = description;
  }

  if (done !== undefined) {
    updatedAttributes.push(`done = :done`);
    expressionAttributeValues[':done'] = !!done;
  }

  updatedAttributes.push(`updated = :updated`);
  expressionAttributeValues[':updated'] = new Date().toISOString();

  const updateExpression = `set ${updatedAttributes.join(', ')}`;

  const params = {
    TableName: process.env.TODO_TABLE_NAME,
    Key: { id },
    UpdateExpression: updateExpression,
    ExpressionAttributeValues: expressionAttributeValues,
    ReturnValues: 'ALL_NEW',
  };

  let updatedTodo;

  try {
    const result = await dynamoDB.update(params).promise();
    updatedTodo = result.Attributes;
  } catch (error) {
    console.error(error);
    throw new createError.InternalServerError(error);
  }

  return {
    statusCode: 200,
    body: JSON.stringify(updatedTodo),
  };
}
```

### Delete a To-do

We first add the `DeleteItem` permission:

```yaml
provider:
  ...
  iamRoleStatements:
    - Effect: Allow
      Action:
        - dynamodb:DeleteItem
```

Then add the new function in our `functions` property in `serverless.yml`:

```yaml
functions:
  ...
  deleteTodo:
    handler: src/handlers/deleteTodo.handler
    events:
      - http:
          method: DELETE
          path: /todo/{id}
```

Below we have our delete function that simply deletes an entry in the database table based on the `id`.

```js
async function deleteTodo(event, context) {
  const { id } = event.pathParameters;

  const params = {
    TableName: process.env.TODO_TABLE_NAME,
    Key: { id },
  };

  try {
    await dynamoDB.delete(params).promise();
  } catch (error) {
    console.error(error);
    throw new createError.InternalServerError(error);
  }

  return {
    statusCode: 200,
  };
}
```

## Closing Notes

We have created a simple to-do CRUD service using Node.js, AWS Lambda and Serverless Framework. We have also made use of middleware libraries to simplify the development of our Lambdas.

There are a number of steps involved in the initial set up but once we have done this, it is straightforward to add create and add new functions.

Thank you for following along and I hope that this simple CRUD service helps in creating any serverless project. ✌🏼