Amazon Simple Queue Service with NodeJs (Amazon SQS)
Tags: aws, sqs, javascript, typescript
Published: 2019-11-24 22:23:10

---
title: Amazon Simple Queue Service with NodeJs (Amazon SQS)
published: true
description: This post will walk through the simple process of using Amazon (Simple Queue Service) SQS with NodeJs
tags: aws, sqs, javascript, typescript
---
As we all know Amazon is providing countless services to make our (as developer) life easier. Simple Queue Service or aka Amazon SQS is boon for microservice or serverless application.

![AWS Simple Queue Service (SQS)](https://thepracticaldev.s3.amazonaws.com/i/4d1x4atneg10592brs41.png)

Need for SQS will be realized mostly when one part of the service works very quickly and another at a different pace. For example, You have a job that sends a daily email to your subscribers. It can get hundreds or thousands of email addresses to send emails to in a matter of milliseconds. However, sending emails is a bit slower process. SQS comes handy in this scenario. Using SQS, your first job could send email addresses to queue, a lambda function is triggered whenever there is a new message in the queue and get the email address and finally send it.

This article is divided into three sections. In section one we setup AWS and create two functions with minimum code and deploy them to AWS. In section two, we would actually work on creating an SQS queue and sending messages. In the last section, we work on `functionTwo` to make it receive and process queue messages.

If you just want to see the code, click [Here](https://github.com/nabinadhikari/sqs-demo)

## Section One - Setup

Let's now talk about how can we implement SQS using NodeJs. In this article, we will be using the [Serverless Framework](https://serverless.com), which makes it easier to get started and deploy our application stack to AWS.

First, install the Serverless Framework globally using this command in your terminal. And configure your AWS account using your key and secret key.

```bash
# Install serverless
npm install -g serverless
# Configure AWS
serverless config credentials --provider aws --key <your_key> --secret <your_secret_key>
```

Next, let's create our sample app using serverless.
```bash
serverless create --template aws-nodejs-typescript --path sqs-demo
```

This should create a project inside the `sqs-demo` folder with the following files.
![Initial Project Structure](https://thepracticaldev.s3.amazonaws.com/i/2gvgj8sz3s2zs48rsazm.PNG)

We need to install all the NPM packages in this project. So let's go to that folder and install them. Following command be used, if you are unsure.
```bash
# Change the current directory to project directory
cd sqs-demo
# Install required NPM packages
npm install
```

We will also need `aws-sdk` to send a message. Please install it as developer dependencies using the following command.
```bash
npm i -D aws-sdk @types/aws-sdk
```

To test sending and receiving a message with SQS, we'd need two lambda functions. One to send a message and another to receive and process it. Let's go to `handler.ts` file and delete everything to start fresh.

Let's create two functions `functionOne` and `functionTwo`, where `functionOne` will send messages to SQS and `functionTwo` would receive and process that message. At this point, your `handler.ts` should have these lines of code.

```typescript
// handler.ts
import { Handler, SQSHandler } from 'aws-lambda';
import 'source-map-support/register';

export const funtionOne: Handler = async (_event, _context) => {
  console.log("I am funtionOne");
}

export const funtionTwo: SQSHandler = async (_event, _context) => {
  console.log("I am funtionTwo");
}
```

We need to put these functions into the `serverless.yml` file in order for Serverless to create them. Simply remove the `hello` function under the `functions` section and add these two. We also need to give them SQS permission. `serverless.yml` file after adding these two functions would look like this.

```yml
service:
  name: sqs-demo

custom:
  webpack:
    webpackConfig: ./webpack.config.js
    includeModules: true

plugins:
  - serverless-webpack

provider:
  name: aws
  runtime: nodejs12.x
  region: ap-southeast-2
  apiGateway:
    minimumCompressionSize: 1024 # Enable gzip compression for responses > 1 KB
  environment:
    AWS_NODEJS_CONNECTION_REUSE_ENABLED: 1
  # SQS Permission given
  iamManagedPolicies:
  - 'arn:aws:iam::aws:policy/AmazonSQSFullAccess'

functions:
  funtionOne:
    handler: handler.funtionOne
  funtionTwo:
    handler: handler.funtionTwo
```

You can deploy these functions using Serverless easily. Just put this command.
```bash
serverless deploy
```
This should create two functions in AWS Lambda. Now we move into section two, where we actually start working on the SQS side.

## Section Two - SQS Setup + Send Message

Let's login to [AWS Console](https://aws.amazon.com) and search for `Simple Queue Service` and go there. Chose the appropriate region on the top right corner of the webpage and hit **Get Started Now** button. This should start the process of creating a new queue.

Enter the queue name you want to give it, I'd put `sqs-demo`. Now there are two types of the queue to choose from between `Standard Queue` and `FIFO Queue`. You can read the description to understand them better, but for this demo purpose, we are going with `Standard Queue`. Now hit the `Quick-Create Queue` button to create it.

Now the queue should be created and ready for use. Your page should look like this. Please note the ARN and URL, we will need it later.
![Queue ARN](https://thepracticaldev.s3.amazonaws.com/i/372xek5fu8y4jfdpw5va.png)

Now let's enable our `functionOne` to send a message. To achieve this we need an SQS package from `aws-sdk`. Simply import it at the top of this file `import {SQS} from 'aws-sdk` and add the following code in the `functionOne`. I will explain it later.

```typescript
try {
    const sqs = new SQS({
      region: 'ap-southeast-2'
    });
    const response = await sqs.sendMessage({
      MessageBody: 'hello world',
      QueueUrl: '<your queue url>'
    }).promise();
    console.log(`Message put on queue`, response);
  } catch (e) {
    console.log('Exception on queue', e);
  }
```

In this snippet, we are creating a new SQS object providing our region (mine is ap-southeast-2). Then we are using its `sendMessage` method to send our message to the queue. We'd need to provide `MessageBody` and `QueueUrl`, which we saved from earlier steps. Next, deploy it using the `serverless deploy` command.

Now we test our `functionOne` function. Go to AWS Lambda service and look for `functionOne` in **Functions** tab and click on its name to go to its detail page. On top right corner there is **Test** button, click on it, enter any name if asked for event name and hit Test. It should display execution result and there should be the console of the successful message sent as below.

![functionOne execution success](https://thepracticaldev.s3.amazonaws.com/i/q62uqa3y4h2b44a33dmg.png)

Now if we go to SQS console, we should be seeing **Messages available** with the higher count.
![Higher message count](https://thepracticaldev.s3.amazonaws.com/i/fewjar6xmyuz7s25jgau.png)

At this point, we have successfully send a message to the queue. Now we work on `functionTwo` to receive this message from the queue and process it.

## Section Three- SQS Receive and Process Message

First of all, let's add SQS event trigger for `functionTwo`. To do this we need to edit `serverless.yml` file under `functionTwo` section, we need to add the following after its handler.

```yml
events:
      - sqs: <your sqs queue arn number copied earlier>
```

Now the function should look like this.
![functionTwo SQS trigger](https://thepracticaldev.s3.amazonaws.com/i/q19hywt8g7eyiqk1et73.png)

Next, change the `functionTwo` to log all `Records` from the `event` object. You can replace `functionTwo` with the following code.

```typescript
export const funtionTwo: SQSHandler = async (_event, _context) => {
  console.log("I am function two");
  for (let message of _event.Records) {
    console.log(`Handling message: ${message}`);
  }
}
```

All this `functionTwo` is doing is logging the message to console. You can change it to send an email or whatever you require the queue for. After this let's deploy it one last time.

As soon as your new `functionTwo` is deployed, it will start processing those messages in the queue and wait for new incoming messages.

![Empty queue](https://thepracticaldev.s3.amazonaws.com/i/ib5l37ubad6b6nqrii4d.png)

You can also check logs for `functionTwo`. Mine is as below.
![functionTwo logs](https://thepracticaldev.s3.amazonaws.com/i/i25mgrewuhfay3ka9enf.png)

Overall, it is quite easy to create a new Amazon Simple Queue Service Queue, put message init and trigger another lambda function to pick those messages and process them. 

You can find the complete project in [GitHub](https://github.com/nabinadhikari/sqs-demo)

I would really appreciate your feedback.

Thank you for reading.