Setup a node app with Elastic Beanstalk and CI/CD
Tags: node, cicd, javascript, aws
Published: 2023-02-12 15:48:20

## Introduction

Suppose we have a node express application that we need to publish to the cloud. After searching, there aren't many relevant articles and resources covering this topic, so I hope this is helpful to fit the exact needs. AWS CloudFormation is a good solution but it has a big barrier to entry. Instead, we will use AWS Elastic Beanstalk to do a lot of the heavy lifting. It will enable all the required AWS resources for my application; in addition, we will automate this process so that when the code is updated with new changes. The code will automatically deploy to Elastic Beanstalk with the latest changes.

## What this article is not
There are many ways to write a node express application. This blog will not attempt to convince how to design or where to host the repository. 

## Requirements
### Node application
We need a node express application. It is an API that is used only for the purposes of stubbing a real node application. 

### AWS CLI setup and access
AWS provides a free tier for to work with. Check out tutorials online for how to create a free tier account and set up the terminal with the AWS CLI.

## Concepts

### Elastic Beanstalk

This service, provided by AWS, does a lot of the heavy lifting for procuring all the infrastructure needed for the application to work. It will connect up all the required dependencies. 

### Elastic Beanstalk Environment

An environment is the space to deploy an application to. All the necessary AWS resources are associated to the environment in order to facilitate the needs of the platform. This tutorial will only use one environment; however, we can have as many environments needed to meet the software lifecycle means.

### Application Version

The version is associated to a source bundle, identified by a label. It represents an application artifact that will get deployed to an Elastic Beanstalk environment.

### Source Bundle

The artifact of the code and the reference for where it is stored. In this case, we will use S3 as the repository for the source bundle.

### Simple Storage Service (S3)

This service is an object store that allows saving any type of data for accessing later. For example, it can store zip files. The source bundle will be stored here.

## How To - Manual Steps

First, we need to carry out the steps manually. This will ensure all dependencies, permissions, and sequence of events are correct. This will ease any hiccups when automating this pipeline in Gitlab.

### Procure an application to upload to Elastic Beanstalk

You can use this example in [https://gitlab.com/ayogsynergy/eb_node_sandbox/-/tree/step1_helloworld](https://gitlab.com/ayogsynergy/eb_node_sandbox/-/tree/step1_helloworld)

### Upload Source Bundle to S3

- Bundle the source code from VCS into a zip file.
    - `git archive -v -o eb-node-sandbox.zip --format=zip HEAD`
- Create a S3 bucket.
    - This example will use `manual-eb`
- Upload the bundle to the S3 bucket.
    - `aws s3 cp eb-node-sandbox.zip s3://manual-eb`

### Create application version

Elastic Beanstalk expects an application version to be specified in order to instantiate an environment.

Usage

```bash
aws elasticbeanstalk create-application-version \
--application-name ${APP_NAME} \
--version-label ${VERSION_LABEL} \
--description="new version ${VERSION_LABEL}" \
--source-bundle S3Bucket=${S3Bucket},S3Key=${S3Key} \
--auto-create-application \
--region ${AWS_REGION}

```

Example

```bash
aws elasticbeanstalk create-application-version \
--application-name eb-node-sandbox \
--version-label v0.0.01 \
--description="new version v0.0.01" \
--source-bundle S3Bucket=manual-eb,S3Key=eb-node-sandbox.zip \
--auto-create-application \
--region us-east-1
```

### Describe application version

- Confirm the application version is created.
    - `aws elasticbeanstalk describe-application-versions --application-name eb-node-sandbox`

## Create the Elastic Beanstalk environment

Next, instantiate an environment that will accept the application version to deploy.

### Create a configuration template

```bash
aws elasticbeanstalk create-configuration-template \
--application-name eb-node-sandbox \
--template-name v1-template \
--solution-stack-name "64bit Amazon Linux 2 v5.6.4 running Node.js 16"
```

### Describe the configuration template

```bash
aws elasticbeanstalk describe-configuration-settings --application-name eb-node-sandbox --template-name v1-template
```
## Create Elastic Beanstalk environment

- Create an options file for use of making environment
    
    ```bash
    [
        {
            "Namespace": "aws:autoscaling:launchconfiguration",
            "OptionName": "IamInstanceProfile",
            "Value": "aws-elasticbeanstalk-ec2-role"
        }
    ]
    ```
- Create an Elastic Beanstalk Environment for the application
    
    ```
    aws elasticbeanstalk create-environment \
    --application-name eb-node-sandbox \
    --template-name v1-template \
    --version-label v0.0.01 \
    --environment-name eb-node-sandbox-env \
    --option-settings file://options.txt
    ```

### Describe the environment

- Confirm the environment is created
    - `aws elasticbeanstalk describe-environments --environment-names eb-node-sandbox-env`

At this point, there is a running Elastic Beanstalk environment accessible via the internet.

Launch the instance and send a request to `GET /` to verify the appropriate `Hello World` response is returned. 

![Hello World Application](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/grz1fbdw2h3912vxwqxs.png)

## Update the environment

Now, let’s simulate a new change in the source code. We will need to publish the new changes. We will update the Elastic Beanstalk environment with the new version.

- Upload new source bundle with a new key name to the S3 bucket
- Create a new application version
    - Associate it to the new respective bundle
    - `eb-node-sandbox-1.0.0-02`
- Update the _existing_ Elastic Beanstalk environment with the new application version
    
    ```
    aws elasticbeanstalk update-environment \
    --application-name eb-node-sandbox \
    --environment-name eb-node-sandbox-env \
    --version-label eb-node-sandbox-1.0.0-02 \
    --region us-east-1
    ```

Run this command to update the environment. It will take some time. Wait until the environment deployment is finished.

---

Try to do this in one sitting, if possible. Otherwise, terminate the environment as AWS will keep the meter running for the infrastructure that you are leveraging in your environment (e.g. EC2). This will help avoid unexpected costs.

---

### Troubleshooting (log analysis)

If the application does not respond successfully, take a look at the logs. Here is some sample log output below.

```
----------------------------------------
/var/log/web.stdout.log
----------------------------------------
Jan 15 14:50:25 ip-172-31-7-140 web: > Elastic-Beanstalk-Sample-App@0.0.1 start
Jan 15 14:50:25 ip-172-31-7-140 web: > node app.js
Jan 15 14:50:25 ip-172-31-7-140 web: Server running at http://127.0.0.1:8080/
Jan 15 15:06:21 ip-172-31-7-140 web: > eb_node_sandbox@1.0.0 start
Jan 15 15:06:21 ip-172-31-7-140 web: > node server.js
Jan 15 15:06:21 ip-172-31-7-140 web: Running on Port: 8080
Jan 15 15:18:20 ip-172-31-7-140 web: > eb_node_sandbox@1.0.0 start
Jan 15 15:18:20 ip-172-31-7-140 web: > node server.js
Jan 15 15:18:21 ip-172-31-7-140 web: Running on Port: 8080
```

By convention, it will default to `npm run start` to boot your application. Make sure your `package.json` has this script defined. It will also run `npm install` prior to running your application, if it detects a `package.json` file.

## CI/CD

Now, automate this publish process to trigger every time the repository code is updated. This example will leverage Gitlab.

- Create a `.gitlb-ci.yml` file to indicate how/what to include in the CI/CD pipeline

```bash
stages:
  - build
  - run

variables:
  APP_NAME: ${CI_PROJECT_NAME}
  ENV_NAME: ${ENV_NAME}
  APP_VERSION: "1.0.0"
  S3BUCKET: "manual-eb"
  AWS_ID: ${AWS_ID}
  AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID}
  AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}
  AWS_REGION: us-east-1

create_app_version:
  stage: build
  image: python:latest
  allow_failure: false
  script: |
    pip install awscli
    echo "Creating zip file"
    VERSION_LABEL=${APP_NAME}-${APP_VERSION}-${CI_PIPELINE_ID}
    S3Key="${VERSION_LABEL}.zip"
    echo "Zipping: ${S3Key}"
    git archive -v -o ${S3Key} --format=zip HEAD

    echo "Creating bundle..."
    echo "Uploading bundle: ${S3Key} ..."
    aws s3 cp ${S3Key} s3://${S3BUCKET}/${S3Key} --region ${AWS_REGION}
    
    echo "Creating app version"
    aws elasticbeanstalk create-application-version \
      --application-name ${APP_NAME} \
      --version-label ${VERSION_LABEL} \
      --description="new version ${VERSION_LABEL}" \
      --source-bundle S3Bucket=${S3BUCKET},S3Key=${S3Key} \
      --auto-create-application \
      --region ${AWS_REGION}
  only:
    refs:
      - main
deploy_version_to_env:
  stage: run
  image: coxauto/aws-ebcli
  when: manual
  script: |
    VERSION_LABEL=${APP_NAME}-${APP_VERSION}-${CI_PIPELINE_ID}
    
    echo "Deploying version: ${VERSION_LABEL} to env: ${ENV_NAME}"
    aws elasticbeanstalk update-environment \
      --application-name ${APP_NAME} \
      --environment-name ${ENV_NAME} \
      --version-label ${VERSION_LABEL} \
      --region=${AWS_REGION}
  only:
    refs:
      - main
```

The pipeline is split in two stages:

- Stage
    - create_app_version
        - Bundle the source code at HEAD and publish to the S3 bucket
        - Create a new application version and associate it to the bundle that was uploaded
- Deploy
    - deploy_version_to_env (manual)
        - Upload the new application version to the (already established and running) Elastic Beanstalk environment

This pipeline will trigger when the `main` branch is updated.

### Setup CI/CD pipeline env variables

- Settings → CI/CD
- Variables → Expand
- Add the following keys
    - APP_NAME: eb-node-sandbox
    - AWS_ID: xxxxx
    - AWS_ACCESS_KEY_ID: xxxx
    - AWS_SECRET_ACCESS_KEY: xxxxxxx


![Elastic Beanstalk Environment Variables](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qmie044eo2fm4i3qr70b.png)



## Conclusion

If you are following this tutorial, make sure to terminate your environment. This will help you avoid unexpected running costs that AWS will bill you for. Refer to the action drop-down menu to `Terminate environment`


![Terminate Environment](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/otqbzy1g4r9y4mr7qh3n.png)

Avoiding manual processes by automating can save a lot of time and unintended mistakes. It is possible to also add a test stage that is not allowed to fail in order to prevent publishing bad software to your environment.