Using Codeship for NodeJS Application Deployments
Tags: codeship, docker, node
Published: 2017-07-12 22:29:19


---
title: Using Codeship for NodeJS Application Deployments
published: true
description: In the first part of this series, I walked through developing a simple API using NodeJS, Express, and PostgreSQL using Docker Compose. In this post, I will cover creating a CI/CD pipeline using Codeship Pro.
tags: codeship, docker, node
cover_image: https://blog.codeship.com/wp-content/uploads/2017/06/Heading_Using-Codeship-for-NodeJS-Application-Deployments.jpg
---

> This post was originally published on the [Codeship Blog](https://blog.codeship.com/using-codeship-for-nodejs-application-deployments/) and is posted here with the author's permission.

[In the first part of this series][1], I walked through developing a simple `todo` API using NodeJS, Express, and PostgreSQL using [Docker Compose][2]. In this post, I will cover [creating a CI/CD pipeline using Codeship Pro][3].

## Requirements for This Tutorial

This tutorial requires you to have a few items before you can get started:

*   Install [Docker Community Edition][4]
*   Install [Docker Compose][5]
*   Download [Todo App Example - Dockerized branch][6] from our Codeship Library
*   [Codeship][7] account
*   [Codeship Jet CLI][8]
*   [Heroku][9] account
*   [GitHub][10], [Bitbucket][11], or [GitLab][12] account
*   You have read Part 1: [Using Docker Compose for NodeJS Developmment][1]

In our example, we are deploying our code directly into Heroku without Docker. While we will be using Docker locally and in CI/CD, Codeship Pro allows us to deploy Docker apps to production even when production is not using Docker, such as classic Heroku. The deployment in this case can be exchanged with any other server, image repository, or Docker host.

Essentially, you can modify this example to suit your needs, by using any Docker image that does what you need, *eg*, deploy to AWS. Our [documentation][13] has examples that you can review.

Once you have everything set up, you can move on to the next section.

## Testing NodeJS Apps with Codeship

[Continuous integration][14] is defined as follows:

> In software engineering, continuous integration (CI) is the practice of merging all developer working copies to a shared mainline (like a master branch) several times a day.

We want to automate all of this with Codeship so every pushed commit will be built and tested prior to allowing a merge. Let’s set up the necessary files and test our pipeline locally.

### Codeship services file

The first file we need to create is the [Codeship services definition file][15]. This will instruct Codeship as to the services we need during the build.

```shell
/> touch codeship-services.yml
```


The `codeship-services.yml` syntax is directly influenced by Docker Compose syntax, and the `docker-compose.yml` file created in the [last post][1] gets us most of the way there. Open the `codeship-services.yml` in your editor, copy the following code, and paste there.

```yaml
web:
  build: .
  links:
    - postgres
  environment:
    DATABASE_URL: postgres://todoapp@postgres/todos
  cache: true
postgres:
  image: postgres:9.6.2-alpine
  environment:
    POSTGRES_USER: todoapp
    POSTGRES_DB: todos
```


The differences from the original `docker-compose.yml` file to define the `codeship-services.yml` file are quite minimal here. Next we can define the actual steps to run during the build and automated tests.

### Codeship steps file

The next file to create is the `codeship-steps.yml` file, which will tell Codeship what steps to run and in what order. You can [learn more about the `codeship-steps.yml` file in our documentation][16]. Go ahead and create the new steps file and open it up in your editor.

```shell
/> touch codeship-steps.yml
```


Each step defined in the `codeship-steps.yml` file is based on what you define as your pipeline. This example starts with a `lint` test and `integration` test, and eventually will deploy the code as well. Your pipeline can be completely customized to suit your specific needs.

```yaml
- type: parallel
  steps:
  - name: lint
    service: web
    command: npm run lint
  - name: tests
    service: web
    command: npm test -- --forceExit
```


The steps here run in parallel: `lint` and `tests`. Running our tests simultaneously works great here, because if either of these tests fail, the build will fail, and saves us some time overall. You can find more information on Codeship Pro step types [here][17].

### Bin scripts

Now that things are more automated, we end up introducing a small race case that we can easily get around with some simple scripts. As the project is built, the integration tests require that `Postgres` is running to perform the migrations. We can't guarantee that will happen with the `codeship-services.yml` file alone. One approach is to check that the database is available before starting the migration.

```shell
/> mkdir bin && touch bin/{wait-for-postgres,ci}
/> chmod +x bin/**
```


#### wait-for postgres

The contents of the `wait-for-postgres` script are the following:

```shell
#!/bin/sh
# wait-for-postgres
set -e

TIMEOUT=60
COUNT=0

until pg_isready -h "postgres" -p "5432" || [ $COUNT -eq $TIMEOUT ];
do
  echo $COUNT
  echo $TIMEOUT
  sleep 1
  COUNT=$((COUNT+1))
done
```


I am relying on the [`pg_isready`][18] function to check for the readiness of the `postgres` database. If it isn't ready, we sleep for a second and try again. We need to make a small change to the `Dockerfile` to add the `pg_isready` function.

```Dockerfile
RUN apk update && apk add postgresql
```


The function times out after 60 seconds, but you can tweak that if you see it taking longer. It typically takes only a few seconds to connect.

#### ci

The `ci` script is as follows:

```shell
#!/bin/sh
set -e

bin/wait-for-postgres
time npm run migrate

time $1
```


Not too much heavy lifting here. This script runs the `wait-for-postgres` script to finish up, then it will perform the migration, and the last line takes the first parameter and will run that. This allows me to change a line in `codeship-steps.yml` to run this script first.

```yaml
- name: tests
  service: web
  command: bin/ci "npm test -- --forceExit" # you only need to change the command here
```


### Local testing with Codeship Jet CLI

We are able to test our setup locally using `Codeship Jet CLI`. If you haven’t done it yet, check out our [Getting Started with Codeship Jet CLI documentation][8]. `Codeship Jet CLI` will run through the steps file just as it would on Codeship. This is a quick way to catch errors early before committing and pushing to your repository.

```shell
/> jet steps
## build stuff, run tests ##
{StepFinished=step_name:"tests" type:STEP_FINISHED_TYPE_SUCCESS}
```


If you see the final result above, you have set everything up correctly. If you receive instead `type:STEP_FINISHED_TYPE_ERROR`, something didn't go right and you should check some things. `Codeship Jet CLI` will produce a log that you can review to try to locate the problem. Having the ability to catch any errors prior to pushing to your repository makes this a particularly powerful tool in the Codeship arsenal.

The finished version of this project is also available on [GitHub][19], in case you get stuck and want to skip ahead.

## Setting Up Your Codeship Pro Project

The next phase is to set up the Codeship project, so that new pushes into your repository will kick off the build. In this step, we will make sure you have a repository set up, create the project, and push a build.

### Source code management setup

Log into your GitHub, Bitbucket, or GitLab account and create a new repo there that you have **admin** access to. Grab the clone URL and switch to the [Codeship application][20].

### Create project

When your repo is ready to go, you can now set up the Codeship project.

*   Navigate to Projects, then click the "New Project" button.

<img src="https://blog.codeship.com/wp-content/uploads/2017/06/image4-1.png" alt="New Project" width="1999" height="1249" class="aligncenter size-full wp-image-5120" />

*   Connect your SCM by selecting the source code management (SCM) tool you set up in the previous step.

<img src="https://blog.codeship.com/wp-content/uploads/2017/06/image2-1.png" alt="SCM" width="1999" height="1249" class="aligncenter size-full wp-image-5121" />

> If you originally signed up with an SCM different than above, we will connect to the service during this step.

*   Choose your repository by copy/pasting the Repository Clone URL link from the previous step. Click the Connect button.

<img src="https://blog.codeship.com/wp-content/uploads/2017/06/image1-1.png" alt="Repository Clone URL" width="1999" height="1249" class="aligncenter size-full wp-image-5122" />

<img src="https://blog.codeship.com/wp-content/uploads/2017/06/image3-1.png" alt="Connect repository" width="1999" height="1249" class="aligncenter size-full wp-image-5123" />

*   Configure your project by clicking the "Select Pro Project" button.<img src="https://blog.codeship.com/wp-content/uploads/2017/06/image5-1.png" alt="Select Pro Project" width="1999" height="1249" class="aligncenter size-full wp-image-5124" />

Your project is set up at this time, and any code committed and pushed to the repository will now run builds automatically.

### Push your first build

Let's run a build now. In your command line, or using your favorite `git` tool, make sure the repo is initialized, connect the remote, add the files, commit, and push.

```shell
/> git init
/> git remote add origin &lt;&lt;YOUR_REPOSITORY_CLONE_URL>>
/> git add .
/> git commit -am "initial build"
/> git push -u origin master
```


If you head back to Codeship and click into your project, you should see the build running. Make sure at this point you get a green build. Otherwise you may need to go back and check some things:

*   Make sure you have added all of your files to your repo.
*   Review your `codeship-services.yml` and `codeship-steps.yml` files.
*   Run `Codeship Jet CLI` locally to double-check it works locally.

Once you have the CI working, you can move on to the deployment section, where we will automatically deploy our code to Heroku.

## Continuous Deployment to Heroku with Codeship Pro

So far, we have the integration steps running, and every new commit will run tests to make sure that the code is ready to merge.

When the branch is `master`, however, we want to run another step to actually **deploy** the application when the tests are passing.

### Creating the Heroku app

You can create an application using the Heroku UI, which is the simplest way to get started. If you are familiar with the Heroku CLI, you can also perform these steps using that as well. Once you have the application created, there are a some steps to get everything Codeship needs for the deployment.

#### Heroku PostgreSQL add-on

These steps will set up the Heroku PostgreSQL add-on for your app to use as its database when deployed.

1.  Click **Resources**.
2.  Under **Add-ons**, search for `postgres`.
3.  In the results dropdown, click **Heroku Postgres**.
4.  Leave the selection as **Hobby Dev - Free**, then click **Provision**.

#### Get your Heroku API key

1.  Click on your avatar in the upper right, then click **Account Settings**.
2.  Near the bottom of the settings page, you will find an API key. Click **Reveal**.
3.  Copy the API key.

Set this up in a `deployment.env` file with your terminal:

```shell
/> echo "HEROKU_API_KEY=YOUR_API_KEY_HERE" > deployment.env
```


#### Create the Heroku procfile

This is a simple one-liner, but it’s required for Heroku to run the application:

```shell
/> echo "web npm start" > Procfile
```


### Encrypting with Codeship Jet CLI

Since we have some sensitive keys that we will need to use, `Codeship Jet CLI` provides some encryption tools to secure your keys. In this instance, it’s an environment variable needed for the deployment, however you can encrypt all of your secrets. For more information, you can read the Codeship documentation on [Encrypting Environment Variables][21].

In the Codeship UI, follow these steps:

1.  Click **Projects**, then the project you are currently working in.
2.  Click **Settings**, then click **General**.
3.  Locate and copy the `AES Key`.

In your terminal, include:

```shell
/> echo "YOUR_AES_KEY" > codeship.aes
```


> NOTE: Make sure that both `codeship.aes` and `deployment.env` are ignored in your `.gitignore` file. Since they contain sensitive data, you don't want these to be pushed into your repository.

All that's left to do is encrypt the `deployment.env` file. We do this using `Codeship Jet CLI`.

```shell
/> jet encrypt deployment.env deployment.env.encrypted
```


The `deployment.env.encrypted` will then be included in your repository and decrypted in Codeship.

### Adding the Heroku deployment service

In Codeship Pro, we are able to create any service we need using Docker. What this means to you is that if it runs on Docker, it runs on Codeship. In this example, we are using a service that Codeship provides specifically for [Heroku deployment][22]. Let's add the service to the end of our `codeship-services.yml` file.

```yaml
web:
  build: .
  links:
    - postgres
  environment:
    DATABASE_URL: "postgres://todoapp@postgres/todos"
  cached: true
postgres:
  image: postgres:9.6.2-alpine
  environment:
    POSTGRES_USER: todoapp
    POSTGRES_DB: todos
deploy: # added this service
  image: codeship/heroku-deployment
  encrypted_env_file: deployment.env.encrypted
  volumes:
    - ./:/deploy
```


Note that we add the `encrypted_env_file` here. This instructs Codeship as to which file to use to find the Heroku API key used inside the container.

### Adding the deployment step

The only step left is to tell Codeship when to use the `deploy` service. Open up the `codeship-steps.yml` file and add the following:

```yaml
- type: parallel
  steps:
  - name: lint
    service: web
    command: npm run lint
  - name: tests
    service: web
    command: bin/ci "npm test -- --forceExit"
- name: deploy # added this step
  tag: master
  service: deploy
  command: codeship_heroku deploy /deploy nodejs-express-todoapp
```


What this `codeship-steps.yml` file indicates now is that if the branch, listed as `tag`, is equal to `master`, it will run the deploy service. If we push to any other branch, everything will run **except** the deployment. You can read more about [the `tag` and `exclude` attributes and limiting steps in Codeship Pro][23].

The image `codeship/heroku-deployment` has the command `codeship_heroku deploy`, which accepts two parameters: the path of the files and the name of the Heroku app. In this case, these are `/deploy` and `nodejs-express-todoapp`. As long as all tests pass and the branch is `master`, our code will be deployed to Heroku.

The `codeship/heroku-deployment` image is running several commands to test for proper access to the application, tarball the code, and deploy to Heroku with their API. Just to reiterate, this can be anything you need it to be, *eg*, deployment to AWS. If it runs in a Docker container, Codeship Pro can run it.

### Starting the build on Codeship Pro

Now we can push these changes into our SCM and let Codeship take it from here.

> Make sure you have added your `codeship.aes` and `deployment.env` files to `.gitignore`. You want to ignore these files so you are not publicly exposing secrets in your repository. Do that now if you haven't already.

```shell
/> git add .
/> git commit -am "adding deployment"
/> git push
```


Head over to your Codeship Pro project and watch your build and deployment from the dashboard.

## Conclusion

The two posts in this series cover quite a bit. Initially, we are tasked with testing and deploying our application manually. This process is not only tedious, it takes up someone’s time and is error prone. The [first post][1] covered setting up our local development and testing workflow using Docker Compose. In this post, we then used Codeship Pro and Jet CLI to automate our steps even further. Now we are able to push a new commit to `master`, which then runs our tests and deploys to Heroku all through Codeship Pro.

Here are some of the resources for you from the two posts:

#### Docker

*   [Docker Community Edition][4]
*   [Dockerfile Reference][24]
*   [Docker Compose Overview][25]
*   [Docker Compose CLI][26]

#### Docker Images Used

*   [Node][27]
*   [PostgreSQL][28]
*   [Codeship Heroku Deployment][22]

#### Codeship Pro

*   [Codeship Pro Documentation][13]
*   [Codeship Jet CLI Docs][8]
*   [Encrypting Environment Variables][21]
*   [Codeship Steps Docs][16]
*   [Codeship Services Docs][15]

#### Article Resources

*   [Using Docker Compose for NodeJS Development][1]
*   [NodeJS Express Todo App Repo][29]
*   [Alpine Based Docker Images Make a Difference in Real World Apps][30]

#### Additional Tools and Resources Used

*   [Jest][31]
*   [Express JS][32]
*   [Heroku][9]
*   [GitHub][10]
*   [Bitbucket][11]
*   [GitLab][12]
*   [PostgreSQL][33]
*   [Todo Backend][34]


 [1]: https://blog.codeship.com/using-docker-compose-for-nodejs-development/
 [2]: https://docs.docker.com/compose/
 [3]: https://codeship.com/features/pro/
 [4]: https://store.docker.com/search?type=edition&offering=community
 [5]: https://docs.docker.com/compose/install/
 [6]: https://github.com/codeship-library/nodejs-express-todoapp/archive/dockerized.zip
 [7]: http://codeship.com/
 [8]: https://documentation.codeship.com/pro/builds-and-configuration/cli/
 [9]: https://www.heroku.com/
 [10]: https://github.com/
 [11]: https://bitbucket.org/product
 [12]: https://about.gitlab.com/
 [13]: https://documentation.codeship.com/pro/
 [14]: https://en.wikipedia.org/wiki/Continuous_integration
 [15]: https://documentation.codeship.com/pro/builds-and-configuration/services/
 [16]: https://documentation.codeship.com/pro/builds-and-configuration/steps/
 [17]: https://documentation.codeship.com/pro/builds-and-configuration/steps/#group-steps
 [18]: https://www.postgresql.org/docs/9.3/static/app-pg-isready.html
 [19]: https://github.com/codeship-library/nodejs-express-todoapp
 [20]: https://app.codeship.com
 [21]: https://documentation.codeship.com/pro/builds-and-configuration/environment-variables/#encrypting-your-environment-variables
 [22]: https://hub.docker.com/r/codeship/heroku-deployment/
 [23]: https://documentation.codeship.com/pro/builds-and-configuration/steps/#limiting-steps-to-specific-branches--tags
 [24]: https://docs.docker.com/engine/reference/builder/
 [25]: https://docs.docker.com/compose/overview/
 [26]: https://docs.docker.com/compose/reference/overview/
 [27]: https://store.docker.com/images/node?tab=description
 [28]: https://store.docker.com/images/022689bf-dfd8-408f-9e1c-19acac32e57b?tab=description
 [29]: https://github.com/codeship-library/nodejs-express-todoapp/
 [30]: https://blog.codeship.com/alpine-based-docker-images-make-difference-real-world-apps/
 [31]: https://facebook.github.io/jest/
 [32]: https://expressjs.com/
 [33]: https://www.postgresql.org
 [34]: http://todobackend.com/