Gitlab CI/CD + NodeJs + pm2
Tags: webdev, devops, javascript, node
Published: 2022-01-14 16:16:59

:hand: Hi this is Suman Sarkar, a web-dev from Kolkata with 5 years of experience in programming and little to none experience with CI/CD. Today I'll talk about how to setup Gitlab CI/CD with self hosted runners.

## :point_right: Things we will cover in this article

1. :confused: What is CI/CD?
2. :thumbsup: Setup a minimal expressjs API with pm2
3. :technologist: Setup our first ever Gitlab pipeline to install & restart our server whenever an update is pushed on the “dev” branch
4. :runner: Install self-hosted runners on a linux server
6. :lock: Register our local runner to Gitlab
7. :key: Add environment variables to Gitlab

## :confused: What is CI/CD?
From my perspective CI/CD or Continuous Integration & Continuous Deployment are processes that you set up for your own convenience so that you don't have to do boring things manually over and over, it is basically automating your workflow when you push an update to your project. Most of us do git pull and then sort of restart the server in order to make the changes into effect, there might be additional steps like building or testing and few other procedures that are specific to your project. I’ll not cover these today, today I’ll only cover how to setup CI/CD for an expressjs application with pm2, Gitlab pipeline and self-hosted runners.

## :thumbsup: Setup a minimal expressjs API with pm2
We start with creating a directory for our Node JS Express API
```shell
mkdir node-cicd-pm2
cd node-cicd-pm2
```
Then we initialise our project with `npm init -y`. This creates a **package.json** file in our project folder with basic information for our project.
Next we add our dependencies by running
```shell
npm i –save express dotenv
```
Lets create our very minimal server by creating our `index.js` and pasting the below mentioned code.
```javascript
const express = require('express');
const dotenv = require('dotenv');

const app = express();
dotenv.config();

app.get('', (req, res) => {
    res.status(200).send('Hello World!');
})

app.listen(process.env.PORT, () => {
    console.log(`Server is running on port http://localhost:${process.env.PORT}`);
})
```
Here, we have required our dependencies express and dotenv then we have added a route that returns 'Hello World!'. We have also added a `.env` file with only 1 variable.
```
PORT="3001"
```
and `ecosystem.config.js` file with the following content
```
module.exports = {
    apps: [{
        name: "node-cicd-pm2",
        script: "./index.js"
    }]
}
```
This will be used later to start our server as a process.

Now, we start our server by running `node index.js` and visit http://localhost:3001/. It works on my machine! :smiling_imp:
![Gitlab CI/CD + NodeJs + pm2](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/544hed6gwezrqpsrjv5i.png)
 
 
## :runner: Setup our first ever Gitlab pipeline
We start with creating a file specifically named `.gitlab-ci.yml`. This is an YML file, if you don't like YML, bad news for you, but you can just copy paste and get things done. 
Now, paste the following code. I'll explain this in detail.
```yml
stages:
  - build_stage
  - deploy_stage
```
Lets talk about stages, stage are the necessary steps that you can group and describe. We have 2 stages build_stage and deploy_stage. Though we are not building anything here but I like to call it the build stage where we'll install the dependencies. We will cover the deploy stage later.
```yml
.base-rules:
  rules:
    - if: '$CI_COMMIT_BRANCH == "dev"'
      when: always
    - if: '$CI_PIPELINE_SOURCE == "push"'
      when: never
    - if: $CI_COMMIT_TAG
      when: never
```
Rules are to describe exactly when your pipeline should run. Here we are specifying that we want to run our pipeline whenever something is pushed onto dev branch by specifying **when** to **always**.
**$CI_PIPELINE_SOURCE** is a special(pre-defined) env. variable provided by Gitlab. It describes the mode our change. These can be the following values push, web, schedule, api, external, chat, webide, merge_request_event, external_pull_request_event, parent_pipeline, trigger, or pipeline. For the same of this article I'll not cover all of them, I am not familiar with most of them anyway.
You can read more about the variables [here on Gitlab](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html).

Next up we have caches. The way every stage works is, it cleans or deletes everything a it has produce during its lifetime. In the build stage we will create a **node_modules** folder which will contain our project's dependencies. When the **build_stage** is finished we don't want it to be deleted. We want it to passed to the **deploy_stage**
```yml
cache: &global_cache
  key: $CI_COMMIT_REF_SLUG
  policy: pull-push
  paths:
    - node_modules/
    - package-lock.json
```
We have created a global cache policy here. The policy is pull-push meaning that the stages using this cache policy can pull from global cache and can push to it as well. In order to create new caches with every update, we must provide a slug or an unique identifier. Here we are using **$CI_COMMIT_REF_SLUG** variable for that. Notice how we are specifying that we only want to cache `node_modules` directory and `package-lock.json` since these are the outputs that are generate with `npm install`.

Let's now define our **build_stage**
```yml
build:
  stage: build_stage
  extends: .base-rules
  script:
    - npm i
  cache:
    <<: *global_cache
    policy: push
  tags:
    - local_runner
```
The build_stage extends the base_rule so that it will run only when something is pushed on the `dev` branch.
In this stage we don't want to pull anything from the global-cache, we just want to push the `node_modules` directory and `package-lock.json` file in the global-cache. We will cover **tags** later int this article.

Later we have the **deploy_stage**
```yml
deploy:
  stage: deploy_stage
  extends: .base-rules
  script:
    - "pm2 start ecosystem.config.js"
  cache:
    <<: *global_cache
    policy: pull
  tags:
    - local_runner
```
In this stage we are pulling the cache from global-cache and then starting our server with `pm2 start` command. By pulling the cache we get our `node_modules` directory with our project dependencies.

If you have followed correctly, you should end up with a file with these content
```yml
stages:
  - build_stage
  - deploy_stage

.base-rules:
  rules:
    - if: '$CI_COMMIT_BRANCH == "dev"'
      when: always
    - if: '$CI_PIPELINE_SOURCE == "push"'
      when: never
    - if: $CI_COMMIT_TAG
      when: never

cache: &global_cache
  key: $CI_COMMIT_REF_SLUG
  policy: pull-push
  paths:
    - node_modules/
    - package-lock.json

build:
  stage: build_stage
  extends: .base-rules
  script:
    - "node --version"
    - npm i
  cache:
    <<: *global_cache
    policy: push
  tags:
    - local_runner

deploy:
  stage: deploy_stage
  extends: .base-rules
  script:
    - "pm2 start ecosystem.config.js"
  cache:
    <<: *global_cache
    policy: pull
  tags:
    - local_runner
```

## :computer: Install self-hosted runners on a linux server
A little bit of background on runners, runners are like workers who does something that a computer should do. Like executing any commands or installing your project dependencies. Behind the scene they are docker containers provided by Gitlab. By default Gitlab uses a Ruby container but you can specify your container type. In this article though we will not use Gitlab's runners, we will install our own runner which is an open-source application made by Gitlab and maintained by the dev community. Self hosted runners are completely free so you don't have to worry about money :money_mouth_face:.

Installing the runner on your server is easy, you just have to run few commands. Visit this [page](https://docs.gitlab.com/runner/install/) for instruction related to your OS environment. I'm running Ubuntu 20.10 so I'll follow with GNU/Linux Binary guide.. If you are using any debian machine then follow me.. Fire up your terminal and run the following commands..
```shell
sudo curl -L --output /usr/local/bin/gitlab-runner "https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-amd64"
sudo chmod +x /usr/local/bin/gitlab-runner
sudo useradd --comment 'GitLab Runner' --create-home gitlab-runner --shell /bin/bash
sudo gitlab-runner install --user=gitlab-runner --working-directory=/home/gitlab-runner
sudo gitlab-runner start
sudo gitlab-runner status

```
Step by step we get the binary, give it executable permissions, create a user called **gitlab-runner** to run the runners process and then start our gitlab-runner service. The gitlab-runner user is created for security purpose so that it doesn't run as root user. It is generally advised by people who are smarter than me and have was more knowledge about operating systems :sweat_smile:.
Now, after the last command you should see something like this
![Gitlab CI/CD + NodeJs + pm2 - Gitlab runner service is running](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/47x27khsncbgu1cjqb3v.png) Again, it worked on my machine so I'm good! :sweat_smile:. We are not done with this step though.. We have to login as the **gitlab-runner** user and install node,  npm and pm2. I could not find any reference to what is the default password of gitlab-runner user so I will just reset it using the passwd command.
```shell
passwd gitlab-runner
```
Setup your new password and login as the gitlab-runner user by running `su gitlab-runner`
For install node I'm using **nvm**. Just follow the same process mentioned below and you should have everything you need.
```shell
curl https://raw.githubusercontent.com/creationix/nvm/master/install.sh | bash
source ~/.bashrc
```
this should install nvm in you machine.
Next, we install node and pm2 globally,
```
nvm install 16.13.2
npm i -g pm2
```

## :lock: Register our local runner to Gitlab
We are almost done with our setup..
Now, we need to register our runner to Gitlab, to do this go to Setting > CI/CD in your repository and expand the "Runners" section.
At the left side you should see "Specific runners" section.
The token should look something like this "fy7f3BqhVzLq3Mr-xxxx"
In your local machine or wherever you have installed you runner just run 
```
sudo gitlab-runner register
```
This should prompt you to specify an instance URL. Type `https://gitlab.com` and press enter.
Then paste the registration token that you found on Gitlab and press enter, next provide a description for your runner
the most important step, providing a tag for your runner or tags. In the `.gitlab-ci.yml` file I had mention the tags as **local_runner** so I will put that here. You can add multiple tags separated by comma but that's not mandatory. Tags will identify the runners to do their job. At last choose **shell** as the executor. The End? Not yet! :'(

## :key: Add environment variables to Gitlab
Now we need to add env variable to Gitlab CI/CD section so that the we can provide a **PORT** to our application. This is important because .env file is not commited to your version control. We add our env variable PORT under Setting > CI/CD > Variables section and we add the variable as protected. Next, super important - we need to make our dev branch as protected branch. Otherwise it won't fine the variables. You can do this from Settings > Repository > Protected branches section in your repo.

:white_check_mark: That is it, we are done with our pipeline setup. If everything is done correctly, when you commit a change on your dev branch it should trigger a pipeline with 2 job and you runner should start the pm2 process at 3001 port.

Thanks for reading this article :technologist: If you face any problems, let me know in the comments down below! :slightly_smiling_face: 
Happy hacking!

![Happy hacking](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/643zyaj7h6i1cyx1mcrq.gif)