Automatically debug and test CI/CD Pipeline with Dashbird
Tags: monitoring, cloud, aws
Published: 2021-02-02 10:32:58

*In this article, we will build a CI/CD pipeline with the AWS Cloud Development Kit (CDK) and debug a test it using Dashbird's observability tool.*

In 2021, **continuous integration and continuous delivery**, or short CI/CD, should be part of every **modern software development process**. It helps **deliver new features and bug fixes much faster**. If the code doesn't have to sit days or weeks before it gets deployed, it has a higher chance that a developer remembers what they were thinking when they wrote it. And with good code quality tools, you can **prevent problems** before **buggy code gets deployed** anywhere.

API-based web or mobile apps are a well-known example for serverless systems, but **CI/CD pipelines can include serverless technology too**! AWS offers **managed services** to **build serverless CI/CD pipelines **in the cloud, so you only have to pay what you use.

> [Learn more about CDK in this crash course article](https://dashbird.io/blog/crash-course-aws-cdk-serverless/).

Prerequisites
-------------

This tutorial requires:

-   [a Dashbird account](https://dashbird.io/register/)
-   [an AWS account](https://portal.aws.amazon.com/billing/signup)
-   [a GitHub account](https://github.com/join)
-   [Node.js](https://nodejs.org/en/)
-   [AWS CDK](https://docs.aws.amazon.com/cdk/latest/guide/getting_started.html)

Setting up the Project
----------------------

The project with all the code needed already exists on GitHub. You just have to **fork it into your own account**. This is required to give AWS CodePipeline access to your GitHub hooks.

To fork the repository, [open it in your browser](https://github.com/kay-is/dashbird-cicd-example) and click the "Fork" button on the top right. 

The next step is to **initialize the project** on your development machine. For this, you can **simply clone your own fork**.

`$ git clone`

`git@github.com:<YOUR_GITHUB_USERNAME>/dashbird-cicd-example.git`

And finally, you need to **update** the `credentials.json`, so the project works with your own fork and AWS account.

`{`

`  "github": {`

  `  "username":"GITHUB_USERNAME",`

`    "repository": "GITHUB_REPOSITORY"`

`  },`

`  "aws": {`

`    "account":"AWS_ACCOUNT_ID",`

`    "region":"AWS_REGION"`

`  }`

`}`

-   `GITHUB_USERNAME` is the name of the GitHub account you forked the project to.
-   `GITHUB_REPOSITORY` should be `dashbird-cicd-example` if you didn't change the repository name
-   `AWS_ACCOUNT_ID` is the ID/number of your AWS account
-   `AWS_REGION` is the AWS region you like to deploy this project to later

You also need a **personal access token** so that the pipeline can access your GitHub repository.

[The GitHub docs](https://docs.github.com/en/github/authenticating-to-github/creating-a-personal-access-token) explain how you get that token. The token needs the `repo` and `admin:repo_hook` scopes. In turn, [the AWS docs](https://docs.aws.amazon.com/secretsmanager/latest/userguide/tutorials_basic.html) explain how to add the token as a secret to the **AWS Secrets Manager**. The token should be a **plain text secret** with the name `github-token`.

Deploying the Project
---------------------

If everything is set up correctly, the project can be **deployed with just two commands**. One command to **bootstrap the CDK** in your AWS account and one to **deploy the project**.

But let's go over the project before we deploy, so you get a sense of what will happen.

In short, this project is basically two systems in one. One is the **CI/CD pipeline** that listens to our repository changes and executes pipeline steps when pushes happen. The other one is the **actual application** we want to deploy.

#### The responsibilities of the files are as follows:

-   `bin/pipeline.js` contains the entry-point for the CDK deployment. It will load the pipeline stack.
-   `lib/pipeline-stack.js` contains the infrastructure code for the CI/CD pipeline. It will be deployed from the command line.
-   `lib/webservice/stage.js` contains the glue code between the pipeline and the application stack. The pipeline will use it for one or multiple deployments of our application.
-   `lib/webservice/stack.js` contains the infrastructure code for the actual application. In this case, an API Gateway with one route that executes a Lambda function. The application will be deployed by our pipeline in the cloud (and to the cloud) when a push happened.
-   `lib/webservice/lambda/handler.js` contains actual application code for the Lambda function. In this case, it contains an error to illustrate Dashird monitoring.

The nice thing about CDK pipelines is that **changes** to our application code, to our application infrastructure, and our pipeline infrastructure are **all handled by that same pipeline**.

Let's deploy the project. If you **haven't used the CDK** before with the account and region combination you added to the `credentials.json`; you need to **bootstrap the CDK first** with the following command:

`$ cdk bootstrap \`

  `--profile account1-profile \`

`  --cloudformation-execution-policies arn:aws:iam::aws:policy/AdministratorAccess \`

`  aws://<AWS_ACCOUNT_ID>/<AWS_REGION>`

If the bootstrap worked correctly, you could now **deploy the project** with this CDK command:

`$ cdk deploy`

This can take a few minutes.

As mentioned before, this will **deploy the CI/CD pipeline** and not the application. After the pipeline is up and running, it will pull the repository and deploy our [API Gateway](https://dashbird.io/knowledge-base/api-gateway/what-is-aws-api-gateway/) and Lambda based application.

If you **open the AWS console** and navigate to CodePipeline (in the category Developer Tools and your chosen AWS region), you can see the **pipeline working on the deployment**. The last step will fail!

Debugging the Error with Dashbird
---------------------------------

Now that the CI/CD pipeline is up and running, we can** finally have look if our application works correctly**. If you **open the CodePipeline** in the AWS console and scroll down to the bottom, you should **see the below error**.

![AWS CodePipeline error](https://dashbird.io/wp-content/uploads/2021/02/CodePipeline-AWS-Developer-Tools-1.png)

Figure 1: AWS CodePipeline error

If you followed [the Dashbird getting started guide](https://dashbird.io/docs/), all your **Lambda functions will be monitored automatically**, even the one that is freshly deployed by our pipeline. This means you can **find that error right in the Dashbird dashboard**; it should look like this:

![Lambda function error in Dashbird ](https://dashbird.io/wp-content/uploads/2021/02/Dashbird.io-2.png)

Figure 2: Lambda function error in Dashbird 

If you have **more than one Lambda function**, you can find it by its generated name, which consists of:

-   The stage name `PreProd`
-   The stack name `WebService`
-   The Lambda function's resource name `Lambda`
-   A unique ID

If you click on this error, you should **see the event that led to it**. Below, you see the **stack trace** that Dashbird provides.

![Dashbird stack trace](https://dashbird.io/wp-content/uploads/2021/02/Dashbird.io-Events-3.png)

Figure 3: Dashbird stack trace

While the filename handler.js, line number 2, and the error type are correct, **the location is different**. It was in the `lib/webservice/lambda` directory on our **development machine**, but when **deployed**, it ended up in the Lambda owned `/var/task directory` inside the AWS Cloud. You should keep in mind to **give your Lambda functions code file a name that helps to find it later**.

To **fix this error**, we simply **remove that erroneous line from our handler.js file**, commit the change and push it to GitHub. This will trigger our **CI/CD pipeline to run again without the error**.

Wrapping up
-----------

Continuous delivery pipelines are crucial for modern software and should be part of your development process, and **serverless technology is a good foundation for building your pipeline**. With the CDK, such pipelines are easy to set up and maintain.\
Because Dashbird follows [observability principles](https://dashbird.io/blog/monitoring-vs-observability/), you don't have to think about monitoring explicitly when creating your systems. You will get **automatic preconfigured insights** into and **alarms** for every AWS service supported by Dashbird, even the Lambda functions that are part of your CI/CD pipeline. This way, you stay up to date on your systems' internal state and **won't get any surprises in production**.

* * * * *

*This project is based on [an AWS own example project](https://aws.amazon.com/blogs/developer/cdk-pipelines-continuous-delivery-for-aws-cdk-applications/) created for the AWS blog. A bit cleaned up for **easier reproduction**, but it's mainly the same, so you can read the AWS article if you want to get some extra information.*