Monitoring Jenkins pipelines using OpenTelemetry and New Relic
Tags: devops, opentelemetry
Published: 2023-03-20 19:01:00

## Shift left by monitoring your continuous integration and continuous deployment pipelines for errors.

Jenkins is a popular open-source automation tool for implementing continuous integration (CI) and continuous development (CD) workflows called pipelines. Continuous integration and continuous deployment are development practices that can help you deploy your applications faster—and with fewer bugs.  But if errors or other issues come up in your pipelines, it can negatively impact deployment and other steps in the DevOps lifecycle. That's why it’s important to monitor not just your applications but also the automation processes around development, including your CI and CD tools. This can help you shift left, which means incorporating more testing on the left side of the DevOps lifecycle, as shown in the next image.

![Dev Ops diagram shows an infinity symbol with Dev on the left (planning and building) and Ops on the right (deployment and operations).](https://newrelic.com/sites/default/files/2022-10/DevOps%20SDLC%20Diagram_Lightmode.png)

Shifting left isn't just about testing your code and automating the build and deployment process. It's about monitoring and testing your CI and CD pipelines, too.

In this post, you’ll learn how to monitor your Jenkins jobs and pipelines using the Jenkins OpenTelemetry plugin and a New Relic quickstart integration. After you are finished, you'll be able to:

- Visualize key metrics on your Jenkins pipelines, such as how long your jobs are taking and error rates.
- Use distributed tracing to pinpoint where issues are coming from in your pipelines.
- Create alerts on your Jenkins pipelines.

The next image shows a New Relic dashboard with some of the Jenkins metrics you’ll be able to visualize.

<figure>
  ![New Relic dashboard shows Jenkins metrics such as length of time for jobs, error rate, and whether they succeeded or failed.](https://newrelic.com/sites/default/files/2022-10/jenkins-quickstart-dashboard.png)
  <figcaption>Install the Jenkins quickstart dashboard for a pre-built view of your key pipeline metrics.</figcaption>  
</figure>

With this integration, you can shift left and get observability for the build and deploy phases of the DevOps lifecycle. Check out the next video or continue reading to get started.

{% youtube ZCF75LTtdz0 %}

## Prerequisites
If you don’t have Jenkins yet, you need to install it. Alternatively, you can use Docker to follow along with this example.  If you’re using Docker, you can get an instance of Jenkins up and running with the following command: 

`docker run --name jenkinsci -d -e OTEL_LOGS_EXPORTER="otlp" -p 8080:8080 jenkins/jenkins:lts`

## Setting up the OpenTelemetry plugin

1. Log into Jenkins (usually `http://localhost:8080`) and log in with your username and password.

2. From the Jenkins dashboard, select **Manage Jenkins** from the lefthand pane.

3. Select **Manage Plugins**.
   ![Manage Jenkins page with Manage Jenkins selected in the lefthand pane.](https://newrelic.com/sites/default/files/2022-10/jenkins-otel-1.png)

4. Select the **Available** tab and type `OpenTelemetry` in the search bar.
   ![OpenTelemetry option comes up when typing OpenTelemetry in search bar.](https://newrelic.com/sites/default/files/2022-10/jenkins-otel-2.png)

5. Select the **OpenTelemetry** checkbox and choose **Install without restart**.
   ![Plugin Manager in Jenkins](https://newrelic.com/sites/default/files/2022-10/jenkins-otel-3.png)

6. After the installation is complete, select **Restart Jenkins when installation is complete and no jobs are running**.
   ![Installing Plugins/Upgrades page in Jenkins](https://newrelic.com/sites/default/files/2022-10/jenkins-otel-4.png)

7. Log into Jenkins again and select **Manage Jenkins**.

8. Under **System Configuration**, select **Configure System**.
   ![Manage Jenkins dashboard in Jenkins has Configure System selected.](https://newrelic.com/sites/default/files/2022-10/jenkins-otel-5.png)

9. Scroll down to the OpenTelemetry section and change the following settings:
   - OTLP Endpoint should be `https://otlp.nr-data.net:4317`.
     ![OTLP endpoint entered in text field](https://newrelic.com/sites/default/files/2022-10/jenkins-otel-6.png)
   - **Authentication** should be **Header Authentication**.
   - The **Header Name** should be `api-key`.
   - For the **Header Value**, select **Add** > **Jenkins**. You’ll need to create a secret text with the following fields:
     - For **Kind**, choose **Secret text**.
     - The **Secret** should be your [New Relic license ingest key](https://docs.newrelic.com/docs/apis/intro-apis/new-relic-api-keys/#overview-keys?utm_source=devto&utm_medium=community&utm_campaign=global-fy23-q4-monitoring_jenkins_pipelines). 
     - ID can be left blank.
     - Add a description such as `New Relic INGEST - LICENSE key`. 
     - Select **Add**.
   ![Jenkins Credentials Provider Add Credentials screen.](https://newrelic.com/sites/default/files/2022-10/jenkins-otel-7.png)

10. After you’ve added your credentials, you’ll be able to select **New Relic INGEST - LICENSE Key** from the Header Value dropdown menu.

11. Select **Save**.
   ![New Relic INGEST - LICENSE Key selected from Header Value drop down in OpenTelemetry menu.](https://newrelic.com/sites/default/files/2022-10/jenkins-otel-8.png)


## Sending Jenkins data to New Relic
After you’ve set up your New Relic credentials in Jenkins, you’re ready to send data to New Relic. Let’s start by setting up a test pipeline that will automatically fail. By testing a pipeline, you’re making sure that the connection to New Relic is set up properly. You can set up a test pipeline with the following steps.

1. In the Jenkins dashboard, select **New Item**.
   ![New Item selected in Jenkins dashboard.](https://newrelic.com/sites/default/files/2022-10/jenkins-otel-9.png)

2. Enter an item name (the name in this example is test) and select **Pipeline**. Then select **OK**.
   ![Pipeline selected from Enter an item name list.](https://newrelic.com/sites/default/files/2022-10/jenkins-otel-10.png)

3. Scroll to the bottom, select **Scripted Pipeline**, and then select **Save**.
   ![Pipeline script selected.](https://newrelic.com/sites/default/files/2022-10/jenkins-otel-11.png)

Some older versions of Jenkins don’t have sample pipelines that you can choose from the list. If this is the case for you, paste in the following script and select **Save**.

```
pipeline {
  agent any
  stages {
    stage('Build') {
      steps {
        git 'https://github.com/jglick/simple-maven-project-with-tests.git'
        sh "mvn -Dmaven.test.failure.ignore=true clean package"
      }
      post {
        success {
          echo "success"
        }
      }
    }
  }
}
```

4. Select **Build Now**. This sample pipeline should fail, which is the expected behavior.
   ![Pipeline test with Build Now option.](https://newrelic.com/sites/default/files/2022-10/jenkins-otel-12.png)

5. Log into New Relic, go to **Services - OpenTelemetry**, and select **jenkins**.
   ![Select jenkins from entity list in New Relic.](https://newrelic.com/sites/default/files/2022-10/jenkins-otel-13.png)

Now you can go to **Transactions** to see your data. You should see all of the pipeline build steps, including both successes and failures. The next image shows a Jenkins dashboard in New Relic. It includes information on top transactions, top 20 transactions, and throughput.

![New Relic dashboard shows Jenkins metrics such as top transactions.](https://newrelic.com/sites/default/files/2022-10/jenkins-otel-14.png)

You can also go to **Distributed Traces** to see data from your Jenkins pipeline.  In this example, you can see exactly where the pipeline starts failing, which is helpful for creating alerts. The next image shows a New Relic dashboard with specific spans in a Jenkins pipeline distributed trace.

![New Relic dashboard shows distributed trace of Jenkins pipeline.](https://newrelic.com/sites/default/files/2022-10/jenkins-otel-15.png)

If you’ve set the environment variable to `OTEL_LOGS_EXPORTER="otlp"`, you can see your Jenkins console **logs** in the Logs tab. The next image shows logs related to a Jenkins pipeline.

![Jenkins pipeline logs in New Relic.](https://newrelic.com/sites/default/files/2022-10/jenkins-otel-16.png)

## Visualize your Jenkins data
Once you have Jenkins data flowing into New Relic, you can add our quickstart dashboard to get an overview of your Jenkins environment. For example, you can see a summary of the number of Jenkins instances available, the number of successful jobs, the top errors, and the job queue status. To get this dashboard, simply follow these steps:

1. Go to the [Jenkins quickstart](https://newrelic.com/instant-observability/jenkins?utm_source=devto&utm_medium=community&utm_campaign=global-fy23-q4-monitoring_jenkins_pipelines) in New Relic Instant Observability, and select **Install now**.

2. Select an account and select **Begin installation**.

3. If you've already completed the earlier steps in this tutorial, select **Done** to move on to the next step.

4. The quickstart deploys the resources to your account. Select **See your data** to get to the dashboard.

<figure>
  ![New Relic dashboard shows Jenkins metrics such as length of time for jobs, error rate, and whether they succeeded or failed.](https://newrelic.com/sites/default/files/2022-10/jenkins-quickstart-dashboard.png)
  <figcaption>Install the Jenkins quickstart dashboard for a pre-built view of your key pipeline metrics.</figcaption>
</figure>

After you're done, you'll have complete visibility into your Jenkins pipelines, from high-level metrics to logs and distributed traces. You can also optionally choose to add alerts when certain thresholds are met, such as when jobs are taking too long or when the error rate gets too high. To do so, you can follow along with the docs to first [create a policy](https://docs.newrelic.com/docs/alerts-applied-intelligence/new-relic-alerts/alert-policies/create-edit-or-find-alert-policy?utm_source=devto&utm_medium=community&utm_campaign=global-fy23-q4-monitoring_jenkins_pipelines) and then [set conditions](https://docs.newrelic.com/docs/alerts-applied-intelligence/new-relic-alerts/alert-conditions/create-alert-conditions?utm_source=devto&utm_medium=community&utm_campaign=global-fy23-q4-monitoring_jenkins_pipelines) on that policy. You can [send alerts](https://docs.newrelic.com/docs/alerts-applied-intelligence/new-relic-alerts/alert-notifications/notification-channels-control-where-send-alerts?utm_source=devto&utm_medium=community&utm_campaign=global-fy23-q4-monitoring_jenkins_pipelines) to the notification channels of your choice. Options include Slack, PagerDuty, email, and more.

Sending your Jenkins pipeline data to New Relic gives you observability into the building and deployment phases of the DevOps lifecycle, giving you more insights into your DevOps practices and helping you shift left.

You can read the full blog post at New Relic [here](https://newrelic.com/blog/how-to-relic/monitoring-jenkins-pipelines?utm_source=devto&utm_medium=community&utm_campaign=global-fy23-q4-monitoring_jenkins_pipelines).