How-To: Setup a unit-testable Jenkins shared pipeline library
Tags: jenkins, tutorial, testing, devops
Published: 2019-01-05 15:17:58

---
title: How-To: Setup a unit-testable Jenkins shared pipeline library
published: true
description: Tutorial on how to create a unit-testable Jenkins shared pipeline library
tags: #jenkins #tutorial #testing #devops
---

(Originally posted on [loglevel-blog.com](https://loglevel-blog.com/how-to-setup-and-develop-jenkins-shared-pipeline-library/))

In this blog post I will try to explain how to setup and develop a shared pipeline library for Jenkins, that is easy to work on and can be unit tested with JUnit and Mockito.

**NOTE:** This blog post is kinda long and touches many topics without explaining them in full detail. If you don't feel like following along a lengthy tutorial you can have a look at the complete example library on [GitHub](https://github.com/kuper-adrian/jenkins-shared-library-example). Also if you have questions, or really any kind of feedback on what I could improve, please leave a comment and I will come back to you asap ;) Additionally, if you are completely unfamiliar with Jenkins Shared Libraries, you should probably read about them first in the [official docs](https://jenkins.io/doc/book/pipeline/shared-libraries/). 

Let's get going!

## Basic Development Setup

First, let's create a new IntelliJ IDEA project. I suggest using the IntelliJ IDEA for Jenkins shared pipeline development, because it is the only IDE I know of, that properly supports Java and Groovy and has Gradle support. So, if you don't have it installed it yet, you can download it [here](https://www.jetbrains.com/idea/download/#section=mac) for Windows, Linux or MacOS. Also make sure to have the Java Development Kit installed, which is available [here](https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html).

When everything is ready, start IntelliJ, create a new project, select Gradle and make sure to set the checkbox on Groovy.

![project-creation-1](https://thepracticaldev.s3.amazonaws.com/i/mvf2arsz0dmpi721uc6l.png)

Next up, enter a GroupId and an ArtifactId.

![project-creation-2](https://thepracticaldev.s3.amazonaws.com/i/6jgw5g2khn8nt9rrc0tp.png)

Ignore the next window (the defaults are fine), click "Next", enter a project name and click "Finish".

![project-creation-3](https://thepracticaldev.s3.amazonaws.com/i/bw2p2kfvz6v3ygq5hgs7.png)

IntelliJ should boot up with your new project. The folder structure in your project should be something like the following.

![default-project-structure](https://thepracticaldev.s3.amazonaws.com/i/jj5vrid6mhbqb0ffyhkm.png)

This is cool for usual Java/Groovy projects, but for our purpose we have to change things up a bit since Jenkins demands a project structure like this:
```bash
(root)
+- src                     # Groovy source files
|   +- org
|       +- somecompany
|           +- Bar.groovy  # for org.foo.Bar class
+- vars
|   +- foo.groovy          # for global 'foo' variable
|   +- foo.txt             # help for 'foo' variable
+- resources               # resource files (external libraries only)
|   +- org
|       +- somecompany
|           +- bar.json    # static helper data for org.foo.Bar
```
So:

1. add a `vars` folder to your project root folder
2. add a `resource` folder to your project root folder
3. delete all files/folders inside `src` and add a new package like `org.somecompany`
4. edit the `build.gradle` file:

```javascript
group 'somecompany'
version '1.0-SNAPSHOT'

apply plugin: 'groovy'
apply plugin: 'java'

sourceCompatibility = 1.8

repositories {
    mavenCentral()
}

dependencies {
    compile 'org.codehaus.groovy:groovy-all:2.3.11'
    testCompile group: 'junit', name: 'junit', version: '4.12'
    testCompile "org.mockito:mockito-core:2.+"
}

sourceSets {
    main {
        java {
            srcDirs = []
        }
        groovy {
            // all code files will be in either of the folders
            srcDirs = ['src', 'vars'] 
        }
    }
    test {
        java {
            srcDir 'test'
        }
    }
}
```
After saving, import your changes to the Gradle project:

![import-changes](https://thepracticaldev.s3.amazonaws.com/i/5negvecnc8xuk8ejp8mq.png)

At this point our project has the right structure to be used as a shared library by Jenkins. But, as you might have seen in the code snippet above, we also added a source directory for unit tests called `test`. Now is the time to create this folder at the root level of the project and add a package `org.somecompany` like we did with `src`. The final structure should look like the following.

![final-project-structure](https://thepracticaldev.s3.amazonaws.com/i/rn55h3ybm728ak0kjdr8.png)

Cool, it's time to implement our shared library!

## The General Approach

First a quick run-down on how we build our library and on why we do it that way:

* We will keep the "custom" steps inside `var` as simple as possible and without any real logic. Instead we create classes (inside `src`) that do all the work.
* We create an interface, which declares methods for all required Jenkins steps (`sh`, `bat`, `error`, etc.). The classes call steps only through this interface.
* We write unit tests for your classes like you normally would using JUnit and Mockito.

This way we are able to:
* Compile and execute our library/unit tests without Jenkins
* Test that our classes work as intended
* Test that Jenkins steps are called with the right parameters
* Test the behaviour of our code when a Jenkins step fails
* Build, test, run metrics and deploy your Jenkins Pipeline Library through Jenkins itself

Now let's get really going.

## The Interface For Step Access

First, we will create the interface inside `org.somecompany` that will be used by **all** classes to access the regular Jenkins steps like `sh` or `error`.
```groovy
package org.somecompany

interface IStepExecutor {
    int sh(String command)
    void error(String message)
    // add more methods for respective steps if needed
}
```

This interface is neat, because it can be mocked inside our unit tests. That way our classes become independent to Jenkins itself. For now, let's add an implementation that will be used in our `vars` Groovy scripts:

```groovy
package org.somecompany

class StepExecutor implements IStepExecutor {
    // this will be provided by the vars script and 
    // let's us access Jenkins steps
    private _steps 

    StepExecutor(steps) {
        this._steps = steps
    }
    
    @Override
    int sh(String command) {
        this._steps.sh returnStatus: true, script: "${command}"
    }
    
    @Override
    void error(String message) {
        this._steps.error(message)
    }
}
```

## Adding Basic Dependency Injection

Because we don't want to use the above implementation in our unit tests, we will setup some basic dependency injection in order to swap the above implementation with a mock during unit tests. If you are not familiar with dependency injection, you should probably read up about it, since explaining it here would be out-of-scope, but you might be fine with just copy-pasting the code in this chapter and follow along.

So, first we create the `org.somecompany.ioc` package and add an `IContext` interface:
```groovy
package org.somecompany.ioc

import org.somecompany.IStepExecutor

interface IContext {
    IStepExecutor getStepExecutor()
}
```
Again, this interface will be mocked for our unit tests. But for regular execution of our library we still need an default implementation:
```groovy
package org.somecompany.ioc

import org.somecompany.IStepExecutor
import org.somecompany.StepExecutor

class DefaultContext implements IContext, Serializable {
    // the same as in the StepExecutor class
    private _steps

    DefaultContext(steps) {
        this._steps = steps
    }

    @Override
    IStepExecutor getStepExecutor() {
        return new StepExecutor(this._steps)
    }
}
```
To finish up our basic dependency injection setup, let's add a "context registry" that is used to store the current context (`DefaultContext` during normal execution and a Mockito mock of `IContext` during unit tests):
```groovy
package org.somecompany.ioc

class ContextRegistry implements Serializable {
    private static IContext _context

    static void registerContext(IContext context) {
        _context = context
    }

    static void registerDefaultContext(Object steps) {
        _context = new DefaultContext(steps)
    }

    static IContext getContext() {
        return _context
    }
}
```
That's it! Now we are free to code testable Jenkins steps inside `vars`.

## Coding A Custom Jenkins Step

Let's imagine for our example here, that we want to add a step to our library that  calls the .NET build tool "MSBuild" in order to build .NET projects. To do this we first add a groovy script `ex_msbuild.groovy` to the `vars` folder that is called like our custom step we want to implement. Since our script is called `ex_msbuild.groovy` our step will later be callable with `ex_mbsbuild` in our Jenkinsfile. Add the following content to the script for now:
```groovy
def call(String solutionPath) {
    // TODO
}
```
According to our general idea we want to keep our `ex_msbuild` script as simple as possible and do all the work inside a unit-testable class. So let's create a new class `MsBuild` in a new package `org.somecompany.build`:
```groovy
package org.somecompany.build

import org.somecompany.IStepExecutor
import org.somecompany.ioc.ContextRegistry

class MsBuild implements Serializable {
    private String _solutionPath

    MsBuild(String solutionPath) {
        _solutionPath = solutionPath
    }

    void build() {
        IStepExecutor steps = ContextRegistry.getContext().getStepExecutor()

        int returnStatus = steps.sh("echo \"building ${this._solutionPath}...\"")
        if (returnStatus != 0) {
            steps.error("Some error")
        }
    }
}
```
As you can see, we use both the `sh` and `error` steps in our class, but instead of using them directly, we use the `ContextRegistry` to get an instance of `IStepExecutor` to call Jenkins steps with that. This way, we can swap out the context when we want to unit test the `build()` method later.

Now we can finish our `ex_msbuild` script:
```groovy
import org.somecompany.build.MsBuild
import org.somecompany.ioc.ContextRegistry

def call(String solutionPath) {
    ContextRegistry.registerDefaultContext(this)

    def msbuild = new MsBuild(solutionPath)
    msbuild.build()
}
```
First, we set the context with the context registry. Since we are not in a unit test, we use the default context. The `this` we pass into `registerDefaultContext()` will be saved by the `DefaultContext` inside its private `_steps` variable and is used to access Jenkins steps. After registering the context, we are free to instantiate our `MsBuild` class and call the `build()` method doing all the work. 

Nice, our `vars` script is finished. Now we only have to write some unit tests for our `MsBuild` class.

## Adding Unit Tests

At this point writing unit tests should be business as usual. We create a new test class `MsBuildTest` inside the test folder with package `org.somecompany.build`. Before every test, we use Mockito to mock the `IContext` and `IStepExecutor` interfaces and register the mocked context. Then we can simply create a new `MsBuild` instance in our test and verify the behaviour of our `build()` method. The full test class with two example test:
```groovy
package org.somecompany.build;

import org.somecompany.IStepExecutor;
import org.somecompany.ioc.ContextRegistry;
import org.somecompany.ioc.IContext;
import org.junit.Before;
import org.junit.Test;

import static org.mockito.Mockito.*;

/**
 * Example test class
 */
public class MsBuildTest {
    private IContext _context;
    private IStepExecutor _steps;

    @Before
    public void setup() {
        _context = mock(IContext.class);
        _steps = mock(IStepExecutor.class);

        when(_context.getStepExecutor()).thenReturn(_steps);

        ContextRegistry.registerContext(_context);
    }

    @Test
    public void build_callsShStep() {
        // prepare
        String solutionPath = "some/path/to.sln";
        MsBuild build = new MsBuild(solutionPath);

        // execute
        build.build();

        // verify
        verify(_steps).sh(anyString());
    }

    @Test
    public void build_shStepReturnsStatusNotEqualsZero_callsErrorStep() {
        // prepare
        String solutionPath = "some/path/to.sln";
        MsBuild build = new MsBuild(solutionPath);

        when(_steps.sh(anyString())).thenReturn(-1);

        // execute
        build.build();

        // verify
        verify(_steps).error(anyString());
    }
}
```
You can use the green play buttons on left of the IntelliJ code editor to run the tests, which hopefully turn green.

## Wrapping Things Up

That's basically it. Now it's time to setup your library with Jenkins, create a new job and run a Jenkinsfile to test your new custom `ex_msbuild` step. A simple test Jenkinsfile could look like this:
```javascript
// add the following line and replace necessary values if you are not loading the library implicitly
// @Library('my-library@master') _

pipeline {
    agent any
    stages {
        stage('build') {
            steps {
                ex_msbuild 'some/path/to.sln'
            }
        }
    }
}
```

Obviously there is still a lot more I could have talked about (things like unit tests, dependency injection, Gradle, Jenkins configuration, build and testing the library with Jenkins itself etc.), but I wanted to keep this already very long blog post somewhat concise. I do hope however, that the general idea and approach became clear and helps you in creating a unit-testable shared library, that is more robust and easier to work on than it normally would be.

__One last piece of advice__: The unit tests and Gradle setup are pretty nice and help a ton in easing the development of robust shared pipelines, but unfortunately there is still quite a bit that can go wrong inside your pipelines even though the library tests are green. Things like the following, that mostly happen because of Jenkins' Groovy and sandbox weirdness:

* a class that does not implement `Serializable` which is necessary, because "pipelines must survive Jenkins restarts"
* using classes like `java.io.File` inside your library, which is [prohibited](https://support.cloudbees.com/hc/en-us/articles/230922128-Pipeline-Using-java-io-File-in-a-Pipeline-description)
* Syntax and spelling errors in your Jenkinsfile

Therefore it might be a good idea to have Jenkins instance solely for integration testing, where new and modified `vars` scripts can be tested before going "live". 

Again, feel free to write any kind of questions or feedback in the comments and take a look at the completed, working example library on [GitHub](https://github.com/kuper-adrian/jenkins-shared-library-example).