Automate your API tests with Jenkins, Postman and Newman
Tags: jenkins, testing, postman, qa
Published: 2019-01-02 21:28:14

---
title: Automate your API tests with Jenkins, Postman and Newman
published: true
description: Build up a Jenkins task to run your Postman collection against your API inside your CI pipeline.
tags: jenkins,testing,postman,qa
---

## A little background
Nowadays CI pipelines are a must on our development dayjob, helping us to shorten delivery cycles and to bring higher quality software.

As a backend developer, APIs have became a cornestone on most of my applications, so testing them became a must.

Having an API Test Stage in our CI pipeline brings us confidence in the quality of the product and alerts us if something had broken.


## Postman
[Postman](https://www.getpostman.com/) is a great tool that helps developers in every stage of our API lifecycle. It has a great [documentation](https://learning.getpostman.com/) and an increasingly community of developers which use it, so it is easy to find posts explaining cool tips and tricks.

### Collections
Postman makes easy to organize our different request into [collections](https://learning.getpostman.com/docs/postman/collections/intro_to_collections). Personally I create a collection for every project I want to test. Then, inside the collection, I create a folder for every entity on the project, trying to cover every endpoint in the API.

<figure>
  <img src="https://thepracticaldev.s3.amazonaws.com/i/1v967smnl4xen35arw2k.png" alt="Requests organized by collections"/>
  <figcaption>Requests are organized by entities</figcaption>
</figure>

### Environments
Postman allows us to create different [environments](https://learning.getpostman.com/docs/postman/environments_and_globals/intro_to_environments_and_globals) to run our requests. Giving a real example, I would tell you I work on three different environments: my local environment, a staging environment and a production environment, so I  create three different environments with variables which represents the changing data between the environments (like the base path for the API requests). This way I can define a single request but target it to my three different environments easily. Think about how easy would be to change or add a new environment!

<figure>
  <img src="https://thepracticaldev.s3.amazonaws.com/i/x0fxp2f4xj7in72uw7ge.png" alt="Using environment variables on request url"/>
  <figcaption>Extracting base path as environment variable to target different environments</figcaption>
</figure>

### Testing requests
One of the greatest features of Postman is that allows you to code [tests](https://learning.getpostman.com/docs/postman/scripts/test_scripts) that runs every time a request run. They are written in JS and have a cool set of assertions to check things like the response code, response body o response headers.

<figure>
  <img src="https://thepracticaldev.s3.amazonaws.com/i/r3zotdaf4r0x96tjwyu1.png" alt="A request with few tests"/>
  <figcaption>A request with few tests</figcaption>
</figure>

*Sweet!* Once we have written a test suite to cover our API, we use the [Collection Runner](https://learning.getpostman.com/docs/postman/collection_runs/starting_a_collection_run/), a built-in tool provided by Postman to run every request of a collection and output a report of tests results.

<figure>
  <img src="https://thepracticaldev.s3.amazonaws.com/i/dljz69xvs7tm28mdt34g.png" alt="Collection runner test report"/>
  <figcaption>How beatiful are the green meadows...</figcaption>
</figure>

*Bonus: Sometimes, we want to run our requests in a certain order and guess what? Postman allows us to do it! Check it at [building workflows issue](https://learning.getpostman.com/docs/postman/collection_runs/building_workflows/)*

### What do we have so far?
At this point, we have build up a Postman collection with a bunch of request that could target our API in all our environments. Besides that, every request has several tests to check if it is working properly. *What a time to be alive!*

##Newman
Do you remember the Collection Runner I mentioned previously? It could run every request in our collection with a simple click. But what about a command line tool to do the same? [Newman](https://learning.getpostman.com/docs/postman/collection_runs/command_line_integration_with_newman/) does it. Newman is a command line Collection Runner for Postman that allows you to run and test a Postman Collection directly from the command line and integrate it easily in CI servers, that makes it a great tool for our purpose: **Automate our API test with Jenkins**.

### Run a collection
In order to run a collection with Newman, we first have to export it from Postman. Since we have some variables playing around in some of our request, we have to export the suitable environment too. Once we have both files, we are able to run the collection with:
```bash
$ newman run our.postman_collection.json -e our.postman_environment.json
```
and we will see a report of the collection run, similar to others testing frameworks.

<figure>
  <img src="https://thepracticaldev.s3.amazonaws.com/i/bhfe61yojs428khstmqj.PNG" alt="Newman CLI test report"/>
  <figcaption>Newman CLI test report</figcaption>
</figure>

### Formatting the output
The report can be formatted in different ways. To achieve this, Newman provides a set of **reporters** to support different output formats like JSON, XUNIT or HTML. Besides that you can create your own reporter to format the report in any format you want.

## Jenkins

Cool, we have defined our *API test suite* with Postman and we are able to run it from command line and obtain a nice report of the results so far with Newman. But now we want to include our awesome new skill in our CI pipeline.

*Disclaimer: We are using Jenkins as an automation server to build up our CI/CD pipelines and we run our CI pipeline in a Docker container which have all the necessary tools installed (such Newman).*

### Mixing things up

Essentially we will need three files to include our API Test inside a CI pipeline:
- _Jenkinsfile_, which defines our pipeline
- _Postman Collection_, which defines our set of request and tests
- _Postman Environment_, which defines the variables of the target environment

We include all of them into project repository, so we can benefit of version control over them and track their changes.

**Enough talking, let’s code!**

We need to create a new stage in our pipeline to run the API Test. We usually put it after unit and integration tests to be sure everything is running as expected before testing the interface.

```js
pipeline {
    ...
    stage('Test API Rest') {
        steps {
            sh 'newman run tests/Newman/our.postman_collection.json -e tests/Newman/env/test.postman_environment.json -r junit,html --reporter-junit-export var/reports/newman/junit/newman.xml --reporter-html-export var/reports/newman/html/index.html'

            publishHTML([allowMissing: false, alwaysLinkToLastBuild: false, keepAll: false, reportDir: 'var/reports/newman/html', reportFiles: 'index.html', reportName: 'Newman API Test', reportTitles: ''])
        }
    }
    ...
}
```

As you can see, we invoke the run command of Newman, providing our collection an the test environment file as parameters. Besides that, we also inform which reporters we want to use (JUNIT, HTML) and where the outputs will be stored.

As a bonus, we use [HTML PUblisher Jenkins plugin](https://wiki.jenkins.io/display/JENKINS/HTML+Publisher+Plugin) to link the HTML report to the build.

I hope you enjoyed this journey as much as I did writing it and, as always, feedback will be welcomed!