Running Postman Collections in Azure Pipelines
Tags: azure, postman, api, testing
Published: 2022-06-18 16:09:54

## Introduction

Running a set of Postman collections within a dedicated Azure Pipeline provides the ability to automate API testing of Development, Testing and even Staging environments with minimal manual intervention. Furthermore, through the use of regular runs, it encourages the adoption of the `shift-left` approach:

> test early and often

Hence, teams can detect regressions and bugs early in the development lifecycle.

### Create Postman Collection
The first step is to gather your Postman API requests under the same Collection. In this example, the collection is named **Post-Office**. Under the **Tests** tab, there is a simple assertion which verifies the expected status code returned after the API call.

*Note: The endpoint *-foo.com is only an illustrative example and you should replace it with your own in your current context.*


![Sample Postman Collection](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/62qw2x49guv7gd11dckq.png)

### Create environments + variables
Ensure that you have created corresponding **Postman Environments** for the different contexts in which your code is being deployed to. *Environments* provide a way to organize variables which are only relevant for a specific context (API keys, external web-services, etc...).

![Postman Environments](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rye523j0b0vt9vr8w7zi.png)

#### Export the collection
You can export the relevant Postman Collection by hovering over it on the sidebar and clicking on the "..." (View more actions). Finally click on **Export**


![Export Postman Collection](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/s6i8o6yut2s6dmuysmpx.png)

A file with a default filename of `*.postman_collection.json` (in this case Post-Office.postman_collection.json) will be downloaded

#### Export Postman Environment Variables
Go to the **Environments** tab on the side bar, then click on the "..." and finally select **export**.

![Export Postman Environment Variables](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6gytbd0nse3bfy9v4n66.png)


### Create a new Pipeline

Once you have exported the collection and environment json files, ensure that you push them to the repository of your web app on Azure DevOps.

Now it is time for some plumbing! The task is to create a new pipeline that will execute your Postman collection scripts automatically.

Go to **Pipelines** and click on **New**. Select the classic editor.


![Pipeline Classic Editor](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8o0gbvwf8tamgwhkumim.png)

### Add Tasks

### Task 1: Install newman package
For the first job, we will be using the newman package from npm to execute our postman collection.

> Newman is a command-line Collection Runner for Postman. It enables you to run and test a Postman Collection directly from the command line. 
>
> -- <cite>https://learning.postman.com/docs/running-collections/using-newman-cli/command-line-integration-with-newman/</cite>


![Install newman package](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vunf0ckqfxucx5lacryt.png)

### Task 2: Execute Postman Collection
To execute the postman collection with newman, we will need to provide the following command:
``` newman run sample-collection.json -e env.postman_environment.json```

The first argument of the `newman run` is the file path of your postman collection. The option `-e` take as input the file path of the Postman Environment file.

![Execute Collection Runner](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/x6tu0td8avq5mwod5hgi.png)

#### Making use of Pipeline variables
In the previous screenshot, we can see that we have hardcoded the environment filename for the option `-e`:
```
newman run sample-collection.json -e env.postman_environment.json
```
Instead of specifying directly the environment filename within the command, we can leverage the use of Pipeline variables.
We create a variable `CUSTOM.ENVIRONMENT_FILE` to store the filename.

![Pipeline Variables](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/d8sibu27p3q2dz1iyn7j.png)

We can then reference the variable inside the command as follows:

![Reference Pipeline Variable](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wqu2v53byw0eystrgnan.png)

```
# Navigate to postman directory
DESTINATION='$(System.DefaultWorkingDirectory)/postman'
cd $DESTINATION

# Absolute path for environment file
ENVIRONMENT_FILE='$(System.DefaultWorkingDirectory)/postman/environments/$(CUSTOM.ENVIRONMENT_FILE)'

newman run collections/Post-Office.postman_collection.json -e $ENVIRONMENT_FILE

```


After the execution of the Postman Collection, the results of all tests and requests can be exported into a custom file.
This is made possible through the use of reporters. In our case, we will use a `JUNIT` reporter to export the test results to a custom file.
```
newman run collections/Post-Office.postman_collection.json  -e $ENVIRONMENT_FILE --reporters cli,junit --reporter-junit-export test-results/outputfile.xml
```
`outputfile.xml` represents the custom file where you want to collect the test results.

View more about custom reporters here: [Newman Custom Reporters](https://learning.postman.com/docs/running-collections/using-newman-cli/newman-custom-reporters/)

### Task 3: Publish Test Results

The final step is to publish the test results and view the results on the **Tests** tab of the pipeline.

Add the *Publish Test Results* task.

![Publish Test Results](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/cdqgd31wazhirs31f97g.png)

View the test results on the **Tests** tab

![Test Results Tab](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8q2chgdrn3y63k0fovks.png)

That's its folks! Thank you 😃

## References
- https://learning.postman.com/docs/running-collections/using-newman-cli/command-line-integration-with-newman/
- https://docs.microsoft.com/en-us/azure/devops/pipelines/get-started/what-is-azure-pipelines?view=azure-devops