Generating Newman Reports for Postman in CLI and Jenkins

In this blog, we are going learn how to run a postman collection and generate reports from Command Line Interface (CLI) and Jenkins using Newman.

Postman is a standalone software testing API (Application Programming Interface) platform to build, test, design, modify, and document APIs. It is a simple Graphic User Interface for sending and viewing HTTP requests and responses.

You can add new collections, update existing collections, update environments, and add and run monitors directly through the API. This enables you to programmatically access data stored in your Postman account.

A command-line interface is a means of interacting with a computer program (or software) by typing line by line commands in your shell (command prompt or terminal).

Newman is a command-line Collection Runner for Postman. It enables you to run and test a Postman Collection directly from the command line. It's built with extensibility in mind so that you can integrate it with your continuous integration (CI) servers and build systems.

Let us see the steps to achieve Newman Reporting for Postman

1. Install Node

2. Install Newman

3. Set System Variables

4. Run Newman

5. Install and generate Html Extra Report

6. Postman Newman integration with Jenkins

7. Generate HTML Extra report in Jenkins.

Step 1: Install Node

· Download the Installer. Download the Windows Installer from NodeJs official website.

· Install Node.js and NPM. After choosing the path, double-click to install .msi binary files to initiate the installation process.

· Check Node.js and NPM Version. Ensure you are using Node.js v4 or above. Your node.js is not installed if you do not get node.js version

Step 2: Install Newman

Install Newman from npm globally on your system, which allows you to run it from anywhere:

Step 3: Set System Variables: Add the Node executable to your system path. Go to the Control Panel > System and Security > System > Advanced System Settings > Environment variables

Step 4: Run Newman

We have completed the reporters installation, but we still we have to execute and translate to our HTML reports.

We can run newman collection in CLI by two methods:

1) Using the collection url from postman

2) Using the collection url from local desktop

Method 1: Using API Key from the postman collection

Select the collection for which you wish to generate the Newman report and click share button

Select Via API option and copy the Collection Access Key

Run the following Newman command in CLI with the appropriate parameters:

Go to the newman path and enter

“newman run accesskey”

Method 2: Using json file location from local desktop

Export the collection from Postman by just selecting Export in the collection folder and then saving them as a *json file. We can run the collection in CLI by mentioning the location of the collection, instead of API access key as below:

“newman run C:\***.json”

Step 5: Install and generate htmlextra report

HTML Extra is an awesome Postman/Newman reporting dashboard which enables us to display postman test results in a more interactive manner.

Install Newman HTML reporter using the command:

“npm -g newman-reporter-htmlextra”

Generate HTML Extra report for the exported collection using the command:

“newman run xxx.json -r htmlextra”

The HTMLExtra report will be saved inside the folder where Newman was executed, inside a folder called Newman, as below

Awesome! The Detailed Newman HTML extra report is generated !

In the summary page, we get a summary of the test results we have run. We get detailed information like number of iterations, assertions, skipped tests, failed tests. From the Total Requests tab, we can get lot of information like baseurl, the request body, response header, and response format. For more details, we can to go to the Failed tests tab to check which scripts failed, status code and assertion error message.

Step 6: Postman Newman integration with Jenkins

Jenkins is an open-source automation server that can act as a continuous integration (CI) server or a continuous delivery (CD) hub. With hundreds of available plug-ins, teams can use Jenkins to build, deploy, and automate almost any project.

Now, we will see the steps to generate the Newman Report in Jenkins.

To Configure Jenkins Job to Run Batch Command:

Go to localhost:8080-Dashboard-New item-Enter item Name-Select Freestyle Project-Click Ok

Install supported plugins like NodeJS Plugin and Test Complete Plugin from manage Jenkins- plugin manager.

To Configure Postman Collection on Jenkins:

In the configuration screen, Select General-Build Steps-Add Build Step-Execute Windows Batch Command and enter the below command and save.

“C:/**/newman run --disable-unicode C:\\**\\AllPrograms_postman_collection.json”

Note:- If you use back slash use one (/) , forward slash use two (\\) inside windows batch command.

We have to run postman collection on Jenkins by selecting ‘Build Now’

Now, we will Generate Newman Reports on Jenkins.

Let us configure and publish Jenkins HTML Reports for Newman.

The Html report for the same collection can be generated by changing the Windows batch command as below:

“C:/**/newman run C:\\Users**\\AllPrograms_postman_collection.json -r htmlextra”

Select “Build Now” for the postman collection in Jenkins and view the console output

The Html Extra report can be accessed from the location mentioned in the console output. It is highlighted in above screen shot, for reference.

So, we have reached the end of the blog. I hope you would have got a good understanding about generating Newman reports, HTML Extra reports, Running Postman Collection from CLI and executing the same in Jenkins.