_edited_edited.png)
Let us first try to understand that what is BDD Cucumber. Cucumber is a testing framework that supports Behavior Driven Development (BDD). It is written in plain English text called Gherkin. It is defined as a scenario of inputs, actions and outcomes. Gherkin interprets human input into the software concept of input/process and actions.
Main Elements in Cucumber are
· Feature File- Basically the user stories are written in the feature file in a predefined format using GIVEN, WHEN, THEN keywords. Other keyword which are used to support this framework are FEATURE, SCENARIO/SCENARIO OUTLINE, BACKGROUND. It is written in plain English.
· Step Definition: This is the part where the actual Automation test script is written.
· Runner class:- This class will use the Junit annotation @RunWith(), which tells JUnit what is the test runner class. It more like a starting point for Junit to start executing your tests.
Runner class is used to run the feature file & generate reports.
So now, we are going to discuss more elaborately about reports, Parallel Execution & rerun of the failed TestCases through the runner class in cucumber.
REPORTS
The output of the any test execution, has to be in format, which immediately depicts the overall results of the execution. Hence, our framework also should have the same capability to create output or generate test execution reports.
What are the various kinds of reports we can generate using Cucumber.
1. JSON / XML/ HTML reports:
When we execute Cucumber Scenarios, it automatically generates an output in the eclipse console. There is a default behavior associated with that output and we can also configure that output as per our needs also. For that we need to have pretty Plugin.
To make the reports to be in more readable format monochrome is set to true.
package Test_Runner;
import org.junit.runner.RunWith;
import io.cucumber.junit.Cucumber;
import io.cucumber.junit.CucumberOptions;
@RunWith(Cucumber.class)
@CucumberOptions(features = {"src/test/resources/test.feature"},
glue = { "step_definition" },
plugin = { “pretty”,"html:target/test-reports/test_htmlreports.html",
"json:target/test-reports/test_JSONReports.json",
"junit:target/test-reports/test_XMLReports.xml"},
monochrome = true)
public class Test_Runner {
}These reports are stored in the Target folder in the Eclipse
2. Cucumber output Reports :
Cucumber itself has provided a nice feature to generate reports. These are very basic reports.
We can create this report by creating a cucumber.properties file under src/test/resources folder. The property File should contain the following
cucumber.publish.enabled = trueThe report link comes in the eclipse console after the execution.
3. Extend Report/Spark Reports
This is the advanced and one of the best reporting. It generates a PDF report & spark html report.
For this we need to add dependency in the pom.xml file. If you are using Java 8+ then add the following dependency
<dependency>
<groupId>tech.grasshopper</groupId>
<artifactId>extentreports-cucumber6-adapter</artifactId>
<version>2.9.0</version>
</dependency>Next Add Extent.properties file under src/test/resources with the following content.
extent.reporter.spark.start=true
extent.reporter.spark.out=test-output/SparkReport/Spark.html
extent.reporter.spark.config=src/test/resources/extent-config.xml
extent.reporter.spark.out=test-output/SparkReport
screenshot.dir=test-output/Screenshots
screenshot.rel.path=../
extent.reporter.pdf.start=true
extent.reporter.pdf.out=test-output/PDFReport/ExtentPDF.pdf
extent.reporter.spark.vieworder=dashboard,test,category,exception,author,device
systeminfo.os=Window
systeminfo.appname=Test_APPThen Add Extent Config to the Project under src/test/resources as extent-config.xml with the following content inside
<?xml version="1.0" encoding="UTF-8"?>
<extentreports>
<configuration>
<!-- report theme --> <!-- standard, dark -->
<theme>standard</theme>
<!-- document encoding --> <!-- defaults to UTF-8 -->
<encoding>UTF-8</encoding>
<!-- protocol for script and stylesheets --> <!-- defaults to https -->
<protocol>https</protocol>
<!-- title of the document -->
<documentTitle>TestAutomation - Cucumber Framework</documentTitle>
<!-- report name - displayed at top-nav -->
<report Name> TestAutomation - Cucumber Report</reportName>
<!-- global date format override --> <!-- defaults to yyyy-MM-dd -->
<dateFormat>yyyy-MM-dd</dateFormat>
<!-- global time format override --> <!-- defaults to HH:mm:ss -->
<timeFormat>HH:mm:ss</timeFormat>
<!-- custom javascript -->
<scripts>
<![CDATA[
$(document).ready(function() {
});
]]>
</scripts>
<!-- custom styles -->
<styles>
<![CDATA[
]]>
</styles>
</configuration>
</extentreports>Now the Last step for this reports is to add the plugin in the Runner Class with the following lines of code.
@RunWith(Cucumber.class)
@CucumberOptions(features = {"src/test/resources/test.feature"},
glue = { "step_definition" },
plugin={"pretty”,"com.aventstack.extentreports.cucumber.adapter.ExtentCucumberAdapter:"},
monochrome = true)
public class test_Runner {
}The output report Looks like the following
4. Allure Reports
Allure Framework is a flexible lightweight multi-language test report tool.
For Allure Reporting we need to add the following dependencies in the POM.xml
<!-- https://mvnrepository.com/artifact/io.qameta.allure/allure-testng -->
<dependency>
<groupId>io.qameta.allure</groupId>
<artifactId>allure-testng</artifactId>
<version>2.17.2</version>
</dependency>
<!-- https://mvnrepository.com/artifact/io.qameta.allure/allure-cucumber6-jvm -->
<dependency>
<groupId>io.qameta.allure</groupId>
<artifactId>allure-cucumber6-jvm</artifactId>
<version>2.17.2</version>
</dependency>
And also the following plugin in the POM.xml
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M5</version>
<configuration>
<argLine>- javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/1.9.7/aspectjweaver-1.9.7.jar"
</argLine>
</configuration>
<dependencies>
<dependency>
<groupId>org.aspectj</groupId>
<artifactId>aspectjweaver</artifactId>
<version>1.9.7</version>
</dependency>
</dependencies>
</plugin>
</plugins>
</build>Then in the runner file add the following
@RunWith(Cucumber.class)
@CucumberOptions(features = {"src/test/resources/test.feature"},
glue = { "step_definition" },
plugin = { "pretty","io.qameta.allure.cucumber6jvm.AllureCucumber6Jvm"
},
monochrome = true)
public class test_Runner {
}Now to view the allure report need to do the following steps:
Download the latest version as zip archive from either Maven Central or GitHub releases.
Unpack the archive to allure-commandline directory.
Navigate to bin directory.
Use allure.bat for Windows and allure for other Unix platforms.
Add allure to system PATH.
The easiest way is to, install nodejs: Download
now install allure-commandline tool using below command:
npm install -g allure-commandlinenow just run the below command:
allure generate --clean && allure openOr u can also execute from command line by going in the directory where allure-results directory is present. Then in the command line write allure serve
The Final Allure Reports Looks like
PARALLEL EXECUTION
One of the most important advantage of test Automation is to reduce the execution time. This can be achieved by executing the test Suites in parallel. Cucumber can be executed in parallel using JUnit and Maven test execution plugins.
In JUnit, the feature files are run in parallel rather than scenarios, which means all the scenarios in a feature file will be executed by the same thread. You can use either Maven Surefire or Failsafe plugin to execute the runner
The steps involved to do parallel execution is as below:
Add Maven Failsafe plugin in the build section of POM.XML for the parallel execution
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>3.0.0-M5</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
<configuration>
<includes>
<include>**/test_Runner.java</include>
</includes>
<parallel>methods</parallel>
<threadCount>4</threadCount>
<perCoreThreadCount>true</perCoreThreadCount>
</configuration>
</execution>
</executions>
</plugin>Here Few Point do keep in mind
1. Always provide the relative path of the runner file under <include > Tab
2. Depending upon the no of parallel execution required, set the number in <threadCount> tab to required specific number instead of useUnlimitedThreads
3. In case of multiple runners one can also set the parallel option to classesAndMethods or classes in addition to methods.
<configuration>
<parallel>classesAndMethods</parallel>
<useUnlimitedThreads>true</useUnlimitedThreads>
</configuration>Add Timeline Formatter
For a visual representation of threads, add the timeline report using the plugin option of CucumberOptions annotation on a JUnit or TestNG runner.
@CucumberOptions(plugin= {"timeline:<report folder>"})Below is a sample report of timeline Formatter.
Last step to execute in parallel. That can be achieved through the commandline. You need to be in the project directory where the runner file is. Then just use mvn verify to execute the features in parallel
Once mvn verify is done , we can see the threads in the command line too.
RERUN OF THE FAILED TESTCASES
Let us first try to understand why rerun of the failed test cases are required & how important it is. Sometime we experience issues with inconsistent test results due to unstable environment. Some test would fail for no obvious reason and pass when you rerun. So if the rerun of failed testcases is automated then we have to put less effort in analyzing the failed testcases report which fails due to unstable environment. By using the rerun plugin we can achieve this in a very simple step but it reduces our effort to a huge extent.
Modify existing runner class.
Add rerun:target/rerun.txt to your plugin, so your runner class will look something like that. So the test_runner will look something like below:
@RunWith(Cucumber.class)
@CucumberOptions(feature={“classpath:feature”}, glue{“path to stepdefs”}, plugin = {“rerun:target/rerun.txt” })This will run all your tests and then list all failed scenarios in rerun.txt file under the target folder. The rerun.txt contains path to the feature file and line number for scenario that was failed.
Create another runner class.
Next step is to create a new runner class to run failed scenarios. The New runner will contain the below codes
@RunWith(Cucumber.class)
@CucumberOptions(features = “@target/rerun.txt”, plugin = {“rerun:target/rerun.txt”})
public class FailedRunner {}Add maven surefire plugin to your pom.xml
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
<configuration>
<includes>
<include>**/*Runner.java</include>
</includes>
</configuration>
</plugin>Running Tests
In order to use rerun option you need to execute maven goals from terminal, open the terminal and go to the folder where your pom.xml is located and then run “mvn clean test”
Comments