Re-Run Failed TestCases Using TestNG iRetryAnalyzer

In Automation Testing, sometimes the test cases fail due to random problems like server issues, browser issues, element not found, network glitch, and so on. Some tests are dependent on each other and when a test fails maybe we want to run the tests again in order to confirm whether it really fails because of the product bug.

There are three ways to re-run failed test cases:

1.testng-failed.xml

2.IRetryAnalyzer Interface

3.IAnnotationTransformer (I) and IRetryAnalyzer (I)

Let’s see everything in detail.

1. Using testng-failed.xml

Here I have 2 test cases, 1 test case fails and the other one that passes.

Create a testing.xml file by Right-clicking on project=> TestNG=> Convert to TestNG

Next, run the testng.xml file by Right Clicking => Run As=>TestNG Suite

Now the Console output will show as 1 test passed and 1 test failed.

Failed test cases will be stored in the testng-failed.xml file in the test-output folder. We can re-run the failed test cases alone manually in this file. Below is the screenshot of the failed test case in the testngfailed.xml file.

2. Using IRetryAnalyzer Interface.

Create a separate Class that implements this IRetryAnalyzer like below

Here limit count can be increased or decreased to re-run the test cases. In the above example, failed test cases will run 3 times till it passes. In case it fails the third time, test execution will stop and TestNG will mark this case as failed.

Next, implement retryAnalyzer attribute using @Test annotation for the failed test cases.

@Test(retryAnalyzer = RetryListener.class)

public void testCase() {

}

In the above scenario, failed test re-runs 3 times and it fails during each execution and is marked as test failed. However, passedTest executed and passed at the first execution as shown below.

3. Using the IAnnotationTransformer interface.

In order to avoid implementing retryAnalyzer attribute every time @Test level for the failed test cases, we create an implemented class of the IAnnotationTransformer interface and override the transform method as below.

Now add the Listener to testng.xml file. Below is the syntax to add a listener for RetryListener.

<listeners>

<listener class-name="iRetryAnalyzer.Transformer""/>

</listeners>

After executing the program, The Test Output is shown below.

Conclusion

If you work on a test automation project, you’d know that the most difficult part of automation is the analysis of test executions. At the end of the execution, With the help of iRetryAnalyzer, you can analyze failed test cases and try to figure out if there’s any false positive/flaky situation caused by network glitch, time-out, or some other problem.