TestNG vs JUnit

TestNG Vs JUnit – What’s the Difference?

TestNG and JUnit is are open-source testing frameworks for Java. Comparing TestNG Vs JUnit, TestNG annotations are easier to use and understand than JUnit. TestNG allows us to create parallel tests, whereas JUnit does not support running parallel tests.

You can execute your existing JUnit test cases using TestNG. TestNG can automatically recognize and run JUnit tests, so that you can use TestNG as a runner for all your existing tests and write new tests using TestNG.

In this post, we'll discuss and compare these frameworks by covering their features and common use cases.

TestNG

TestNG is an open-source testing framework where NG stands for 'Next Generation'. It is architected to simplify a broad range of testing needs starting from unit testing to integrated system testing

Features of TestNG testing framework:

· Supports annotations.

· TestNG uses more Java and Object Oriented features.

· Separates compile-time test code from run-time configuration/data info.

· Flexible runtime configuration and data driven testing

· It processes Unit tests with passing parameters into the test methods.

· Making use of Threads enhances testing performance better as execution time is reduced.

· Supports Dependent test methods, parallel testing, ordering and grouping of tests, load testing, and partial failure.

· In Built HTML reporter

· XML based test configuration

JUNIT

JUnit is an open-source testing framework. JUnit empowers Test Driven Development (TDD), where tests are written before writing actual code.

JUnit follows the approach of “testing first then coding”. This emphasizes setting up test data and testing the small piece of code first and then implementing the same. This increases productivity and helps in improving code stability. JUnit finds the bug early in code compared to other test frameworks.

Features of JUnit testing framework:

· There is a humongous list of Annotations to identify, execute, and support many features for the test methods and assertions for verifying the expected results.

· It provides Test Runner for executing the tests.

· JUnit provides a basic built-in template so that you may write small, simple test cases in no time.

· JUnit tests help you to write independent modules, thereby bettering the coverage of the test and the quality of the application.

· Junit provides robust assertions to test expected outcome

· Until the test execution is sailing through smoothly, you may relax watching at the green-colored test progress bar that shows while execution is in progress whereas it alerts you in ‘red’ as soon as the test fails a verification checkpoint.

· Test suites can be created in order to put a sequence or related set of test cases together.

· JUnit is non-supportive of parallel test execution

· JUnit aids the development of test-driven testing in a big way

· Junit tests can be integrated into CI/CD Pipeline

Feature Comparison chart for TestNG and JUnit

Following table compares JUnit vs TestNG on different features. The table excludes very specific or common features that are present in both these frameworks, such as testing exceptions, timeout settings, lifecycle callback methods etc.

​Feature	​JUnit	​TestNG	​Conclusion
Annotations	​Annotations Based	​Annotations Based	Both JUnit 5 and TestNG are annotation based. They are similar in nature and behavior.
​Ease of Use	Running tests need a certain dependency in JUnit	Writing tests and configuring them is easy in TestNG than JUnit	TestNG is better in terms of ease of use.
​IDE Support	​Supported on major IDEs such as Eclipse and IntelliJ IDEA.	Supported on major IDEs such as Eclipse and IntelliJ IDEA.	​Both of them are similar and provides easy integration with major IDEs.
​Data Provider	​Supports multiple ways to provide test data, such as methods, Enum, CSV, CSV Files etc.	Supports data provider methods and from test suite xml file.	JUnit is better for injecting test methods input data
Test Suite	In JUnit, Test Suite is run using @RunWith and @Suite annotations.	​TestNG uses an XML file to run Suite test.
HTML Reports	​JUnit needs external plugin maven-surefire-report-plugin to generate HTML reports.	TestNG automatically creates HTML reports for the test run.	TestNG HTML reports look outdated but it’s simple to use.
Running from Java Main Method	We can use JUnit 5 launcher API to run tests from java main method.	We can use TestNG run() method to execute tests from the java main method.	Both of them supports execution of test cases from java main method.
Assertions	JUnit provides enough assertion methods to compare expected and actual test results.	TestNG provides enough assertion methods to compare expected and actual test results.	Both of them are similar in terms of Assertions support.
Assumptions	JUnit supports assumptions to skip tests based on certain conditions.	TestNG doesn’t support assumptions.	JUnit is better if you want to skip tests based on conditions.
Test Order	Junit does not support Test Order.	TestNG supports Test Order for ordering test methods according to priority attribute.	TestNG is better when you want to execute tests in specific order.
Disable Tests	JUnit supports many ways to disable and enable tests. For example, based on OS, JRE, system properties.	TestNG supports disabling tests but it’s limited in functionality.	JUnit is better when you want to disable or enable tests based on conditions.
Parallel Execution	JUnit 5 doesn’t support parallel execution yet.	TestNG supports parallel execution if run through XML suite.	​TestNG is better for parallel execution as of now, JUnit 5 development is going on to support this feature.
Listeners	JUnit supports listeners through Launcher API, there is no mechanism to add listeners using annotations.	TestNG supports various types of listeners and can be added using annotations.	TestNG listener support is much better compared to JUnit 5.
​Dependency Tests	The dependency tests are missing in JUnit.	Dependency tests are present in TestNG.	​This is an advanced feature in TestNG over JUnit. With this feature, one method can be made as dependent on other such that the method will run only after the dependent method runs and passes else the dependent test won't run

Comparison of annotations and implementation between JUnit and TestNG

1. Test method annotation

There is no difference in the way we specify a method as a test method in both JUnit and TestNG.

​JUnit	TestNG
@Test	@Test

2. Suite related annotation

A method with annotation @BeforeSuite is executed once before the current test suite runs. This annotation is applicable only in TestNG.

​JUnit	TestNG
Not Applicable	@BeforeSuite

3. Annotation for a method before Class

This is the annotation for the method to be executed once before the first test method in the class is run.

​JUnit	TestNG
@BeforeAll	@BeforeClass

4. Annotation for a method before Test

This annotation executes once before the methods declared inside <test> tag of testng.xml. This annotation is available for TestNG only.

​JUnit	TestNG
Not Applicable	@BeforeTest

5. Annotation for the method to be executed before every method with @Test invokes

​JUnit	TestNG
@BeforeEach	@BeforeMethod

6. Annotation for the method to be executed after every method with @Test invokes

​JUnit	TestNG
@AfterEach	@AfterMethod

7. Annotation for method after Test

This annotation executes once after the methods declared inside <test> tag of testng.xml. This annotation is available for TestNG only.

​JUnit	TestNG
​Not Applicable	@AfterTest

8. Annotation for method after Class

This is the annotation for the method to be executed once after the last test method in the class is run.

​JUnit	TestNG
@AfterAll	@AfterClass

9. Suite related annotation

A method with annotation @AfterSuite is executed once after the current test suite runs. This annotation is applicable only in TestNG.

​JUnit	TestNG
Not Applicable	@AfterSuite

10.Group related annotation

The annotation is available only in TestNG.

The method with annotation @BeforeGroups runs before the test methods belonging to a particular group runs.

​JUnit	TestNG
Not Applicable	@BeforeGroups

The method with annotation @AfterGroups runs after the test methods belonging to a particular group runs.

​JUnit	TestNG
Not Applicable	@AfterGroups

11. Annotation to disable the execution of the Test method.

JUnit 5 provides an annotation to disable a specific test execution.

​JUnit	TestNG
@ignore	@Test(enabled=false)

Below is a code snippet for how we implement in Junit.

Output:

TestNG provides an attribute for @Test i.e. ‘enabled’ with the boolean value which decides if the execution of the method would be disabled or enabled

12. Timeout annotation

The annotation is same for JUnit 5 and TestNG

JUnit	TestNG
@Test(timeout=2000)	@Test(timeOut=2000)

Below is an example in Junit for Timeout test.

Code example for TestNG

As you can see from the test results, only timeoutTestTwo() is passed because its execution time was less than the timeout time defined. The timeoutTestOne() execution got failed because it took more time to complete than the timeout duration configured.

The timeout can be configured in the testNG.xml also as shown below

<suite name="Time test Suite" time-out="500" verbose="1" >

13. Expected Exception attribute

Exception class states that when the test executes, the exception of the given class is thrown. Exception test makes sure that when there is this predefined exception thrown, it is gracefully caught and notified on the logs. This is supported both in JUnit and TestNG with variation in the way both are declared.

JUnit	TestNG
​@Test(expected=NullPointerException.class)	@Test(expectedException=NullPointerException.class)

Lets look at an example for Junit. Add an error condition inside the showMessage() method.

Add an exception ArithmeticException to the displayMessage() test case.

The only difference in TestNG is “expectedException” is used in place of “expected” beside the @Test annotation.

14. Order of Execution related annotations

Both JUnit and TestNG support explicitly setting the order of tests for execution. In other words, setting the priority for test cases.

JUnit	TestNG
@TestMethodOrder(Alphanumeric.class)	@Test(priority=1)

JUnit 5 has annotation @TestMethodOrder() with MethodOrderer package’s built-in class – Alphanumeric.class or OrderAnnotation.class or Random.class as an input parameter for the annotation.

In the below code we can see that MethodOrderer orders all the tests alphabetically irrespective of the order they are declared.

Example for Junit

TestNG includes the attribute ‘priority’ for @Test annotation, which accepts a numerical value.

Example for TestNG

In TestNG, lower the priority number; higher is the priority of the test case method. Priority can contain only integer value. Here we can see that testcase 2 whose priority is -1 is executed first

15. Custom Test name

JUnit	TestNG
@DisplayName	Not applicable

By default, whenever we run a test, the test class and the test method name is printed in console. Junit5 provides a unique feature where we can mention custom descriptive names for class and test methods using @DisplayName annotation.

In TestNG there is no way to provide a custom name.

16. Dependent Tests

JUnit	TestNG
Not applicable	@Test(dependsOnMethods)

Junit doesn’t offer Dependency testing.

TestNG supports dependency testing. When the second test method wants to be dependent on the first test method, then this could be possible by the use of "dependsOnMethods" attribute. If the first test method fails, then the dependent method on the first test method, i.e., the second test method will not run. Let us understand through an example.

17. Parameterized Tests

JUnit	TestNG
@RunWith(Parameterized.class)	@Parameters

@DataProvider

Parameterized unit tests are helpful for testing the same code under several conditions. The main idea is to make the unit test method reusable and to test with a different set of inputs.

In Junit5, we have the advantage of test methods consuming data arguments directly from the configured source. By default, JUnit 5 provides a few source annotations like:

@ValueSource: we can use this with an array of values of type Short, Byte, Int, Long, Float, Double, Char, and String

@EnumSource – passes Enum constants as parameters to the test method

@MethodSource – passes external methods generating streams

@CsvSource – uses CSV values as a source for the parameters

Similarly, we have other sources like @CsvFileSource if we need to read a CSV file from classpath and @ArgumentSource to specify a custom, reusable ArgumentsProvider.

Here is a Junit example for parameterized test

Explanation of the code:

· Create a class(preparam). In this example, we are going to input two numbers by using sum (int,int) method which will return the sum of given numbers

· Create a parameterized test class

· Create a constructor that stores the test data. It stores 3 variables

· Create a static method that generates and returns test data.

· Creating a two-dimensional array (providing input parameters for addition). Using asList method we convert the data into a List type. Since, the return type of method input is collection.

· Using @Parameters annotation to create a set of input data to run our test.

· The static method identified by @Parameters annotation returns a Collection where each entry in the Collection will be the input data for one iteration of the test.

Consider the element

{1,2,3}

Here

firstNumber =1

secondNumber=2

expectedResult=3

Here each array element will be passed to the constructor, one at a time as the class is instantiated multiple times.

· Using @Before annotation to setup the resources (preparam.class here). The @Before annotation is used here to run before each test case. It contains precondition of the test.

· Using @Test annotation to create our test.

· Creating an assert statement to check whether our sum is equivalent to what we expected.

Parameterization in TestNG happens either through @Parameters annotation or through passing parameters in testing.xml file

Similar to TestNG Parameters, DataProviders are a means to pass data to test scripts in TestNG. Using DataProvider in TestNG, we can easily inject multiple values into the same test case. It comes inbuilt in TestNG and is popularly used in data-driven frameworks.

18. Parallel tests

This feature is only available with TestNG. Usually, test cases are kind of threads that are invoked one after the other. However, if you wish to save on the execution time, you can control this in TestNG by setting the tests to be run in parallel and providing the number of threads that need to run at once in testing.xml file. In the below snippet we can see the testng.xml.

19. Listeners

Listeners listen to every event that occurs within tests. Listeners are supported both in JUnit as well as TestNG. So, if you wish to perform certain tasks or show a specific message in the log before the test starts, after the test finishes, when the test is skipped, when the test is passed or failed, we have these Listener functions which enable us to do this

JUnit uses the Listener class, and TestNG uses a Listener interface.

TestNG write a listener class defining the Listener interface methods, and the second step is to call this Listener class name using @Listeners annotation in the main class.

JUnit also inherits the methods from the Listener parent class following which a Listener runner class is defined to apply the listener functions on one or more main class.

Conclusion

In this blog, we have compared TestNG and JUnit frameworks. We learned the common features supported in both the frameworks as well as additional features supported only in TestNG. Apparently, there are a couple of extra features only in TestNG, like parallel execution and dependency tests. Most of the features supported by TestNG are also available in JUnit.

There are slight deviations in terms of syntax, terminologies in JUnit vs TestNG for the common features. Those who undermined the strength of JUnit over TestNG would have realized until now i.e., JUnit is also one of the powerful automation frameworks.

And that brings us to the end of my blog “Junit vs TestNG”. In this article, we had a quick look at various ways of writing tests with each of these two test frameworks.

Hope you found my blog interesting and helpful. Please do give a thumbs up if you like it and do not forget to post your valuable feedback.