_edited_edited.png)
Selenium Webdriver is a powerful automation tool for testing web applications. It’s an open-source library bundled with a rich set of features. Using it for testing a website is just like simulating the actions of a real user. All these abilities together make it a perfect tool for automation. But we need proper approach to get the best out of it.
In most cases, the logic used in the implementation of test scenarios was spot-on, but the design and scalability were a matter of concern. This is a common sight for anyone who has sailed through the waters of Selenium test automation
With the Selenium framework, the ‘One size fits all’ approach does not apply to test automation. Though there are no thumb rules for the design and development of scalable automation tests, there are certain principles that you must follow when writing tests using the Selenium framework.
In this blog, we will discuss the best practices for Selenium test automation which might help you develop well-designed and scalable test suites.
A majority of the Selenium best practices discussed subsequently are agnostic of the programming language being used for Selenium test automation. So, regardless of the language you use for automation testing with Selenium, these Selenium best practices will still be of value.
1. Avoid blocking the sleep calls
It is widely known that the behavior of web applications depends on many external factors such as network speed, device capabilities, access location, load on the back-end server and more. These factors make it quite challenging to predict the actual time taken to load a specific web element. Here, adding a delay or timeout before performing any actions on the web element will delay the execution while allowing the particular web element to load.
Delay achieved using a blocking sleep call (e.g., Thread.sleep in Java, time.sleep in Python) blocks the test thread for the specified number of seconds. For a single-threaded application, it will block the thread and effectively the process as well. Blocking sleep calls are instrumental in adding the required delay, but the duration of delay depends on numerous factors. There is no guarantee that the delay being added will work all the time. For example, if you have added a delay of 5 seconds, the specified web element does not load even after 10 seconds.
driver = webdriver.Chrome()
driver.get("https://dsportalapp.herokuapp.com/")
time.sleep(5000)
What is the potential alternative to blocking sleep calls? Selenium provides Implicit wait and Explicit wait that handle delays more efficiently than sleep. Implicit wait informs the browser to wait for a specified time duration for all the web elements present on the page. If the element is available faster than the implicit delay time, the execution moves to the next line of code execution. For example, if an implicit wait of 10 seconds is added for a specified element but the element loads in 3 seconds, the script does not wait for the remaining 7 seconds.
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
On the other hand, the explicit wait is another type of dynamic Selenium wait that is used to stop the script execution on a particular condition for a specified duration. WebDriverWait and ExpectedConditions can be used to achieve condition-based waits.
2. Name the Test cases and Test Suites appropriately
When working in a team, there are cases where your team members may be required to enhance the tests that you had written. If you revisit the same test after a couple of months, you might not be able to figure out the purpose of a test, until you go through the complete implementation.
3. Set the browser zoom level to hundred percent
Sometimes during the Selenium test automation process, you may notice that the test implementation is not working correctly on specific browsers. This is typically the case when performing cross browser testing on outdated browsers like Internet Explorer.
Irrespective of the web browser on which automation testing with Selenium is performed, setting the browser zoom level to 100 percent is one of the Selenium best practices that should be followed. This setting gives a native mouse feel and ensures that the native mouse events are set to the correct coordinates.
JavascriptExecutor js = (JavascriptExecutor) driver;
js.executeScript("document.body.style.zoom='100%'");
Along with this setting, Protected Mode Settings in Internet Explorer for each zone must be the same else you may end up getting a ‘NoSuchWindowException’ exception.
4. Maximize the browser window
One of the first actions performed by a tester for Selenium test automation is taking the web page’s screenshot. Screenshots are taken during testing the process to help developers in debugging issues and help key stakeholders track product development progress. Screenshots also help detect whether test failure is due to application failure or problem in the test script being used for automation testing with Selenium.
By default, Selenium does not open the browser window in the maximized mode. This can affect the screenshot of the web page that is typically attached in test reports. Maximizing the browser window immediately after the test URL is loaded ensures that a screenshot of the entire web page is captured.
driver.manage().window().maximize();
This is one of the Selenium best practices that should be followed irrespective of the browser on which test automation is performed.
5. Choose the best-suited Web locator
One of the challenges with Selenium test automation is that automation tests have to be modified if there are changes in the implementation related to locators used in the test code. ID, Name, Link text, XPath, CSS Selector, DOM Locator, etc. are some of the frequently used web locators in Selenium webdriver.
With so many web locators, it is necessary to choose the right one to minimize the impact on tests caused due to changes in the user interface. Link Text is usually preferred if there is a dynamic situation. ID, Class, and Name are not only the easiest to use but also less brittle than other web locators.
There are situations where the only option left is XPath. XPath engines might vary from one browser to another. Hence, there is no guarantee that XPath for one browser would work seamlessly for another browser. A browser like Internet Explorer does not have a native XPath engine for locating web elements. Hence, JavaScript XPath Query Engine usually is used for finding elements by XPath in IE. This will be slower than the native XPath engine. XPath is more brittle as reordering elements in a page or introducing a new web element can cause the existing implementation with XPath to fail. If XPath is the only option, you should use Relative XPath instead of Absolute XPath.
For testing internationalized applications, we might not be able to use LinkText or partialLinkText if the anchor tags do not contain any ID or class. For localization or internationalization testing, partial href should be used so that even if the language on the site changes, the href link still points to the same location.
The ideal web selector order is: id > Name > CSSSelector > XPath.
6. Create a Browser Compatibility Matrix for cross browser testing
To start with, it is a challenging task to narrow down a list of browsers (browser versions, too) and operating systems to run automated tests on. To manage this task, it is recommended that you use a browser compatibility matrix.
Cross Browser testing is a challenging task as you need to prioritize testing on different browser plus OS combinations. If we include browsers and their browser versions, it would add up to a vast number. Formalizing a list of (browser + OS + device) combinations is of prime importance as it would help in prioritizing the combinations that have to be taken up for cross browser testing. This formalized list is also called Browser Matrix or Browser Compatibility Matrix.
Browser Matrix is vital information drawn from product analytics, geolocation, and other detailed insights about audience usage patterns, stats counter, and competitor analysis. Browser Matrix will help cover all the relevant browsers (that matter to your product), thereby reducing the development and testing efforts.
Required Data points for Browser Compatibility Matrix
When drafting a browser compatibility matrix, you need to get certain data points around which your whole strategy will revolve.
1. Platform
Platform defines the means by which your user is accessing your website. It can be either laptop, desktop, mobile, or a tablet. You need to be sure what is the most liked platform by your audience and ensuring its compatibility is utmost important.
2. Desktop Browser Usage
If the user is using desktop for the same, better note what browser is the maximum used browser for a specific geo. Check for all combinations of mostly used browsers with different windows and mac operating systems.
3. Mobile Browser Usage
If the users love to surf on mobile, you need to ensure the mostly used browser there and again the browser, and mobile devices combinations.
4. Compare different platforms
Figure out what your users are preferring over the other and take the preferred ones into immediate attention. You must know what are the best combinations for your users in every aspect.
Collating data from the sources listed above will help teams better understand their audience (local or global). Additionally, research-based on these data points will help teams better understand the device-browser-OS environments most relevant to the website’s audience.
Once the matrix has been determined, developers and QAs need to ensure that their development and test efforts are focused on the finalized set of browsers. Once the application is developed, QAs need to test this software application on real device-browser combinations rather than emulators. This helps them test software in real user conditions and explore bugs that an end-user may encounter in the real world.
You can see a sample browser compatibility matrix below
Browser | Version tested | Operating system | Desktop Market share(%) | Overall Market share(%) | Support level |
Latest stable release | Windows 10 | 54.56 | 29.88 | Full | |
Mozilla Firefox | Latest stable release | Windows 10 | 13.76 | 7.54 | Full |
Safari | Latest stable release | Mac OS X (10.11) | 7.04 | 3.86 | Limited |
Microsoft Edge | 13(Latest stable release) | Windows 10 | 12.40 | 6.79 | Full |
Internet Explorer | 11(Latest stable release) | Windows 10 | 6.38 | 3.49 | Limited |
Others | N/A | N/A | 2.96 | 1.62 | N/A |
7. Implement Logging and Reporting
If a particular test in an extensive test suite fails, it can become challenging to locate the failing test case. Logging can be a huge savior in such cases as console logs at appropriate places in the test code help develop a better understanding of the code and help in zeroing on the problem.
Some of the popular log levels debug, info, warning, error, and critical. Adding unnecessary logs in the test implementation can cause delays in the test execution. Hence, it is recommended to add logs with level error (and/or critical) in scenarios that aid in tracking the cause of failure.
Along with logging, reporting is an integral part of Selenium test automation as it helps in determining the status (pass/fail) of the tests. This is where automation test reports can play a huge role as it helps in keeping track of the progression of test suites (or test cases) and corresponding test results. Reports also help minimize the time required for the maintenance of test data due to improvement in the readability of the test output. Analyzing features and accessing test coverage becomes easy with automation test reports.
Selenium test automation without logging and reporting defeats the sole purpose of using the Selenium framework. That is why logging and reporting are considered one of the best test practices in automation.
8. Use Design pattern and principles
When writing Selenium test automation scripts, you must keep a check on its maintainability and scalability. This is possible if changes in the web page UI requires minimal or no changes in the test script. Suppose the scripts are not appropriately maintained, i.e. different scripts using the same web element. In that case, whenever there is a change in the web element, corresponding changes must be made at multiple places in the test script.
This is where Page Objects, a popular web UI automation pattern, comes handy as it enhances test maintenance and reduces code duplication. In-Page Object Model (POM), a centralized object repository is created for controls on a web page. The web page is implemented as a separate class. Hence, every web page being tested will have its corresponding page object class.
This eases code maintenance as Selenium automation scripts do not directly interact with the page’s web elements. Instead, a new layer (page class/page object) resides between the test code and controls on the web page.
Along with better maintainability, using POM helps in reducing the code size as page object methods defined in different page classes can be reused across multiple test automation scripts.
Leveraging Page Object Model is one of the Selenium best practices that can aid in:
Improving test maintenance
Minimizing code changes due to updates in product UI
Enhancing code reusability
Simplifying the visualization and model of the web page under test
9. Use BDD Framework with Selenium
Behavior Driven Development, popularly called BDD, is a popular development approach that helps in writing test cases in plain English language called Gherkin. This means that along with developers and testers, members with minimal technical know-how can participate in the development of tests.
BDD frameworks help in filling the void between business people and technical people as all of them get the opportunity to work on the enhancement of the tests. Gherkin files created for BDD testing consist of a combination of features, steps, and scenarios, along with relevant Gherkin keywords such as Given, When, Then, etc. The format of feature files and keywords being used is uniform irrespective of the BDD framework being used. This makes it easier to shift from one BDD framework to another as the learning curve is very low.
BDD tests are more usable when compared to TDD tests as changes in business specification or feature specification would involve minimal changes in corresponding BDD features and scenarios. When compared to TDD (Test Driven Development), BDD tests have an improved shelf-life as tests are created using business and feature specifications. Some of the popular BDD frameworks are Cucumber, Behave, SpecFlow, etc.
10. Follow a uniform directory structure
When working on tests that use the Selenium framework, it is essential to focus on the test code’s maintainability. A standard project can consist of src and Test folders. The src folder can contain sub-directories that contain Page Objects, Helper functions, and files that contain web locator information used in test scenarios. The Test folder can include the actual test implementation.
We don’t have a standard rule when it comes to a directory structure for Selenium test automation. However, Selenium best practices recommend us to have a directory structure that separates the test implementation from the test automation framework. This helps in better organization of the test code.
11. Identifying and Prioritizing test cases
Testing complex web applications can be challenging at times. Prioritizing certain test cases over others makes it easier to achieve test coverage goals. QAs must have clarity on which test cases are critical and need to be executed on priority.
For example, a login page is a vital part of any web-application. Naturally, automating tests to verify login page makes sense. This is because the login page rarely undergoes any modifications but offers an important function. Thus, testing it would be easy, and running the tests would cover a high priority task in the pipeline early on.
The test case prioritization of regression testing is done taking into account the business requirements, previous test cycle experience on functioning of the existing features, and the delivery timelines.
In order to identify test case scenarios, testers must understand the functional requirements of the system. Once the test case scenarios have been identified, the non-functional requirements must be defined. Non-function requirements include operating systems, security features and hardware requirements.
12.Use Data-Driven Testing for Parameterization
A website should be tested against different combinations of browsers, devices, and OS combinations (i.e., multiple datasets). Hard coding of test values in test automation scripts is not a scalable solution as it would lead to unnecessary bloatware and possible repetition of test code.
A better solution is using parameterization for achieving data-driven automation testing with Selenium. Parameterization helps in executing test cases against different input combinations or data sets. More extensive the data set, the better the test coverage. This, in turn, helps in improving product quality and implementing good Selenium test practices.
13. Do not use a single driver user implementation
WebDrivers in Selenium are not interchangeable. The situation of executing automated cross browser tests on a local machine is entirely different from being executed on a continuous build server. In that environment, it will be wrong to assume that the next test will use Firefox WebDriver (or Chrome WebDriver or any other WebDriver).
When integration tests are executed are in a continuous build environment, the test will only receive a RemoteDriver (i.e., WebDriver for any target browser). Amongst all Selenium best practices, it is recommended to use Parameter notes to manage different browser types and get the code ready for simultaneous execution. Small frameworks can be created in Selenium using LabelledParameterized (@Parameters in TestNG and @RunWith in JUnit). This practice will be useful in ensuring that the implementation is flexible enough to work with different browser types.
14.Come up with Autonomous Test case design
The automation test design solely depends on the goal that you are planning to achieve from the test. Things can get complicated when the test suite consists of several tests. One of the most critical Selenium best practices is to avoid inter-dependency between different tests in a test suite and separating the specification of what the test is supposed to do from the test execution strategy.
The other major advantage of using autonomous tests is you can explore parallelism to expedite the test execution. If tests are dependent, the outcome of one test also affects the second test. Hence, they cannot be executed in parallel as they are largely interconnected. You could make use of relevant decorators & markers in case you still cannot avoid having dependency between tests and want to skip execution of a test if its dependent test has failed.
However, it is recommended to come up with autonomous tests whenever possible. Otherwise, you might miss the opportunities to leverage Selenium and test framework that can do wonders to the test execution!
15. Use Assert and Verify in appropriate scenarios
There are numerous cases in automation testing with Selenium, where you would want to halt the test execution on encountering a hard error. For example, you are using Selenium to automate testing of the Gmail login page, but the web locator being used for locating the sign-in box is not correct. In this case, assert should be issued as the remaining tests will falter out as they are dependent on the sign-in page.
Asserts should only be used when you want to halt the test execution in case of a hard failure like the one mentioned above. If the assert condition is false, execution stops, and further tests will not be executed. On the other hand, Verify should be used where the criticality of the error is low, and you still want to proceed with test execution irrespective of the status of Verify condition.
16. Avoid Code duplication or wrap Selenium calls
One of the most common practices for Selenium test automation is avoiding unnecessary duplication of code. You might be using different web locators such as XPath, ID, etc. for accessing the web elements present on the web page. The code that is frequently used in the implementation should be created as a separate API, so that code duplication is minimal.
Avoiding duplication also helps in the reduction of code size and improves the maintainability of the test code. Wrapping Selenium calls is one of the Selenium best practices that can significantly impact maintaining a complex test suite.
17. Leverage Parallel testing in Selenium
A primary factor for Selenium’s popularity is its support for parallel testing. Almost all popular test frameworks such as PyTest, PyUnit, TestNG, Cucumber, etc. provide features for executing tests in parallel on a Selenium Grid.
Parallel testing in Selenium lets you execute the same tests simultaneously on different environments (i.e., a combination of browsers, platforms, and device emulators). Using Selenium, it is recommended to enable parallel testing in the implementation as it reduces the test execution time by a significant margin.
Executing tests in parallel is a faster approach than sequential testing, as it provides testers with wider test compatibility in less span and reduces test time considerably.
Also, for continuous integration and continuous delivery, you need to run functional tests frequently and quickly. And if you run tests in parallel, it will not only buy you more time but also allows you to have detailed reports of test data. Development teams can later use these reports to find issues in the code and fix them quickly in order to optimize CI/CD.
Bonus - Now that we have looked at the top Selenium best practices, it is time, we also deep dive into some of the worst Selenium practices that should be avoided when performing automation testing!
Worst Practices for Automation testing in Selenium
1. File Downloads
Downloading a file on any web platform is initiated by clicking on a link or a button that prompts users to download the file. The same operation can be automated using Selenium, but the downside is that API does not show any progress regarding the file download. Hence, you would not be aware of whether the download functionality is being tested or not.
Moreover, downloading files is not an essential aspect of testing user interaction with the web platform. Instead, the download link located using Selenium (and any required cookies) should be passed to an HTTP request library like libcurl.
2. Captchas
CAPTCHA or ‘Completely Automated Public Turing test to tell Computers and Humans Apart’ is specifically designed to check whether a human is performing the necessary operation. In short, it is designed to prevent automation.
Disabling CAPTCHAS in the test environment and adding a hook for bypassing the CAPTCHA are the two strategies that can be used for getting around CAPTCHA checks in the test environment.
3. Two-factor Authentication
Two-factor authentication (2FA) is a security mechanism where an OTP (One-Time Password) is generated using Authenticator mobile apps. Gmail has an authenticator app that will make random codes to authenticate whether the actual account holder is trying to login to a new system (or app).
Automating 2FA in Selenium can be a considerable challenge, and it might not guarantee performance. It is better to avoid automating 2FA in test environments. You also have the option of disabling 2FA for a specific set of users to use their credentials (on any test system). Alternatively, you can also configure test machines so that 2FA is disabled for a certain family of IP addresses (or selected IP addresses).
4. Link Spidering or Crawling
It is not recommended to use WebDriver to perform link spidering - in other words, to spider through links. You can do it, but WebDriver is definitely not the ideal tool for this task because it needs time to start up. That can take up to a minute, depending on how your test is written, just to get to the page and traverse through the Document Object Model.
Furthermore, writing the logic of traversing pages and capturing links is simply a waste of time. As the time taken is variable, Selenium should never be used for spidering. Instead, use curl commands or libraries such as BeautifulSoup for crawling the web.
5. Automation of Logins on gmail, facebook and other related platforms
Using automation (with Selenium or other test frameworks) for logging in to Gmail, Facebook, or any such websites is against their policies. Secondly, the testing can be highly unreliable.
Instead, the developer can use the third-party APIs provided by the corresponding website as it is more reliable, less subject to change, and more secure to use. Using Selenium automation for login purposes on these websites should be avoided at any cost.
6. Avoid Test Dependency
Parallelism is an integral part of cross browser testing (and Selenium automation testing) as it aids in expediting the test execution. However, the execution sequence is not guaranteed.
The development of inter-dependent tests should be completely avoided as it is a bad practice and does not guarantee reliability. The test suite might pass in one test execution, whereas the same test suite can fail in the next cycle. The practice of autonomous test case design should be followed when coming up with Selenium automation test cases (or test suites).
7. Performance testing
Selenium framework is designed for automation testing. Performance testing using Selenium and WebDriver is not recommended, as it is not optimized for doing that task and you might not get the expected results.
On a website, there are many external factors such as browser startup speed, the speed at which HTTP server response is received, etc. that are beyond the tester’s control. Rather than choosing Selenium WebDriver for performance testing, you should select ready-made tools that are specifically designed for performance testing.
To discover what to improve, you need to be able to analyze overall performance independent of environmental differences, identify poor code practices, and break down the performance of individual resources (i.e., CSS or JavaScript).
8. Testing HTTP Response codes
HTTP status codes are standard response codes given by website servers on the Internet. The codes help identify the cause of the problem when a web page or other resource does not load properly. In automated functional testing, checking the status code is not a particularly important detail of a test's failure; the steps that preceded it are more important. It's better to keep API testing at this layer.
Conclusion
Selenium automation seeks to reduce manual testing efforts, increase execution speed, and identify the maximum number of bugs at the earliest. However, in order to get the most out of their Selenium scripts, testers must follow the best practices highlighted above. This will also help in establishing a reliable test cycle.
That wraps up the best and worst practices for Selenium automation testing. Hope you found my blog helpful and would love to know your valuable feedback in the comments.