Selenium – Headless Webdriver Tests

Selenium – Headless Webdriver Tests

Testing trends

Selenium WebDriver tests will no doubt make up a massive part of most modern UI test automation suites, and for good reason. It’s arguably the best tool out there for the job and it’s incredibly easy to use and it will integrate with just about any language.

Due to its ease of use, building a large number of tests in a short period of time is very simple. Over a long period of time, with a team of automation testers all contributing, this can lead to an absolutely massive number of tests…which might be a problem.

UI tests are absolutely fantastic for mimicking real users, but because it is doing just that, they are far from the quickest tests to run and running a large number of them can be a very lengthy process. Test automation is often seen as an inexpensive, almost free way of testing (once the tests are written of course) as most companies will run them on an automation server, which runs behind the scenes and frees up everyones time to do other things. And while this may be true, poorly managed test suites can cause huge bottlenecks within a deployment or testing process.

This article is by no means going to solve the problems associated with an oversized test suite, but it will address another approach to running WebDriver tests, which some of you may not be aware of.

Headless WebDriver tests are essentially a way of running your normal WebDriver test but without a visible browser launching. It’s running the same test it otherwise would have, just completely in the background with no browser UI.

So why would you want to do that? If it’s just running the same script, how is it any better (or worse) than a normal WebDriver test? Let’s have a look…

Probably the biggest benefit of headless browsers and one that I was getting at earlier when talking about our growing test suite is the execution time. Because there is no browser UI, we’re essentially saving time on all the places where a test would typically have to wait for the rendering of HTML or dealing with complex CSS or JavsScript calls. Now, without having to worry about that, tests can run anywhere from 2x to 20x faster depending on the nature of your test. If you can imagine the time saving from running all your tests even just 2x faster.

Another benefit is just solely based on its convenience. There’s a lot of situations where you are testing in the UI but might not necessarily need to be. Using an example where you may just need to perform a task as a pre-requisite such as creating an account on a website for a test that requires an account to created first. You use the UI to do it but you’re not testing the UI at this point. Because headless tests run in the same way, you can do anything you’d normally do so a task like this would make perfect sense.

One other example would be content testing or any other kind of testing where you are scraping data from a page, if you’re simply needing to visit a page and scrape content to use in a test, you don’t need a browser UI to do that, and you can do it in a lot less time using a headless browser.

There are a number of similar benefits of a similar vein, but it’s not all good news. Unfortunately, the biggest disadvantage of headless browser testing is the fact that it isn’t a true reflection of a real user experience. While the speed benefits may tempt you in to running all of your tests headless, you’d need to be careful that you’re not missing or disguising bugs. When tests are run in a browser you can see, you’re able to pick up any visual errors such as incorrectly located elements, or broken layouts. When everything is run headless, you’re unable to see these problems, so tests may tell you one story, when in reality it’s something completely different.

Another disadvantage is how difficult it makes fixing or debugging tests when they do go wrong. Being unable to see the test as you step through the code can make it pretty difficult to work out what’s going on.

Now you understand a brief summary of the pros and cons, it’s probably worth while you experimenting and getting it working yourself so you can see where it works for you and where you think the benefits really lie.

To make a test headless is simple, you can either do it at a test level if you create your WebDriver instance per test, or, if you want to stick to the kind of code we created in our framework series, you could do something like this:

var chromeOptions = new ChromeOptions();
chromeOptions.AddArguments("test-type");
chromeOptions.AddArguments("chrome.switches", "--disable.extensions");
chromeOptions.AddArguments("headless");
DriverBase.Instance = new ChromeDriver(TestContext.CurrentContext.TestDirectory, chromeOptions, implicitWait);

The key bit here is the AddArgument call where we add headless as an argument. Once this is passed as an option to the Chrome WebDriver, all test runs will be headless.

As a takeaway, when is best to use headless browser testing? And the answer to that entirely depends on you and your testing requirements. But it’s important to note that headless testing can be frowned upon due to it not being the same as a real user. Your users won’t have headless browsers so why should you use them to test?

If the situation lends itself to headless testing, and it doesn’t compromise the quality of your testing, then absolutely use it but always make sure that when you’re weighing up whether it’s right to use or not, never put performance as the sole reason to use it. If performance comes at the expense of quality, it’s definitely not the right choice.