Avoiding the trap of fragile end-to-end tests
There’s a popular principle in software that you should always try to respect. It’s called the DRY rule and stands for don’t repeat yourself. This means that you shouldn’t write the same code twice. If you do, that’s a good time to clean up your code by refactoring. The same refactoring used in the red-green cycle - that is, without breaking the tests! Why? Simply because each repetition is more code for you to change if the behavior of your application changes. It’s also more code to go wrong.
Quite often when writing end-to-end tests, you need to visit the same page across multiple tests. Sometimes you need to click on a button or fill in a field. When these are one line statements, often in a different order, it’s easy to miss that you are repeating yourself.
Take this snippet:
@Test
public void aStudentUsesTheCalculatorToMultiplyTwoBySixteen() {
webDriver.get(baseUrl);
WebElement submitButton = webDriver.findElement(By.id("submit"));
WebElement leftField = webDriver.findElement(By.id("left"));
WebElement rightField = webDriver.findElement(By.id("right"));
WebElement typeDropdown = webDriver.findElement(By.id("type"));
// ACT: Fill in 2 x 16
leftField.sendKeys("2");
typeDropdown.sendKeys("x");
rightField.sendKeys("16");
submitButton.click();
// ASSERT
WebDriverWait waiter = new WebDriverWait(webDriver, 5);
WebElement solutionElement = waiter.until(
ExpectedConditions.presenceOfElementLocated(
By.id("solution")));
}This user journey was to test multiplication. Imagine you had other tests to validate journeys for different calculation types. That test could include the code to find page elements and interact with them in order to multiply. If you wrote more tests, you could quickly end up with similar logic in lots of tests.
The tests work, right? Isn't that good enough?
Let's have a look at the bit of our form where we selected the calculationType:
<form action="#" th:action="@{/calculator}" th:object="${calculation}" method="post">
...
<div class="col">
<select id="type" th:field="*{calculationType}" class="form-control form-control-lg">
<option value="ADDITION" selected="true"> + </option>
<option value="SUBTRACTION" selected="true"> - </option>
<option value="MULTIPLICATION"> x </option>
<option value="DIVISION"> / </option>
</select>
</div>
...
</div>
<div class="row">
<div class="col">
<button id="submit" type="submit" class="btn btn-primary">=</button>
</div>
...Now, imagine what would happen if we changed this from a drop-down to radio boxes? Or a designer wanted to restructure and modify the style of our site, creating new HTML pages. We’d see our tests break and have to find all the tests that think they know how our page works and change them!
I've seen people have to pay penance for scattering logic about their website across lots of tests. A few changes across a number of web pages can suddenly make a lot of brittle tests go bright red. 😳 The fix is not always that straight forward either.
Martin Fowler, a founder of the Agile movement and a great engineer whose writing you should vacuum up, introduced the PageObject model. It applies object-orientated programming to how we think of a web page. Rather than the code we had above, we’d put all those page-specific methods and selectors in one place. A selector is that By.Id("left") from the code above, which tells our test to look for <input id=”left">.
The previous tests were fragile since little changes to the HTML could break it, and this style would become increasingly brittle as our application grew. Let's create a PageObject for our calculator together so that we don't have those issues!
As you saw, we created a new class with each selector annotated using WebDriver's annotation@FindBy(By.id("a selector")). We created an instance of the CalculatorPage object, and then called WebDriver's PageFactory.initElements(webDriver, calculatorPage) using our WebDriver instance and PageObject; this factory turns our @FindBy annotated fields into WebElements based on the content in the page.
Did you see how much more readable our test was? Rather than messing around with selectors, we were able to call calculatorPage.multiply(2, 16) and let our PageObject remote control fill in the appropriate form fields!
What happened here?
Our PageObject allowed us to use object orientation to encapsulate the selectors of a page. That means that we created a class for the CalculatorPage and put a method in called multiply(). Any changes to either the process of multiplication, what its page looks like, or calculation in general, are all built into this one class. We can write lots of tests which include multiplication, but only have one place which defines how we control and interpret the UI.
Since you have all your knowledge about the behavior of a web page in one place, this also protects you from scattering dependencies on the structure of your web page through lots of tests. There is only one dependency on the structure of any given page; that is the page object which you have modeled from it!
Now it’s your turn!
Grab the code and add a test to see if the title is "Open Calculator." You want to use By.id("title") to get back WebElement. You can then call getText() on it to retrieve the title of the page to use in your assertion. Put those selectors and a getTitle() method into your page object.
git clone https://github.com/OpenClassrooms-Student-Center/AchieveQualityThroughTestingInJava.git
git checkout p3-c5-sc1
Or use your IDE to clone the repository and check out the branch p3-c5-sc1. You can now explore the examples from the screencast and try running the tests with:
mvn test
Avoiding the inverted pyramid
You are now armed with all the tools to test at each tier of the pyramid but beware. It’s easy to get seduced by the dark side of testing. Rather than picking what seems easy today, consider an approach that will allow you to have continued confidence in your automated tests.
It's easy to stray from the balance between timely feedback and confidence by writing more tests towards the top of the pyramid, slipping into the anti-pattern known as the inverted pyramid.
Have you ever thought about changing a screenplay when you were hoping for a plot twist in your favorite show? Probably not. When we talk about a new feature in an application, it is just as unnatural to talk fundamental code changes, especially when you can just say things like “Display a read-only error to the user.” While this is the right language to use, as it describes behavior in a user interface, it may lead you to pick the wrong test: hearing this, most will immediately think of writing an end-to-end test. But if you have an existing integration test proving you can show errors to the user, a simple unit test would be enough.
The further you get along in an application, the easier it is to default to tests higher up on the pyramid. The more end-to-end tests you write, the slower your feedback gets, and the longer it takes for developers to know they are on the right track. You may even convince yourself that it’s faster to do manual testing! Telltale signs you can look out for, which indicate that your pyramid is starting to invert:
Your tests often break, and it’s normal to have to re-run them until they pass.
Your tests are unreliable to the point where you need to manual test just to be sure.
Your integration and end-to-end tests are so slow, you’ve had to make them run separately. You may do this for a good reason, but if they are taking hours, you must question what you can push down to your unit tests.
Your team has stopped caring whether all the tests run.
New unit tests are not being written, or only have a handful of happy path tests.
Many bugs are found through manual testing, or by users soon after a release.
Stay vigilant! But if you make a mistake, don't worry. On a healthy team, the person who breaks the tests either brings in doughnuts or gets shot with a nerf gun. This is the environment you want to create. 🙂
Let's recap!
PageObject models allow you to use object orientation to encapsulate the selectors of a page and provide meaningfully named methods which let you perform web driver operations on that page.
Using the PageObject model protects you from scattering dependencies on the structure of your web pages.
Beware the inverted pyramid. It’s easy to be lured into testing everything at the end-to-end tier of the pyramid. These tests rapidly slow down, and can become unreliable.
