Tips Learned From Years of Automated End-to-End Testing
Tags: testing, devops
Published: 2022-11-15 19:37:33

This blog post was initially featured on the [Keyhole Dev Blog](https://keyholesoftware.com/2022/11/14/tips-learned-from-years-of-automated-end-to-end-testing/) on Nov 14, 2022.

Imagine for a moment that we’re getting ready to publish a new app or feature. Following the principles of Test Driven Development (like we always do), we have created a full suite of unit tests. We’re never pressed for time, so we’ve also built out full coverage integration and functional tests.

In order to ensure our front-end is behaving as expected, we’ll need to either manually step through the application or just push our commit to the main branch and let our continuous integration pipeline do the building and testing for us. But, if we wrote our end-to-end (E2E) tests without automation in mind, we might find the results lacking in usefulness…

This post isn’t a discussion on what E2E testing is nor a tutorial on how to get started. For that, resources like [Smartbear](https://smartbear.com/solutions/end-to-end-testing/), [CircleCI](https://circleci.com/blog/what-is-end-to-end-testing/), and [Playwright](https://playwright.dev/) have already published articles and tutorials that do a great job of covering that ground. In this post, we’ll talk through a few tips I’ve picked up over 5 years of championing fully automated end-to-end testing.


## Early and Often
A major perk of automated testing is the ability to run whenever we want. Unlike manual E2E testing, there is no concern with tying up any developers to execute or monitor the runs. With that in mind, the first tip is to run our end-to-end tests early and often. A suite of E2E tests can be utilized as soon as our app is ready to build and then frequently throughout every promotion into downstream environments.

Running them this early and frequently will help verify that with every build the expected behaviors still exist. The earlier we catch unexpected behavior, the lighter the effort will be to correct it. When a team first begins to build out their E2E testing, the main workflows can be validated first almost as a way of smoke testing, and then adapted from there to cover more of the application.


## Structured to Self-Certify
Now that we’ve established our testing rhythm, how will we write our tests so that they are effective and efficient? We could add steps after every action that ensure the action is executed correctly, but I would argue that in most cases this adds unnecessary bloat.

That’s why the second tip is that tests should be structured so they are self-certifying. To give an example, an app with a button that triggers a modal would have a testing step that clicks the button, followed by a step that interacts with the modal. If the modal exists, the test continues, and we can be reasonably assured that the button is working.

Instead of adding steps that only affirm behavior, our test is written such that the affirmation is baked into the next interaction. If the modal didn’t pop up, our interaction step would rightfully fail. What we’re left with is a streamlined and readable test structure that is primarily interacting with the application under test.


## Data = Complexity
Finally, to a tip that was learned the hard way, data adds complexity. In a modern development pipeline with multiple environments, having a dependence on data means that we will need an infrastructure set up to maintain consistent data throughout all of our environments. This can be a massive effort by itself. Instead, writing tests that are data-independent or, at the very least, clean up after themselves will save us a lot of frustration.

With a CRUD app in mind, at a high level, this may look like writing steps to create a new record. Then, use that record to step through the read, update, and delete operations. The data would then live with the test and could be written in such a way that it can be utilized in any environment.


## Bringing It Together
Bringing it all together, we now have a suite of end-to-end tests that are data-independent, self-certifying, and set up to run automatically after every build. The time that our team was previously spending manually verifying that our app meets behavior expectations and requirements is now passed on to an automated pipeline. This should give us much more time to do something more productive, like creating new features or ignoring the bugs that we totally meant to squash last week.

Thanks for reading! Let me know what you think in the comments below, and if you liked this post, there are many more on the [Keyhole Dev Blog](https://keyholesoftware.com/blog/).