Cypress: Remote Tests Execution Made Simple
By: Alexander Andryashin
Tags: Selenium, Test Automation, Kubernetes, Selenium Test Automation, Cypress

Hi there! This is our next article in the long cycle of articles related to efficient browser automation infrastructure. Full list of articles is published on our website. Those of you who are following our activities should already know that several years ago we created Moon — an efficient solution for running Selenium tests in Kubernetes cluster. At the time Selenium was de-facto worldwide browser automation standard. But at the same time web applications were more and more using single-page approach with a lot of business logic being executed on the client side. Selenium was created in 2004 when the majority of web applications consisted of plain HTML with small inclusions of Javascript generated on the server side. Testing new generation of single-page web applications requires features like inspecting network traffic being produced by AJAX requests, efficiently interacting with DOM tree and CSS styles, subscribing to Javascript events and so on. Lacking these features Selenium is not well suitable for testing single-page web applications and so a new generation of tools appeared several years ago. We have already covered tools like Playwright and Puppeteer in our previous articles but were never talking about Cypress. Today we are going to cover this tool.

What is Cypress?

Cypress is an extremely popular Javascript framework for testing modern web applications. Cypress architecture is very similar to what Selenium 1 (aka Selenium RC) had. During browser startup a Javascript extension is loaded to browser. Thus you get full control on what’s happening in the tested web application. Yes, we know such approach has limitations related to Javascript security limitations in browser. But what you get is more important:

Visual step-by-step debugging of running tests. You can navigate back and forth to every single step of your test and see what happens in browser for every such step. This is very efficient for finding potential issues. Automatic waiting. Cypress out of the box can wait for page elements to appear before actually trying to do something with these elements. In Selenium community such feature is usually delivered by third-party libraries like Selenide. Powerful mocking and stubbing. This includes not only basic Javascript method stubs, but also an ability to control that methods were called with correct arguments, an ability to fully control Javascript clock and even powerful network control features. Powerful network control. Asynchronous HTTP requests in a single page application have the same importance as the blood system in human body. Thus it’s very important to fully control what’s being sent over network while you test. Cypress networking feature not only visualizes HTTP requests or does assertions on request URL, body or headers. You can also completely stub responses to every network request, delay their execution or make them fail. This allows to organize efficient frontend testing without even launching the backend.

Parallel Cypress tests execution

Becoming popular any new browser automation tool sooner or later needs to scale well. That means you should be able to run more and more tests in parallel. New tools are very often being tried on pet projects with a few dozens of tests to run overall. Such limited number of tests can be easily executed on a single virtual machine or server. But when you try to adopt the same tool to a real project in your company total number of CPUs needed for browsers can grow fast. To still be able to test everything fast, you may need more than one server and this is where your problems usually begin.

Previously it was only possible to run Cypress tests across multiple servers by using online paid solution called Cypress Dashboard. Hopefully, Moon now supports Cypress out of the box. By installing Moon you now get:

Unlimited parallel Cypress tests execution. Browser sessions are automatically being distributed across any number of hosts in Kubernetes or Openshift cluster. Cost-efficient Cypress infrastructure. Because of automated scaling Moon consumes only as many servers or virtual machines as currently needed by your tests. Because of that you can dramatically lower your infrastructure cost, especially when using cloud platforms such as AWS, Google Cloud or Microsoft Azure. A user interface for parallel Cypress tests. This interface allows you to easily watch automated tests being executed and debug them step by step.

This is how it looks like in real life:

An example project demonstrating Cypress usage with Moon can be found here. This project is a minimalist Cypress project with 1 more dependency added to package.json - @aerokube/cypress-moon . This package automatically downloads cypress-moon executable for your platform. This executable is needed to automatically execute your Cypress tests remotely. Basic usage of this command is as follows:

$ cypress-moon https://moon.example.com/cypress/chrome

Once you execute this command in your Cypress project directory — Moon creates a new Chrome browser and all project source codes are automatically executed in this browser. In the example project above cypress-moon command is executed from package.json scripts section.

By default Moon executes all the tests from your project in the same browser. If you wish to execute individual test spec files or only a set of files matching some pattern — then you have to add spec parameter to Moon URL. This parameter accepts the same values as Cypress --spec flag:

$ cypress-moon https://moon.example.com/cypress/chrome?spec=cypress/integration/my-spec.js

$ cypress-moon https://moon.example.com/cypress/chrome?spec=cypress/integration/some*.js

Supported Browsers and Cypress versions

Initially Cypress was compatible with Chrome browser only. Nowadays it also supports Chromium, Firefox, Microsoft Edge and Electron-based applications testing. Moon comes with a set of prebuilt images with stable version of supported browsers: Chromium, Chrome, Electron, Firefox and Microsoft Edge. Detailed information about available images can be found on Aerokube images page. To choose what browser to use you simply change Moon URL in cypress-moon command call:

$ cypress-moon https://moon.example.com/cypress/firefox

$ cypress-moon https://moon.example.com/cypress/electron

$ cypress-moon https://moon.example.com/cypress/edge

$ cypress-moon https://moon.example.com/cypress/chromium

By default this will launch the latest available browser version. We recommend to always run your tests against browser image compatible with your Cypress version. To specify an exact Cypress version — use this version in Moon URL:

More powerful stuff

In addition to simple Cypress tests execution you can use more powerful features just by changing Moon access URL:

You can easily test on different screen resolutions:

$ cypress-moon https://moon.example.com/cypress/firefox/cypress-8.3.0?screenResolution=1280x1024

You can override operating system environment variables used by browser. One of the most common use cases is setting preferred browser language:

$ cypress-moon https://moon.example.com/cypress/firefox/cypress-8.3.0?env=LANG%3Dde_AT.UTF-8&env=LANGUAGE%3Dat:de

If you need multiple environment variables — just pass env parameter multiple times and don't forget about correct URL encoding.

Default Cypress behaviour is running headless (invisible) browsers. If you need to debug your running tests — make your browser visible:

$ cypress-moon https://moon.example.com/cypress/firefox/cypress-8.3.0?headless=false

You can add free-form title to every running browser allowing to distinguish different test runs in the user interface:

By default Moon will remove browser when Cypress tests execution finished. In some cases you may want to leave it running for further investigation. This can be easily achieved with noExit parameter. Browser will be running until you explicitly stop cypress-moon command for example by pressing Ctrl+C or sending termination signal from another process:

$ cypress-moon https://moon.example.com/cypress/firefox/cypress-8.3.0?headless=false&noExit=true

This is how it looks like:

Moon is running in Kubernetes cluster on top of hardware servers or virtual machines. To easily deploy just the same infrastructure for local tests debugging you can use minikube tool. This tool quickly creates a small Kubernetes cluster on your workstation and you can then deploy Moon to this cluster. Detailed information on how to do this is described in Moon documentation. We also demonstrated how to do this in this video.

Conclusion

With the introduction of Cypress parallel tests execution, Moon now covers four the most popular browser automation frameworks: Selenium, Playwright, Puppeteer and Cypress. What is the most important you can run any combination of tests using different frameworks in parallel on the same Moon cluster. This is especially useful when migrating from one technology to another or having multiple teams using different browser automation stacks. If you are interested in having a powerful cluster but don’t have engineers for software maintenance, take a look at Moon Cloud — a Moon distribution being maintained by Aerokube team.