Get Rid Of Toxic Bugs On Your Apps With Detox!
Tags: e2e, tdd, javascript, mobile
Published: 2018-06-05 19:56:03

---
title: Get Rid Of Toxic Bugs On Your Apps With Detox!
canonical_url: https://marmelab.com/blog/2018/06/05/get-rid-of-toxic-bugs-on-your-apps-with-detox.html
cover_image: https://marmelab.com/images/blog/detox/banner.jpeg
published: true
description: End-to-end tests are common in the browser land, much less for native apps. A good solution is Detox, a gray box E2E testing framework for mobile apps.
tags: e2e, tdd, js, mobile
---

_Note: This post was originally posted on [marmelab.com](https://marmelab.com/blog/2018/06/05/get-rid-of-toxic-bugs-on-your-apps-with-detox.html)._

During the [React Europe Conference](https://marmelab.com/blog/2017/05/24/minutes-of-react-europe-2017.html) last year, I discovered a promising E2E testing framework for mobile apps called [Detox](https://github.com/wix/detox). Since then, I've kept in mind the will to give it a try. Recently, I developed an [example app](https://github.com/marmelab/beerexplorer-detox) just for that purpose. It allowed me to see what's under the hood of Detox. Here is my feedback.

<div style="overflow: hidden">
<iframe width="720" height="405" src="https://www.youtube.com/embed/Ndof_lgJmx8?rel=0&amp;showinfo=0" style="margin-left: -1px" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>
</div>

## What Are End-to-End Tests?

In software development, we list **3 distinct kinds of automated tests**:

- **Unit Tests**: As their name suggest, they test functions individually, **in isolation from the rest** of the codebase. They're used to prevent unexpected code changes and to ensure that functions do what they are supposed to do.
- **Integration Tests** (or **Service Tests**) are responsible for the proper connection between code parts and APIs. They test the application components altogether from a technical perspective.
- **End-To-End Tests (E2E)**: They allow to test the application as a whole, **in its execution environment**, as a human could do.

According to [Martin Fowler](https://martinfowler.com), all these kinds of tests can be classified into a [Test Pyramid](https://martinfowler.com/bliki/TestPyramid.html) from the slowest / most expensive to the fastest / least expensive. 

![Test Pyramid](https://marmelab.com/images/blog/detox/test_pyramid.png)

At the bottom of the pyramid, **Unit-Tests** must be the most common tests. Utopically, each function must be tested. Some integration tests and a bit less E2E tests are needed to ensure that the entire stack is working well.

Whereas E2E tests are very important, some people sometimes go too far with an excessive E2E test coverage. Another diagram called [Ice-Cream Cone](https://medium.com/@fistsOfReason/testing-is-good-pyramids-are-bad-ice-cream-cones-are-the-worst-ad94b9b2f05f) represents this anti-pattern as well.

![Ice Cream Cone](https://marmelab.com/images/blog/detox/icecream.png)

But everyone agrees that writing and debugging E2E tests is a **tedious task**. 

## Introducing Detox

Detox was first released in 2016 by [Tal Kol](https://twitter.com/koltal) and [Rotem Mizrachi-Meidan](https://twitter.com/rotemmiz), 2 engineers working at [Wix](https://wix.com/). Wix is a cloud-based platform which allows non-technical users to create their own website.

![Wix Engineering](https://marmelab.com/images/blog/detox/wixengineering.png)

Detox defines itself as **a Gray Box End-To-End testing automation framework for mobile apps**. That means that it brings the same context aware test capabilities we're already using in browser apps through [Selenium](https://www.seleniumhq.org/). This way, Detox allows to break off from manual [Quality Insurance](https://en.wikipedia.org/wiki/Quality_assurance) test processes, which are time-consuming and incomplete.

Contrary to [Appium](https://github.com/appium/appium), its the main competitor, Detox uses JavaScript both on the server side and on the client side. Despite this strong requirement, Detox allows using Jest, Mocha, AVA, or any other JavaScript test runner you like.

## Gray Box Testing vs Black Box Testing

As a Gray Box test framework, Detox shares both White and Black Box capabilities. Let's see together what it means.

[Black Box test frameworks](https://en.wikipedia.org/wiki/Black-box_testing) allows to take over an execution context (a browser, a software, mobile apps, etc) and send control commands to them.

This test methodology does not allow to access to the internal state of the application though. That's why it's necessary to manually check the existence of elements to ensure test the state after a transition.

```js
function* navigate() {
    yield driver.navigate().to(`http://localhost/#/login`);
    yield driver.wait(until.elementLocated(By.css('#loginform')));
}
```

Gray Box frameworks are extending [White Box test frameworks](https://en.wikipedia.org/wiki/White-box_testing) capabilities. This way, they do the same thing as Black Box frameworks, except that they access the internal state of the execution context.

![Black Box vs Grey Box](https://marmelab.com/images/blog/detox/blackgreywhitebox.png)

Accessing the internal state of the execution context permits to know when the application is idle, and to synchronize operations adequately. That's why Detox is more powerful than most classic E2E test frameworks.

## Less Flakiness

If you've already used an E2E test framework before, you've certainly encountered some strange, random and unexpected errors. These errors are therefore called "flakiness errors". As you encounter them, you feel like our good old Harold and it's not very funny.

![E2E Flakiness is not so funny](https://marmelab.com/images/blog/detox/harold.jpg)

To mitigate this behavior, we usually add some `sleep` (or `timeouts`) calls into the test suite, to ensure that the application is in an idle state before resuming the test process. Even though this "hack" works, it results in slower tests, without really solving the problem because on a slow test system, the sleep delay can sometimes be not enough.

```js
function* login() {
    yield driver.findElement(this.elements.loginButton).click();
    yield driver.sleep(5000);
}
```

Thankfully, as a Grey Box Framework, Detox is able to **access the application state** and then to determine if the application is in an idle state or not. To achieve this `idle` synchronization task, Detox rely on 2 natives Grey Box drivers called [EarlGrey](https://github.com/google/EarlGrey) (for iOS) and [Espresso](https://developer.android.com/training/testing/ui-testing/espresso-testing.html) (for Android).

![Detox Inner Working](https://marmelab.com/images/blog/detox/earlgrey_espresso.png)

Because **Detox runs in JavaScript**, it communicates with drivers using a JSON based protocol to invoke control commands on devices.

<div class="tips">

A special synchronization mechanism has also be developed for React-Native apps, so Detox supports React Native

</div>

## A Concrete Use Case

As already said in the introduction, I've developed a dedicated application to give Detox a try. Since I'm a beer lover, I couldn't resist creating a simple beer registry app called [beerexplorer](https://github.com/marmelab/beerexplorer-detox).

![Detox Example App](https://marmelab.com/images/blog/detox/detox.gif)

<div class="tips">
    
Most blog posts about Detox build applications with <a href="https://facebook.github.io/react-native/">React-Native</a>, this one is no exception. Still, it's important to know that Detox is fully cross-platform and supports both iOS and Android.

</div>

## Detox Setup

I've tried to setup Detox to run it on my own Android phone. Despite all my efforts, I've not been able to make it work. So I went back to an iOS emulator.

The Detox setup is relatively simple. It consists of installing the [detox](https://www.npmjs.com/package/detox) npm package, then calling 2 commands: `detox build` and `detox test`.

Then, Detox uses an existing configuration defined in `package.json` to determine which test runner and configuration it should use. All available devices configurations are stored under the "detox.configurations" key. Android and iOS device configurations can be mixed.

```js
/* package.json */

{
  "name": "beerexplorer",
  "detox": {
    "test-runner": "jest",
    "runner-config": "e2e/config.json",
    "configurations": {
      "ios.sim.debug": {
        "binaryPath": "ios/build/Build/Products/Debug-iphonesimulator/beerexplorer.app",
        "build": "xcodebuild -project ios/beerexplorer.xcodeproj -scheme beerexplorer -configuration Debug -sdk iphonesimulator -derivedDataPath ios/build",
        "type": "ios.simulator",
        "name": "iPhone 7"
      }
    }
  }
}
```

When I call the `detox test` command, Detox looks in the `runner-config` configuration file for the `setupTestFrameworkScriptFile` to execute before running tests. I've called this file `init.js`.

```js
// e2e/config.json

{
    "setupTestFrameworkScriptFile" : "./init.js"
}
```

Here is the test init file:

```js
// e2e/init.js

const detox = require('detox');
const config = require('../package.json').detox;

beforeAll(async () => {
  await detox.init(config);
});

afterAll(async () => {
  await detox.cleanup();
});
```

Tests can either run on a local emulator, a hidden emulator, or even on a distant CI like [Travis](https://travis-ci.org/)!

## Detox Usage

Out of the box, Detox provides a small but powerful set of tools, that allows to control the device, select elements in the UI, and execute actions on these elements.

<div class="tips">
    
Detox tools are asynchronous. Therefore you have to use Promises, or, like in my code below, ES6 async/await.

</div>

### Device

The `device` object allows to control the device directly, without relying on the tested application. Here are some examples of usages from [the documentation](https://github.com/wix/detox/blob/master/docs/APIRef.DeviceObjectAPI.md).

```js
// Launch app with specific permissions
await device.launchApp({ permissions: { calendar: 'YES' } });

// Simulate "home" button click 
await device.sendToHome();

// Simulate geolocation
await device.setLocation(32.0853, 34.7818);
```

Some `device` functions are specific to a given platform, such as `device.reloadReactNative` for React-Native and `device.shake` for iOS.
 
### Selectors / Matchers

As with other test frameworks, Detox gives the possibility to match UI elements in different ways.

The easiest (and recommended) way to match elements is to use ids. Sadly, this technique is only available on React-Native.

```js
// id declaration
<Touchable testID="BeerListItem">
...
</Touchable>

// element selection
await element(by.id('BeerListItem'));
```

It's also possible to match elements with other methods like `text`, `label`, `type` or `traits`. More informations on the corresponding [matchers documentation](https://github.com/wix/detox/blob/master/docs/APIRef.Matchers.md#byidid).

### Actions And Expectation

Once selected, it's possible to [trigger actions](https://github.com/wix/detox/blob/master/docs/APIRef.ActionsOnElement.md) and [execute assertions](https://github.com/wix/detox/blob/master/docs/APIRef.Expect.md) on elements. As an example, here is a test suite from the homepage of the "beerexplorer" project.

```js
describe('home', () => {
  beforeEach(async () => {
    await device.reloadReactNative();
  });
  
  it('should have a list of beers', async () => {
    await expect(element(by.id('BeerList'))).toBeVisible();
  });

  it('should go to detail on beer touch', async () => {
    await element(by.id('BeerListItem')).atIndex(0).tap();
    await expect(element(by.id('DetailBackground'))).toBeVisible();
  });

  it('should show all beers', async () => {
    await waitFor(element(by.label('Lindemans Kriek'))).toExist().whileElement(by.id('BeerList')).scroll(50, 'down');
    await expect(element(by.label('Lindemans Kriek'))).toExist();
  });
});
```

As you can see, tests are very expressive and easy to read. There's no need to add  more test about the existence of an element between transitions, thanks to the idle state synchronization.

## Conclusion

Although satisfied by Detox at the end, I'm still disappointed by the difficulty of setting up an E2E test suite on Android. Because of my poor experience on mobile application tests, I don't pretend to give you the more accurate opinion. But I still think this framework (and its documentation) still to be improved for Android.

Apart from that, the developer experience with Detox is very pleasant. I never found myself in difficulty when writing tests. Also, live preview in the emulator is very empowering.

Nevertheless, If you're testing your application on iOS only, feel free to give it a try. You won't take many risks, except to be greatly satisfied by the clarity and the stability of the tests.

If you want to read more on the subject by other authors, I recommend the following:

* [The Practical Test Pyramid](https://martinfowler.com/articles/practical-test-pyramid.html)
* [Detox: Gray Box End to End Testing Framework for Mobile Apps](https://hackernoon.com/detox-gray-box-End-To-End-testing-framework-for-mobile-apps-196ccd9564ce)
* [Testing in React-Native - Jest & Detox](https://pillow.codes/testing-in-react-native-jest-detox-d7b3b79a166a)
* [Gray Box vs Black Box](https://reqtest.com/testing-blog/grey-box-testing/)