Best tips & tricks for E2E Maestro with React Native
Tags: reactnative, maestro, testing
Published: 2023-05-07 20:21:36

A collection of practical tips and techniques that simplify your developer experience of Maestro E2E framework!


---

## 1. Installing Maestro

Let's add a new script to the `package.json` file:

```json
{
  "scripts": {
    "install-maestro": "MAESTRO_VERSION=1.36.0 curl -Ls 'https://get.maestro.mobile.dev' | bash"
  }
}
```

I define the version we now support using the environment variable `MAESTRO_VERSION`. The use of the same version by our team and CI is guaranteed.

docs: https://maestro.mobile.dev/getting-started/installing-maestro

---

## 2. Identify the app as being run by Maestro. 

When an app is run by e2e, we occasionally need to configure the behavior of the app:

1. Disable animation / pause video;
2. Stop sending analytics;
3. Turn off React Native LogBox `LogBox.ignoreAllLogs()`.


For this case you need to add [`arguments`](https://maestro.mobile.dev/api-reference/commands/launchapp#sending-launch-arguments) to a [`launchApp`](https://maestro.mobile.dev/api-reference/commands/launchapp) command

```yaml
- launchApp:
    appId: "com.example.app"
    arguments: 
       isE2E: "true"
```

And then on JS side using [`react-native-launch-arguments`](https://github.com/iamolegga/react-native-launch-arguments) package you can access that arguments

```tsx
import { LaunchArguments } from "react-native-launch-arguments";

LaunchArguments.value().isE2E // true
```

---

## 3. Dot env file support `.env`

You need to create `.env` file with next text:

```bash
-e UNIVERSAL_USERNAME=user-e2e@test.com
-e UNIVERSAL_PASSWORD=123qwe
```

After that you can use that file as argument for `test` or `record` commands: 

```json
{
  "scripts": {
    "record": "$HOME/.maestro/bin/maestro record @.env"    
    "test": "$HOME/.maestro/bin/maestro test @.env"
  }
}
```

Additionally, you can combine various env files (`maestro test @.env @.env.prod`), in which case all variables from the most recent file will replace, extend, or reset env variables from earlier files.


```json
"test": "$HOME/.maestro/bin/maestro test @.env"
"test-prod": "yarn test @.env.prod"
"test-staging": "yarn test @.env.staging"  
```

Example:

```bash
# .env file:
-e BACKEND=staging
-e USERNAME=test@test.com
-e PASSWORD=123qwe
#-----------------
# .env.prod
-e BACKEND=                 # unset variable
-e USERNAME=myprod@test.com # edit existing 
-e SKIP_UNBOARDING=true     # add a new variable 
```

The following values will be available in the Maestro environment:

```yaml
${BACKEND}         // null
${USERNAME}        // "myprod@test.com"
${PASSWORD}        // "123qwe"
${SKIP_UNBOARDING} // "true"
```
---

## 4. Different `appId` between platforms or environments

It's common practice to differentiate production apps from non-production (staging/beta) apps using a Bundle ID (for example: `com.example.myapp` & `com.example.myapp.staging`)

```yaml
# .maestro/my-test.yaml
appId: ${APP_ID}
name: My test name
---
- launchApp # used an `appId` specified above
```

Then using `-e MY_VAR=myVal` you can pass `APP_ID` in each test:

```json
{
  "scripts": {
    "test": "$HOME/.maestro/bin/maestro test"
    "test-android-staging": "yarn test -e APP_ID=com.example.myapp.staging"
    "test-android-prod": "yarn test -e APP_ID=com.example.myapp"
  }
}
```

and finally, an example of a terminal command:

```bash
yarn test-android-staging .maestro/my-test.yaml
# ^^^ Run test using staging app

yarn test-android-prod .maestro/my-test.yaml
# ^^^ Run test using prod app
```

---

## 5. Filtering tests to run using tags


You have the ability to run a particular set of tests using [`tags`](https://maestro.mobile.dev/cli/tags) in Maestro.

Your nightly builds, for instance, frequently verify the following crucial features:

```yaml
appId: com.example.myapp
name: Sign-in email+password
tags:
  - "on:pre_release"
  - "on:pull_request"
  - "on:nightly_build"
  - "feature:auth"
  - "backend:prod"
  - "backend:pre_prod"
  - "backend:staging"
---
- launchApp
# ...
```

You can use `--include-tags="on:nightly_build"` and path to your directory with `yaml` files to run all test that include an `on:nightly_build` tag:

```json
{
  "scripts": {  
    "test": "$HOME/.maestro/bin/maestro test"
    "nightly-test": "yarn test --include-tags='on:nightly_build' ./path_to_flows"
    "pr-test": "yarn test --include-tags='on:pull_request' ./path_to_flows"
  }
}
```

---

## 6. Complicated `waitFor` logic

I had the issue that Maestro couldn't wait for the appearance of the "A" or "B" elements. 

But using a `repeat` & `runFlow` we can simulate this logic: 

```yaml
- evalScript: ${output.myStatus = 'unknown'}
- evalScript: ${output.attemptsCount = 0}
# ^^^ specify initial parameters

- repeat:
    while:
      true: ${output.myStatus === 'unknown'} 
    # run until the status changes
    commands:
      - runFlow:
          when:
            visible: "Booking failed"
          commands:
            - evalScript: ${output.myStatus = 'error'}
      # ^^^ first `runFlow` wait for an error case
      - runFlow:
          when:
            true: ${output.myStatus === 'unknown'}
          commands:
            - runFlow:
                when:
                  visible: "Booking success"
                commands:
                  - evalScript: ${output.myStatus = 'success'}
      # ^^^ second nested `runFlow` will wait for a success case

      # The remaining logic deals with a timeout case
      - runFlow:
          when:
            true: ${output.attemptsCount > 10} # Check attempt limit 
          commands:
            - evalScript: ${output.myStatus = 'timeout'}
      - evalScript: ${output.attemptsCount = output.attemptsCount + 1} # Incerunmen an attempt counter


# ^^^ After that you can run any logic based on `output.myStatus`
# for example throw an error if status isn't `success`
- assertTrue: ${output.myStatus === 'success'}

```


## 7. Useful bash commands

**Android Debug** (required to run metro in background)

```bash
./android/gradlew assembleDebug -p ./android # build debug apk
find ./android -type f -name "*.apk"         # find apk file
yarn start                                   # run metro bundler
adb reverse tcp:8081 tcp:8081                # open port
adb install "<path_to_apk_file>"             # `path_to_apk_file` - result of `find` command above
```

**Android Release** (bundle JS with apk)

```bash
./android/gradlew assembleRelease -p ./android # build debug apk
find ./android -type f -name "*.apk"           # find apk file
adb install "<path_to_apk_file>"               # `path_to_apk_file` - result of `find` command above
```


## 8. Run on CI using GitHub Actions

⚠️ If you want to run Maestro test using GitHub Actions on Android **you can't use default ubuntu runners** as nested virtualization is disabled (that required by Android Emulator).


See `run-on: *` table:

|Runners|Android|iOS|Price (min)|Spec|
|---|---|---|---|---|
| [large ubuntu](https://docs.github.com/en/actions/using-github-hosted-runners/using-larger-runners) | ✅ | ⛔ | [from $0.016 to $0.256](https://docs.github.com/en/billing/managing-billing-for-github-actions/about-billing-for-github-actions#per-minute-rates) | 4-64CPU 15-256GB RAM |
| [`ubuntu-*`](https://github.blog/2023-03-01-github-actions-introducing-faster-github-hosted-x64-macos-runners/) | ⛔ | ⛔ | [$0.008](https://docs.github.com/en/billing/managing-billing-for-github-actions/about-billing-for-github-actions#per-minute-rates) | 2CPU 7GB RAM |
| [`custom buildjet`](https://buildjet.com/for-github-actions)  | ✅ | ⛔ | [from $0.004 to $0.048](https://buildjet.com/for-github-actions/docs/about/pricing) | 2-34CPU 8-64GB RAM | 
| [`macos-*-xl`](https://github.blog/2023-03-01-github-actions-introducing-faster-github-hosted-x64-macos-runners/) | ✅ | ✅ | [0.32$](https://docs.github.com/en/billing/managing-billing-for-github-actions/about-billing-for-github-actions#per-minute-rates) | 12CPU ??GB RAM |
| [`macos-*`](https://github.blog/2023-03-01-github-actions-introducing-faster-github-hosted-x64-macos-runners/) | ✅ | ✅ | [0.08$](https://docs.github.com/en/billing/managing-billing-for-github-actions/about-billing-for-github-actions#per-minute-rates) | 3CPU 14GB RAM |



Simple Github Actions workflow `.github/workflows/mobile-e2e.yml` to run Android & iOS e2e test using Maestro:

- no caching
- no versionlock (xcode,java,nodejs,cocoapods) 

```yaml
name: Maestro E2E

on: [push]

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true # auto cancel prev. run

env:
  MAESTRO_VERSION: 1.27.0
  ANDROID_ARCH: x86_64

jobs:
  ios_e2e:
    name: iOS
    runs-on: macos-12 # or macos-12-xl
    steps:
      - uses: actions/checkout@v3

      - name: Installing Maestro
        run: curl -Ls "https://get.maestro.mobile.dev" | bash # will use `MAESTRO_VERSION` from env

      - name: Installing Maestro dependencies
        run: |
          brew tap facebook/fb
          brew install facebook/fb/idb-companion

      - name: Install node_modules
        run: yarn install --frozen-lockfile

      - name: Install Pods
        working-directory: ios
        run: pod install

      - name: Build app for simulator
        working-directory: ios
        env:
          DERIVED_DATA_PATH: my_build
        run: |
          xcrun xcodebuild \
            -scheme "Myapp" \
            -workspace "Myapp.xcworkspace" \
            -configuration "Release" \
            -sdk "iphonesimulator" \
            -destination "generic/platform=iOS Simulator" \
            -derivedDataPath "${{ env.DERIVED_DATA_PATH }}"
          
          echo "Print path to *.app file"
          find "${{ env.DERIVED_DATA_PATH }}" -type d -name "*.app"
      # ^^^ Path to *.app file (based on derivedDataPath + working-directory):
      # ./ios/my_build/Build/Products/Release-iphonesimulator/Myapp.app

      - name: Run e2e tests
        env:
          APP_PATH: "./ios/my_build/Build/Products/Release-iphonesimulator/Myapp.app"
                    # ^^^ change this path to your *.app file
        run: |
          echo "Launching iOS Simulator"
          xcrun simctl boot "iPhone 14 Pro"
          
          echo "Installing app on Simulator"
          xcrun simctl install booted "${{ env.APP_PATH }}"
          
          echo "Start video record"
          xcrun simctl io booted recordVideo video_record.mov & echo $! > video_record.pid

          echo "Running tests with Maestro"
          $HOME/.maestro/bin/maestro test .maestro/ --format junit

      - name: Stop video record
        if: always()
        run: kill -SIGINT $(cat video_record.pid)

      - name: Store video record
        if: always()
        uses: actions/upload-artifact@v3
        with:
          name: e2e_ios_report
          path: |
            video_record.mov
            report.xml

  android_e2e:
    name: Android
    runs-on: macos-12 # or buildjet-4vcpu-ubuntu-2204, ubuntu-22.04-4core, macos-12-xl
    steps:
      - uses: actions/checkout@v3

      - name: Installing Maestro
        run: curl -Ls "https://get.maestro.mobile.dev" | bash # will use `MAESTRO_VERSION` from env

      - name: Install node_modules
        run: yarn install --frozen-lockfile

      - name: Build apk for emulator
        working-directory: android
        run: |
          ./gradlew assembleRelease --no-daemon -PreactNativeArchitectures=${{ env.ANDROID_ARCH }}

          echo "Print path to *.apk file"
          find . -type f -name "*.apk"

      - name: Install Maestro and run e2e tests
        uses: reactivecircus/android-emulator-runner@v2
        env:
          APK_PATH: ./android/app/build/outputs/apk/release/app-release.apk
                    # ^^^ change this path to your *.apk file
        with:
          api-level: 33 # Android 13
          arch: ${{ env.ANDROID_ARCH }}
          script: |
            adb install "${{ env.APK_PATH }}"
            $HOME/.maestro/bin/maestro test .maestro/ --format junit

      - name: Store tests result
        uses: actions/upload-artifact@v3
        with:
          name: e2e_android_report
          path: |
            report.xml
```