Exploring Cirrus CI for Android
Tags: android, ci, testing
Published: 2019-12-23 13:05:06

I recently wrote about my experience using various cloud-based CI services for [running Android instrumented tests on CI](https://dev.to/ychescale9/running-android-emulators-on-ci-from-bitrise-io-to-github-actions-3j76) for opensource projects, and how I landed on a solution using a custom [GitHub Action](https://github.com/ReactiveCircus/android-emulator-runner).

As mentioned in that [article](https://dev.to/ychescale9/running-android-emulators-on-ci-from-bitrise-io-to-github-actions-3j76), **most** cloud-based CI providers today don't have **KVM** enabled in the host VMs which is required for running hardware-accelerated x86 / x86_64 emulators in a docker environment.

I recently came across [Cirrus CI](https://cirrus-ci.org/), a less-known cloud-based CI provider with native KVM support and it's completely free for opensource projects. Despite having a positive experience with GitHub Workflow using my custom [android-emulator-runner](https://github.com/ReactiveCircus/android-emulator-runner) action that runs on macOS VMs, I was keen to try out KVM-enabled containers provided by Cirrus CI as I'd still be more comfortable with a solution based on a standard Linux/docker environment which [Google has been putting more effort into recently](https://github.com/google/android-emulator-container-scripts).

In this article I'm going to share some of my experiences with **Cirrus CI** - specifically, how I migrated **[FlowBinding](https://github.com/ReactiveCircus/FlowBinding)**'s instrumented tests from GitHub Action to Cirrus CI task, along with some of the features provided by Cirrus CI you might find useful for Android in general.

-------------------------

## Cirrus CI Overview

Cirrus CI doesn't have a fancy product / marketing website like some of the other providers such as [CircleCI](https://circleci.com/). The main entry point is  [cirrus-ci.org](https://cirrus-ci.org/) which covers feature descriptions, pricing information, examples, and comprehensive documentations.

### Features

A list of features can be found [here](https://cirrus-ci.org/features/). For Android (and opensource projects in particular) some of the more interesting ones are:

- Free for opensource projects (public repositories on GitHub)
- Provides Linux, Windows, macOS and FreeBSD containers
- Provides KVM-enabled containers
- Containers on community clusters (free plans) can use a maximum of 8.0 CPUs and up to 24 GB of memory
- Supports local and remote build cache
- Supports build matrix

Here's a comparison with other CI services found on Cirrus CI's website:

![Alt Text](https://thepracticaldev.s3.amazonaws.com/i/5rxy5bpca3u60ogc9cnu.png)

### Pricing

Cirrus CI's pricing models can be found [here](https://cirrus-ci.org/pricing/).

As mentioned before, Cirrus CI is free for public repositories. There is a **per-user** [concurrency limit](https://cirrus-ci.org/faq/#are-there-any-limits) of 8 Linux VMs, but this is quite generous for a free plan and should be more than enough for most projects.

Commercial plans for private repositories are available at $10/seat/month which is again quite affordable compared to some of the other more popular services, but for my projects I haven't needed to upgrade.

### Installation

Cirrus CI currently only supports repositories hosted on GitHub. To get started, go to the Cirrus CI Application on [GitHub Marketplace](https://github.com/marketplace/cirrus-ci) and setup a plan for your account or organization.

![Alt Text](https://thepracticaldev.s3.amazonaws.com/i/n4nno7ey9k92dqetssnn.png)

That's basically all you need to do to setup Cirrus CI for a project. Everything else (except a couple of security options) is configured in a `.cirrus.yml` file in your project's root directory.

### Writing CI Tasks

**Task(s)** are the building block of a `.cirrus.yml` configuration file. A task defines a sequence of **instructions** to run and the **environment** to execute these instructions in. The equivalent concepts of **tasks** and **instructions** in GitHub Actions are **jobs** and **steps** respectively. Here's a simple task that assembles a debug APK:

```
assemble_task:
  container:
    image: reactivecircus/android-sdk:latest
  assemble_script:
    ./gradlew assembleDebug
```

Tasks in `.cirrus.yml` have the `task` suffix. In this case `assemble_task` is the only task we define in the config.

**Script** is one of the [supported instructions](https://cirrus-ci.org/guide/writing-tasks/#supported-instructions). Similarly scripts need to have the `script` suffix e.g. `assemble_script`.

`container` is a field where we can define the docker image used for the task. **[reactivecircus/android-sdk](https://github.com/ReactiveCircus/docker-android-images#android-sdk-image)** is a docker image with the minimum Android SDK components required for building Android projects.

A comprehensive guide for writing tasks can be found [here](https://cirrus-ci.org/guide/writing-tasks/).

It's worth noting that **code completion** support from IntelliJ IDEA / Android Studio is available when writing `.cirrus.yml`, thanks to the YMAL plugin bundled in the IDE and the supported [schema store](http://schemastore.azurewebsites.net/json/).

![Alt Text](https://thepracticaldev.s3.amazonaws.com/i/sn75k8urivvja5rlaiz2.png)

### Dashboard

Cirrus CI's build dashboard is hosted at [cirrus-ci.com](https://cirrus-ci.com/). Once logged in with your GitHub account you'll see a list of builds for all your projects using Cirrus CI. Individual tasks and build results are available for each build.

![Alt Text](https://thepracticaldev.s3.amazonaws.com/i/rq36v5d9cusdwwxv549j.png)

While the UIs don't look as pretty as some of the more popular products, overall I found them effective and responsive.

## Running Instrumented Tests

Now let's look at how we can write a task that runs Android instrumented tests on hardware-accelerated emulators.

We'll start by converting the following GitHub workflow that uses the **android-emulator-runner** action for running instrumented tests on macOS:

```
jobs:
  test:
    runs-on: macOS-latest
    steps:
    - name: checkout
      uses: actions/checkout@v2

    - name: run tests
      uses: reactivecircus/android-emulator-runner@v2
      with:
        api-level: 28
        arch: x86_64
        target: default
        emulator-options: -no-window -gpu swiftshader_indirect -no-snapshot -noaudio -no-boot-anim
        script: ./gradlew connectedDebugAndroidTest
```

Converted task in `.cirrus.yml`:

```
connected_check_task:
  name: Run Android instrumented tests
  only_if: $CIRRUS_BRANCH == "master" || CIRRUS_PR != ""
  env:
    API_LEVEL: 28
    TARGET: default
    ARCH: x86_64
  container:
    image: reactivecircus/android-emulator-28:latest
    kvm: true
    cpu: 8
    memory: 24G
  create_device_script:
    echo no | avdmanager create avd --force --name "api-${API_LEVEL}" --abi "${TARGET}/${ARCH}" --package "system-images;android-${API_LEVEL};${TARGET};${ARCH}"
  start_emulator_background_script:
    $ANDROID_HOME/emulator/emulator -avd "api-${API_LEVEL}" -no-window -gpu swiftshader_indirect -no-snapshot -noaudio -no-boot-anim -camera-back none
  wait_for_emulator_script:
    adb wait-for-device shell 'while [[ -z $(getprop sys.boot_completed) ]]; do sleep 3; done; input keyevent 82'
  disable_animations_script: |
    adb shell settings put global window_animation_scale 0.0
    adb shell settings put global transition_animation_scale 0.0
    adb shell settings put global animator_duration_scale 0.0
  run_instrumented_tests_script:
    ./gradlew connectedDebugAndroidTest
```

`name` is a field where we can define a custom name for the task.

`only_if` is a keyword that controls whether the task should be executed. `$CIRRUS_BRANCH == "master" || CIRRUS_PR != ""` here means we only want to run this task on commits on the master branch or any pull request.

Environment variables can be specified in the `env` block. Here we are only specifying the configurations of the system image used for creating the emulator. Other common environment variables such as `JAVA_TOOL_OPTIONS` or `GRADLE_OPTS` can also be specified here.

`container` defines the host VM used for running the task and its configurations. We use [reactivecircus/android-emulator-28](https://github.com/ReactiveCircus/docker-android-images#android-emulator-images) as the docker image which has the minimum SDK components for running hardware-accelerated emulator on API 28.

The most interesting bit here is `kvm: true` where we [enable KVM](https://cirrus-ci.org/guide/linux/#kvm-enabled-privileged-containers) for the container to take advantage of native virtualization when running the modern x86 / x86_64 emulators.

We've also given the container the maximum amount of CPU (8) and memory (24G) as the container will be running an Android Emulator instance as well as Gradle commands, both of which are known to be processor and memory intensive.

`create_device_script` and `wait_for_emulator_script` create a new AVD instance, launch an Emulator, and wait until it's fully booted and ready for use. These are not needed in the GitHub workflow as they are taken care of by the [android-emulator-runner Github Action](https://github.com/ReactiveCircus/android-emulator-runner).

We then add the `disable_animations_script` to disable the global animations on the emulator. This is again run by default when using the [android-emulator-runner GitHub Action](https://github.com/ReactiveCircus/android-emulator-runner).

Finally in the `run_instrumented_script` we define the Gradle task for running all Android tests in all modules.

Here's the build summary for **FlowBinding** which has 160 tests across 10 modules:

![Alt Text](https://thepracticaldev.s3.amazonaws.com/i/do9i0tttfj1cty49fzis.png)

The build took **~20 mins** which is almost identical to the GitHub Actions equivalence. Note that there's an extra **1-2 mins** of scheduling period for KVM-enabled containers due to the additional virtualization layer.

### Build matrix

With GitHub Actions we can leverage **build matrix** to run the tests across multiple API levels:

```
jobs:
  test:
    runs-on: macOS-latest
    strategy:
      matrix:
        api-level: [21, 22, 23, 24, 25, 26, 27, 28]
    steps:
    - name: checkout
      uses: actions/checkout@v2

    - name: run tests
      uses: reactivecircus/android-emulator-runner@v2
      with:
        api-level: ${{ matrix.api-level }}
        script: ./gradlew connectedDebugAndroidTest
```

We can do the same with Cirrus CI using [matrix modification](https://cirrus-ci.org/guide/writing-tasks/#matrix-modification):

```
connected_check_task:
  name: Run Android instrumented tests (API $API_LEVEL)
  only_if: $CIRRUS_BRANCH == "master" || CIRRUS_PR != ""
  env:
    matrix:
      - API_LEVEL: 21
      - API_LEVEL: 22
      - API_LEVEL: 23
      - API_LEVEL: 24
      - API_LEVEL: 25
      - API_LEVEL: 26
      - API_LEVEL: 27
      - API_LEVEL: 28
  container:
    image: reactivecircus/android-emulator-${API_LEVEL}:latest
    kvm: true
    cpu: 8
    memory: 24G
...
```

We've defined a matrix of `API_LEVEL` values so the same `connected_check_task` will be executed for each of these values. Note that we are able to specify the name of the docker image dynamically because these images are named in the same `reactivecircus/android-emulator-${API_LEVEL}` pattern. The repository for these images can be found [here](https://github.com/ReactiveCircus/docker-android-images) (all x86 images for API 21 and above are available).

With GitHub Actions you can [exclude](https://help.github.com/en/actions/automating-your-workflow-with-github-actions/workflow-syntax-for-github-actions#example-excluding-configurations-from-a-matrix) some of the configurations generated by the build matrix. This is not currently supported by Cirrus CI.

### Build cache (local)

Many Gradle tasks are [incremental](https://docs.gradle.org/current/userguide/more_about_tasks.html#sec:up_to_date_checks) and [cacheable](https://docs.gradle.org/current/userguide/build_cache.html), which can significantly improve build times.

The local Gradle build cache is located at `~/.gradle/cache` and we are able to get fast incremental builds on CI when tasks outputs are available in this directory.

With **CircleCI** saving and restoring the local Gradle cache directory is easy:

```
jobs:
  build:
    executor: android
    steps:
      - checkout
      - restore_cache:
          key: gradle-{{ checksum "buildSrc/dependencies.gradle" }}-{{ checksum "gradle/wrapper/gradle-wrapper.properties" }}
      - run:
          name: Assemble
          command: ./gradlew assemble
      - save_cache:
          key: gradle-{{ checksum "buildSrc/dependencies.gradle" }}-{{ checksum "gradle/wrapper/gradle-wrapper.properties" }}
          paths:
            - ~/.gradle/cache
```

The cache will only be invalidated when `buildSrc/dependencies.gradle` or `gradle/wrapper/gradle-wrapper.properties` changes which is usually when we update our library dependencies or Gradle version. By doing this we can avoid re-downloading all the dependencies on each build on CI.

**Cirrus CI** also provides a similar [`cache` instruction](https://cirrus-ci.org/guide/writing-tasks/#cache-instruction) that works similarly:

```
connected_check_task:
  name: Run Android instrumented tests
  only_if: $CIRRUS_BRANCH == "master" || CIRRUS_PR != ""
  env:
    API_LEVEL: 28
    TARGET: default
    ARCH: x86_64
  container:
    image: reactivecircus/android-emulator-28:latest
    kvm: true
    cpu: 8
    memory: 24G
  gradle_cache:
    folder: ~/.gradle/caches
    fingerprint_script: cat buildSrc/dependencies.gradle && cat gradle/wrapper/gradle-wrapper.properties
  ...
  cleanup_script:
    - rm -rf ~/.gradle/caches/[0-9].*
    - rm -rf ~/.gradle/caches/transforms-1
    - rm -rf ~/.gradle/caches/journal-1
    - rm -rf ~/.gradle/caches/jars-3/*/buildSrc.jar
    - find ~/.gradle/caches/ -name "*.lock" -type f -delete
```

The Cirrus CI doc recommends using [a technique](https://cirrus-ci.org/examples/#caching) for avoiding re-upload of the cache when the build outputs are unchanged - adding a `cleanup_script` that removes the non-deterministic files generated on each build at the end of the task. This improvement is based on the expectation that re-generating these non-deterministic files and always executing a few tasks on every build has much less impact than uploading hundred of Mb of the cache files on every build.

But there's one area where **CircleCI** performs significantly better - the time taken for downloading and uploading these cache archives.

For roughly **500Mb** of cache archive, with **CircleCI** it only takes about **15 seconds** to [download](https://app.circleci.com/jobs/github/ReactiveCircus/FlowBinding/497) (restore) it, and **25 seconds** to [upload](https://app.circleci.com/jobs/github/ReactiveCircus/FlowBinding/492) it.

With **Cirrus CI**, [downloading](https://cirrus-ci.com/task/5982829204209664) **~500Mb** of cache archive takes nearly **1 minute** while [uploading](https://cirrus-ci.com/task/4999028994998272) takes over **1 minute**.

### Build cache (remote)

Gradle also has built-in support for [HTTP remote build cache](https://docs.gradle.org/current/userguide/build_cache.html#sec:build_cache_configure_remote) which works similarly as a local build cache except the build outputs come from a remote server rather than the local `~/.gradle/cache` directory.

**Cirrus CI** is the first CI service I've used that provides native support for Gradle's [remote HTTP build cache](https://cirrus-ci.org/examples/#build-cache). This can be configured by adding the following to your `settings.gradle` file:

```
ext.isCiServer = System.getenv().containsKey("CIRRUS_CI")
ext.isMasterBranch = System.getenv()["CIRRUS_BRANCH"] == "master"
ext.buildCacheHost = System.getenv().getOrDefault("CIRRUS_HTTP_CACHE_HOST", "localhost:12321")

buildCache {
  local {
    enabled = !isCiServer
  }
  remote(HttpBuildCache) {
    url = "http://${buildCacheHost}/"
    enabled = isCiServer
    push = isMasterBranch
  }
}
```

Note that if your project uses the `buildSrc` directory, the build cache configuration should also be applied to `buildSrc/settings.gradle`. This can be done by putting the build cache configuration above into a separate `gradle/buildCacheSettings.gradle` file and applying it to both `settings.gradle` and `buildSrc/settings.gradle`.

### Optimizing script execution order

Before looking at the final build time improvement, let's take a look at a technique for optimizing your CI pipeline when running instrumented tests on an Emulator.

Here's a visualization of our current `connected_check_task`:

![Alt Text](https://thepracticaldev.s3.amazonaws.com/i/y8hzmtn5t91jajrz4h33.png)

We create a new AVD instance, launch an Emulator in the **background**, **wait until the Emulator is fully booted**, and finally run the `./gradlew connectedDebugAndroidTest` task.

Is there anything else we could be doing while waiting for the Emulator to come online (usually takes about **1 minute**)?

As mentioned earlier many Gradle tasks are **incremental**, this means that once a task has been executed, its `outputs` will be stored in the `build` directory within the sub-project (module) and running the same task again (without changing its `inputs`) won't actually execute the task itself. Instead the `outputs` of the task will be immediately loaded from the `build` directory.

The `assemble<BuildVariant>AndroidTest` task that generates the test APKs is incremental.

Let's first run it locally with no build cache:

```
./gradlew clean assembleDebugAndroidTest --no-build-cache
...
BUILD SUCCESSFUL in 59s
936 actionable tasks: 934 executed, 2 up-to-date
```

The build took about 1 minute with 934 out of 936 tasks executed.

If we run it again without cleaning the build directory:

```
./gradlew assembleDebugAndroidTest
...
BUILD SUCCESSFUL in 3s
914 actionable tasks: 914 up-to-date
```
The build completes in 3 seconds and no tasks were executed as they are all `up-to-date`.

Now since the `connectedDebugAndroidTest` Gradle task we run at the end effectively **depends on** the `assembleDebugAndroidTest` task for generating the test APKs, by first running the `assembleDebugAndroidTest` task explicitly while waiting for the Emulator to boot, the `connectedDebugAndroidTest` task will be significantly faster as all of its sub-tasks would be `up-to-date` at that point.

![Alt Text](https://thepracticaldev.s3.amazonaws.com/i/l8h5sliwbixubn7peiuk.png)

### Final result

Here's the final `.cirrus.yml` file:

```
connected_check_task:
  name: Run Android instrumented tests (API $API_LEVEL)
  timeout_in: 30m
  only_if: $CIRRUS_BRANCH == "master" || CIRRUS_PR != ""
  env:
    matrix:
      - API_LEVEL: 21
      - API_LEVEL: 22
      - API_LEVEL: 23
      - API_LEVEL: 24
      - API_LEVEL: 25
      - API_LEVEL: 26
      - API_LEVEL: 27
      - API_LEVEL: 28
    JAVA_TOOL_OPTIONS: -Xmx6g
    GRADLE_OPTS: -Dorg.gradle.daemon=false -Dkotlin.incremental=false -Dkotlin.compiler.execution.strategy=in-process
  container:
    image: reactivecircus/android-emulator-${API_LEVEL}:latest
    kvm: true
    cpu: 8
    memory: 24G
  create_device_script:
    echo no | avdmanager create avd --force --name "api-${API_LEVEL}" --abi "${TARGET}/${ARCH}" --package "system-images;android-${API_LEVEL};${TARGET};${ARCH}"
  start_emulator_background_script:
    $ANDROID_HOME/emulator/emulator -avd "api-${API_LEVEL}" -no-window -gpu swiftshader_indirect -no-snapshot -noaudio -no-boot-anim -camera-back none
  assemble_instrumented_tests_script:
    ./gradlew assembleDebugAndroidTest
  wait_for_emulator_script:
    adb wait-for-device shell 'while [[ -z $(getprop sys.boot_completed) ]]; do sleep 3; done; input keyevent 82'
  disable_animations_script: |
    adb shell settings put global window_animation_scale 0.0
    adb shell settings put global transition_animation_scale 0.0
    adb shell settings put global animator_duration_scale 0.0
  run_instrumented_tests_script:
    ./gradlew connectedDebugAndroidTest
```

With remote build cache enabled, I managed to cut the build time to **~15 mins** for **FlowBinding**. This is a significant improvement compared to the **~20 mins** build time with **GitHub Actions** (mostly due to the unlimited build cache provided by **Cirrus CI**).

![Alt Text](https://thepracticaldev.s3.amazonaws.com/i/w91e46x7uztwrsawrkz3.png)

## Other Features

After having a positive experience migrating the instrumented tests to run on **Cirrus CI**, I decided to explore some of the other features provided by the service and see how they work in broader Android CI use cases.

### Task dependency

By default all tasks defined in `.cirrus.yml` run in parallel. [Task execution dependencies](https://cirrus-ci.org/guide/writing-tasks/#task-execution-dependencies) can be configured by using the `depends_on` keyword. In the following example, the `publish_artifacts_task` will only be executed after both the `assemble_task` and `connected_check_task` have completed successfully.

```
assemble_task:
  ...

connected_check_task:
  ...

publish_artifacts_task:
  depends_on:
    - assemble
    - connected_check
  deploy_snapshot_script:
    ...
```

### Encrypted secrets

Sometimes a CI task might need access to some secrets. For example, building a release APK may require the encryption key for the release keystore, while publishing a library to Maven Central requires the Sonatype Nexus credentials.

**Cirrus CI** supports [encrypted variables](https://cirrus-ci.org/guide/writing-tasks/#encrypted-variables) which can be safely added to the `.cirrus.yml` file.

To use a secret in a CI task, go to the Cirrus CI settings page and follow the instruction to generate the encrypted variable:

![Alt Text](https://thepracticaldev.s3.amazonaws.com/i/b3gjhq0syrlwgp8dlseu.png)

Then add the generated `ENCRYPTED[xxx]` as a regular environment variable in the `env` block of a task:

```
publish_artifacts_task:
  depends_on:
    - assemble_and_check
    - connected_check
  env:
    SONATYPE_NEXUS_USERNAME: ENCRYPTED[abcd]
    SONATYPE_NEXUS_PASSWORD: ENCRYPTED[1234]
  container:
    image: reactivecircus/android-sdk:latest
  deploy_snapshot_script:
    ./gradlew clean androidSourcesJar androidJavadocsJar uploadArchives --no-daemon --no-parallel
```

### Artifacts

**Cirrus CI** supports storing build artifacts and making them available for download from the dashboard via the [`artifacts` instruction](https://cirrus-ci.org/guide/writing-tasks/#artifacts-instruction). The following example produces the **JUnit XML artifacts** for the `unit_tests_task`:

```  
unit_tests_task:
  container:
    image: reactivecircus/android-sdk:latest
    cpu: 8
    memory: 24G
  unit_test_script:
    ./gradlew test
  always:
    junit_artifacts:
      path: "**/test-results/**/*.xml"
      type: text/xml
      format: junit
```

### Conditionally skipping builds

Sometimes you may want to skip a CI task if the there's no change in the source code (e.g. changing the README.md file). For example we might want to **only** run the `connected_check_task` if there are changes to the Kotlin, XML or Gradle sources in the commit or pull request. This can be implemented with the [`skip` keyword](https://cirrus-ci.org/guide/writing-tasks/#conditional-task-execution) and the [`changesInclude` function](https://cirrus-ci.org/guide/writing-tasks/#supported-functions):

```
connected_check_task:
  only_if: $CIRRUS_BRANCH == "master" || CIRRUS_PR != ""
  skip: "!changesInclude('.cirrus.yml', '*.gradle', '*.gradle.kts', '**/*.gradle', '**/*.gradle.kts', '*.properties', '**/*.properties', '**/*.kt', '**/*.xml')"
  ...
```

![Alt Text](https://thepracticaldev.s3.amazonaws.com/i/2hz82h65lt279ye47u8k.png)

Note that this is also supported by GitHub Actions via the [path filters](https://help.github.com/en/actions/automating-your-workflow-with-github-actions/workflow-syntax-for-github-actions#onpushpull_requestpaths).

### Manual trigger

A task can also be triggered **manually** from the Cirrus CI dashboard or the GitHub Checks page. This is often useful in deployment (e.g. publishing a new APK) where we need to be able to control when to trigger the task.

[Manual task](https://cirrus-ci.org/guide/writing-tasks/#manual-tasks) can be configured by adding `trigger_type: manual` to the task:

```
publish_artifacts_task:
  trigger_type: manual
  depends_on:
    - assemble_and_check
    - connected_check
  ...
```

![Alt Text](https://thepracticaldev.s3.amazonaws.com/i/uaolcs0voeh63lf29aks.png)

![Alt Text](https://thepracticaldev.s3.amazonaws.com/i/t1m9gvdulchu2iztcas6.png)

### Delayed execution - Required PR Labels

Tasks such as instrumented tests require special execution environment and have a longer feedback cycle, which make them more expensive to run. Therefore when a new PR is created we might want to run the fast checks (assemble and unit tests) immediately, and only trigger the expensive checks after an initial review.

Cirrus CI provides a feature called [Required PR Labels](https://cirrus-ci.org/guide/writing-tasks/#required-pr-labels). Tasks can specify a list of `required_pr_labels` to only get triggered once these labels have been added to the pull request.

In the following example the `connected_check_task` will only be run after the `initial-review` label has been added to the pull request.

```
connected_check_task:
  required_pr_labels: initial-review
  only_if: $CIRRUS_BRANCH == "master" || CIRRUS_PR != ""
  ...
```

On the GitHub Checks page, tasks which are waiting for PR labels are considered "neutral":

![Alt Text](https://thepracticaldev.s3.amazonaws.com/i/exmioeeku26lo5twp1zp.png)

To trigger these tasks, apply the required `initial-review` label to the PR:

![Alt Text](https://thepracticaldev.s3.amazonaws.com/i/e63jaqfr1rvu5ipq4y0a.png)

The `connected_check_task` will now be triggered.

### Build notification

A major feature missing from Cirrus CI is notification. When a build fails, the status will be reflected on the GitHub Checks page, but Cirrus CI doesn't have built-in mechanism for sending an Email notification.

This can be worked around by adding a [GitHub Action](https://github.com/cirrus-actions/email) that sends emails when a GitHub Check Suite completes, but you'll need to provide your own SMTP server information.

## Cirrus CI Android Templates

If you've made it this far, chances are you want to try out Cirrus CI for your own projects. To help you get started, I've created some [cirrusci-android-templates](https://github.com/ReactiveCircus/cirrusci-android-templates) for common use cases such as building APKs, running instrumented tests, and publishing library artifacts.

## Summary

Overall I have been really impressed with **Cirrus CI**. I was initially attracted by the KVM-enabled containers for running instrumented tests. Not only was I able to migrate FlowBinding to use [Cirrus CI for running instrumented tests](https://github.com/ReactiveCircus/FlowBinding/pull/59), it was also a lot of fun discovering and trying a few other cool features along the way.

It's clear that the Cirrus CI Team understands the unique challenges when it comes to doing CI for Android. Having used quite a few CI solutions for Android including Jenkins, Buddybuild, Buildkite, CircleCI, Bitrise, and GitHub Actions, I'd like to highlight some of the key competitive advantages Cirrus CI brings to the table and why you might want to consider using it for Android:

- Excellent documentations and examples
- Completely free for opensource projects with very generous concurrency limit
- KVM-enabled containers for running hardware-accelerated Emulators
- Powerful VMs - up to 8.0 CPUs and 24 GB of RAM for Linux containers
- Built-in support for Gradle build cache (both local and remote)
- First-class  GitHub integration with features such as [Required PR Labels](https://cirrus-ci.org/guide/writing-tasks/#required-pr-labels)

There are also a few things that could use some improvements:

- Being a smaller player in the market means less efforts have been put into marketing, branding, and UX of the product especially the build dashboard
- Downloading and uploading cache archives takes noticeably longer than some of the other services
- Currently only GitHub is supported, although [BitBucket support is planed](https://github.com/cirruslabs/cirrus-ci-docs/issues/9)
- There's no built-in support for Email notifications

None of these was a deal breaker for running instrumented tests. But due to the slower download and upload speeds for build cache, I have yet to migrate the rest my CI pipelines from **CircleCI**.

## What's Next

**Cirrus Lab** (the team behind Cirrus CI) recently announced [Cirrus Emulators](https://www.cirrus-emulators.com/), a cloud service that provides hardware-accelerated Android Emulators which can be integrated with any CI services using a CLI. I'll share my experience with it once the product is publicly available.

Thanks for reading!

-------------------------

Thanks to [Fedor Korotkov](https://twitter.com/fedor) for the review.