Running Instrumented Tests on Github Actions with Firebase Test Lab
Published: 2023-05-08 19:21:23

  
Hello guys welcome back to another article, In the previous article we learnt about [generating UI state 
in android](https://medium.com/@peterkagure/generating-ui-state-in-android-for-jetpack-compose-using-sealedx-241b52fd5d95).
In today's article we will learn how to run instrumentation tests on GitHub actions via
Firebase Test Lab


Normally when we set up CI/CD pipelines we usually follow 3 general steps
namely: Build,Test and Deploy.
In android development we have different type of tests
* Unit Test
* Instrumented Test 

Unit Test usually run on the JVM and only test a single portion of your code 
i.e a class or a function

Instrumentation tests usually run on an android device (Physical or Emulator).These
tests usually need specify android components to run hence the need for an android 
device

On a CI/CD platform like github actions running unit tests is usually straight foward since 
you just setup the JDK and run the gradle command as shown below 

```
on:
  push:
    branches: [ main ]
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Set up JDK
        uses: actions/setup-java@v1
        with:
          java-version: 11
      - name: Cache gradle
        uses: actions/cache@v1
        with:
          path: ~/.gradle/caches
          key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*') }}
          restore-keys: |
             ${{ runner.os }}-gradle-    

      - name: Run Unit Tests with Gradle
        run: ./gradlew test

```

But when it comes to instrumented test it is a bit tricky because we don't have a android emulator 
on our CI server.While there are a few solution to this like using this 
[action](https://github.com/marketplace/actions/android-emulator-runner) on a macOS runner 

The solution I am about share makes it so easy 

## In comes Firebase Test Lab 
As per the [description](https://firebase.google.com/docs/test-lab) 
, Firebase Test Lab is a cloud-based app testing infrastructure that lets you test your app on a range of devices and configurations,
so you can get a better idea of how it'll perform in the hands of live users.

Assuming you have an android project in which you have instrumented tests already written you can proceed

### 1. Setup a firebase project
For this you need to set up you android app with a firebase project 
For more info on how to get started [check here](https://firebase.google.com/docs/android/setup)

### 2. Set up your firebase Test Lab Service Account 
For use to use the firebase test lab services we need a service account file to help the service 
authenticate us.
To do that we need to go to the Google cloud console 
[here](https://console.cloud.google.com/) and create a service account (Every firebase project is 
essentially a Google cloud project). To create a service account to be used in a CI system 
check [here](https://firebase.google.com/docs/test-lab/android/continuous) (In the example above they use Jenkins
but the same can be used with github actions)

### 3. Build your androidTest Apk 
Once you have defined a device configuration, you should now build and androidTest apk. The androidTest apk 
packages all of our tests defined in the androidTest source set into an apk that will be used to 
execute the tests. The androidTest apk is usually upload to firebase test lab and the apk is executed 
in the cloud.

### 4. Define a device configuration 
For use to execute the apk we need a device. We need to specify some device configuration to test firebase test lab
what kind of device we want our apk to run on. Note that you can run your tests on multiple different types 
of devices just to make sure you app runs correctly in different devices

To define a device configuration we create a YAML file in the root of our project
e.g `android-device.yml`

In there we add the following 
```angular2html
android-pixel-4:
  type: instrumentation
  app: app/build/outputs/apk/debug/app-debug.apk
  test: app/build/outputs/apk/androidTest/debug/app-debug-androidTest.apk
  device:
    - model: redfin
      version: 30
      locale: 'en'
      orientation: portrait

```
In the above example we create a device configuration called `android-pixel-4` with the following settings :
* The `app`- key are a path to the actual debug apk 
* `Test` - Path to the android test apk 
* `device` - Device specific settings
    * Model - Device ID of a device supported on firebase test lab. For more device options check [here](https://firebase.google.com/docs/test-lab/android/available-testing-devices)
    * Version - Version of the android OS 
    * Locale - 
    * Orientation - Orientation of the device when the tests run


### 5 : Run our tests 
Now we are done with the setup, let's run our tests
To run our tests on our workflow add the following to your workflow file
We use this [action](https://github.com/marketplace/actions/firebase-test-lab-action)
to access the firebase test lab APIs in our CI environment

```
      - name: Build App with Gradle
        run: ./gradlew build

      - name: Build Android Test APK
        run: ./gradlew assembleAndroidTest

      - name: Run Android tests on Firebase Test Lab
        uses: asadmansr/Firebase-Test-Lab-Action@v1.0
        with:
          arg-spec: 'android-device.yml:android-pixel-4'
        env:
          SERVICE_ACCOUNT: ${{ secrets.FIREBASE_TEST_LAB_SERVICE_ACCOUNT }}

```

The arg-spec specifies the device we defined in the `android-device.yml` appended with the device
settings. We then put the service account in our GitHub action secrets and reference it in the file

If all is good we should see our test results in the firebase console like this 

![Firebase Test Lab console](https://res.cloudinary.com/dhuqr5iyw/image/upload/v1683572040/blog-posts/firebase-test-lab_koqyjg.jpg)


## Troubleshooting

If for some reason you workflow fails at the firebase test lab stage, check the log messages, they can help in 
deciphering the problem but here are a few possible causes :

* You wrongly configured the service account i.e. you gave the wrong permissions when creating the service account. 
  Go over the instructions again
* You probably defined a wrong path to your apks
* The tests probably were uploaded but failed, Check your tests 
* You passed a model id for your device configuration


I hope you liked this article and I hope it helped you run your instrumentation test more reliably on your CI workflow

For the example project I used here you can find it on GitHub 
[here](https://github.com/chege4179/ExpenseTrackerApp)

You can also follow me on [github](https://github.com/chege4179) and [twitter](https://twitter.com/peter__me)


See you next time 👋👋