100% Flakiness-free UI test automation with Kaspresso and Allure TestOps
Tags: android, testing, devops
Published: 2023-11-22 08:17:00

> This article was written by Nihal Agazade and originally posted to the [Codemagic blog](https://blog.codemagic.io/100-flakiness-free-ui-test-automation-with-kaspresso-and-alluretestops/).

UI testing in mobile app development is a challenging task due to multiple factors such as flaky tests, saving screenshots, printing useful logs and readability of test codes. We will be talking about a new UI testing framework Kaspresso for Android apps and how it is different from other frameworks when it comes down to solving these challenges.

## What is Kaspresso and why should it be chosen?

Kaspresso is based on Espresso and Kakao DSL which helps it have a great readability of test codes and comes with many built-in features:

1. Kakao DSL based test code readability. For our better understanding, let’s quickly check how Kaspresso Kakao DSL wrappers differ from Espresso:

Espresso:

```
@Test
fun LoginActivity() {
    onView(withId(R.id.login_button))
        .check(ViewAssertions.matches(
               ViewMatchers.withEffectiveVisibility(
                       ViewMatchers.Visibility.VISIBLE)))
    onView(withId(R.id.login_button)).perform(click())
}

```
Kaspresso:

```
@Test
fun LoginActivity() {
    MainScreen {
        LoginButton {
            isVisible()
            click()
        }
    }
}
```

2. 100% flaky test protection, thanks to its built-in protection rules.
3. Allure Test Ops support

To learn more about what else Kasspresso offers, feel free to check the [official documentation](https://kasperskylab.github.io/Kaspresso/).

## How to get started with Kaspresso?

1. First and foremost, `mavenCentral()` needs to be declared inside the root `build.gradle`. Make sure that `jcenter()` is removed from the project along with the dependencies that require `jcenter()`.

> Note: Recent native android projects have it defined inside settings.gradle instead:

```
pluginManagement {
    repositories {
        gradlePluginPortal()
        google()
        mavenCentral()
    }
}
```

2. Open app/build.gradle and integrate Kaspresso into your project by adding the following line:

```
dependencies {
    androidTestImplementation("com.kaspersky.android-components:kaspresso:1.5.3")
}
```

By this, we are good to go with Kaspresso unless Allure Testops needs to be integrated which requires additional steps to be followed, and we will be talking about it in the next section.

We briefly talked about what features Kaspresso introduces and what makes Kaspresso different from other frameworks. Let’s go through some of the key features for further clarification.

## Painful flaky tests

Test flakiness is something we can hardly avoid considering that our code constantly changes, adding new screens, new functionalities etc. There are multiple reasons that could be the actual cause of flaky tests, such as:

1. Real devices or emulators
2. Test code
3. Production code
4. General infrastructure
5. Or even instability with the testing framework you are using

Kaspresso has it reduced to the very minimum with its DSL wrappers and Interceptors. Interceptors are built-in and help achieve many useful things, e.g:

- add custom actions to each framework operation like writing a log or taking a screenshot;
- overcome flaky operations by re-running failed actions, scrolling the parent layout or closing the android system dialog;

For an example, `ScreenshotStepWatcherInterceptor` is utilised to take pictures whether tests fail or pass and it is part of `TestCase()` constructor:

```
class SimpleLoginActivity : TestCase(
    kaspressoBuilder = Kaspresso.Builder.simple().apply {
        stepWatcherInterceptors.add(ScreenshotStepWatcherInterceptor(screenshots))
    }
)
```

`TestCase()` is a class inherited from `com.kaspersky.kaspresso.testcases.api.testcase.TestCase` which is out of the box ready for use.

## Screenshots

Kaspresso has made the process of taking screenshots much easier via its interceptors. It is possible to take snapshots of every screen while tests are executed and save these screenshots under a desired file name through the `device.screenshots.take("desired_file_name")` method:

```
   @Before
   @Test
   fun test() {
       run {
           step ("Products dropdown is available") {
           device.screenshots.take("screenshot_file_name")
           ProductsScreen {
               testActivityButton {
                   isVisible()
                   isClickable()
               }
             }
           }
       }
   }
```

Permissions must be granted for screenshots to be taken by importing `androidx.test.rule.GrantPermissionRule`, otherwise there will be no `READ_EXTERNAL_STORAGE` nor `WRITE_EXTERNAL_STORAGE` in place. So, at the top of the Kotlin Class/File where the kotlin test code is stored, `GrantPermissionRuleneeds` needs to be imported:

```
import androidx.test.rule.GrantPermissionRule
```

and then, within the same file, `@Rule` must be declared which is inherited from `org.junit.Rule`:

```
@get:Rule
    val grantPermissionRule: GrantPermissionRule = GrantPermissionRule.grant(
        android.Manifest.permission.READ_EXTERNAL_STORAGE,
        android.Manifest.permission.WRITE_EXTERNAL_STORAGE
    )
```
Besides these changes, `AndroidManifest.xml` needs to be configured for a complete permission cycle:

```
<manifest>
    <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
    <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" /> 
    <application>
    </application>
</manifest>
```
Kaspresso `device.screenshots.take` object is similar to `device.uiDevice.takeScreenshotfrom` the `uiautomator` library, however the key difference would be custom file names that can be easily managed through `device.screenshots.take`, especially when there are lots of snapshots and it can be tricky to find the appropriate ones.

> Note: A video mp4 file is saved to `sdcard/Documents` along with screenshots and you can find an exact file path in the logs which is explained in the following section.


## Logging

Custom notes can be added to the logs, and for that purpose `testLoggerobject` can be used with the `i` method:

```
fun test() {
       run {
           testLogger.i("Either hardcoded or dynamic values through env vars $VALUE")
           MainScreen {}
```

However, there is another challenge at this point and it is about splitting test internals into different steps, so that which tests fail and which ones pass can be observed easier.

What we can do is to wrap our code in `try catch` block along with adding manual logs to the output. Something like:

```
@Test
    fun sampleTest() {
        try {
            Log.i("Step 1:")
            MainScreen {
                loginActivityButton {
                    isVisible()
                }
            }
            Log.i("Step 1: Succeeded")
        } catch (e: Throwable) {
            Log.i("Step 1: Failed")
            throw e
        }
     anotherTest(){
      try {
       ...
       Log.i("Step 1:")
     } catch(e: Throwable){
      Log.i("Step 2: Failed")
    }
}
```

Works! Though with all the `try catch` blocks, the code will lose its readability and potentially, lead to flakiness.

In the Kaspresso world, Steps can be added in order to simplify code readability and more importantly, by keeping its flaky reduction rules. Steps can be used by declaring the `run{}` method:

```
fun test() {
       run {
           step ("Testing user registration auth") { ... }
           step ("Testing user login auth") { ... }
           ...
       }
   }
```

and then we can observe our crafted logs:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/neh2lzyx6q7kuncq5xpy.jpeg)

## Complete automation with Codemagic CI/CD and uploading results to Allure Testops

We can further automate the process and upload test results to Allure Testops.

Thanks to Codemagic, we can automatically run UI tests, collect logs, and deploy results to Allure Testops on every change or daily.

Allure Testops is a DevOps ready test reporting platform where we can analyse test results through graphical reports and gathering results from servers in real-time while the build job is being executed. In order to achieve it with Codemagic, we will be utilising `allurectl` which is a command line wrapper of Allure Testops API. Allure Testops functions in two modes:

1. CI mode
2. Non-CI mode

`allurectl` detects automatically which one is in use through finding the required environment variables specified in Codemagic as explained below. So, let’s get the steps together for an easy flow of the process:

1. Sign up with Allure Testops.
2. Go to your profile, and create a secret token

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/y6a5xkhppgp5ry014c83.png)

3. As soon as your request for registration is approved by the Allure Testops team, you will receive an active endpoint to your work environment.
4. Create a project and fetch its ID.

The very vital step not to miss is to enable Allure support in our test code via withForcedAllureSupport. In order to enable it, we need to go through a couple of steps in our code:

1. Import `com.kaspersky.components.alluresupport.withForcedAllureSupport`
2. Add withForcedAllureSupportto TestCase() constructor which will look like:

```
class YOUR_ACTIVITY_TEST : TestCase(
    kaspressoBuilder = Kaspresso.Builder.withForcedAllureSupport()
)
```
After this, as soon as our tests are executed, there will be a directory created (`sdcard/Documents/allure-results`) and they will be pulled out of the device and uploaded to Allure Testops by Codemagic.

If you have not signed up with Codemagic and yet added your repo, you can follow the steps as explained in the Codemagic documentation [here](https://docs.codemagic.io/getting-started/adding-apps/).

1. Go to your repo settings in the UI and you will find the Environment variables tab right next to `codemagic.yaml` and add your Allure secrets that we went through how to create them above.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fvm8pptcl4abdt5ul3wh.jpeg)

2. Add the following lines in `codemagic.yaml`:

```
workflows:
  run-android-ui-test:
    name: Upload test results to Allue Testops
    instance_type: linux
    environment:
      groups:
        - allure_credentials
    scripts:
      - name: Set Android SDK location
        script: |
                        echo "sdk.dir=$ANDROID_SDK_ROOT" > "$CM_BUILD_DIR/local.properties"
      - name: Launch Emulator
        script: |
            cd $ANDROID_HOME/tools
            emulator -avd emulator &
            adb wait-for-device            
      - name: Install Allure CLI
        script: | 
            set -e
       wget https://github.com/allure-framework/allurectl/releases/download/2.2.1/allurectl_linux_amd64 -O ./allurectl
            chmod +x ./allurectl
      - name: Execute tests
        script: |
                        ./gradlew :app:connectedDebugAndroidTest
      - name: Pull test files from the device and upload to Allure Testops
        script: |
            set -e
            adb pull sdcard/Documents/allure-results
            ./allurectl upload allure-results
```

> Note: When installing Allure CLI and building with a Mac machine, the platform must be changed in the command line to `allurectl_darwin_amd64` for Mac Pro machines. In order to switch the machine type from Linux to Mac Pro in codemagic.yaml, changing `instance_type` value to `mac_pro` suffices.

> Note: As the Apple Virtualization Framework does not support nested virtualization, Android emulators are not available with Apple silicon (M1 or M2) machines.

From this point on, Codemagic will start its automation process and print out the build/test logs:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/l06rars1zrfx1yyhkhkh.jpeg)

A link to the Allure Testops environment where the test results can be accessed will be printed in the logs:


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zi5vhxtkvo24udailix0.jpeg)

Worth pointing out that Codemagic has a possibility to schedule builds. It means that your tests will be run at a certain time you set up and you will already have had the test results in hand, and ready for review in Allue Testops. Let’s see how we can schedule a build:

1. Open your application in Codemagic.
2. Switch to the Scheduled builds tab and click the Add new schedule button.
3. Select the Branch and the Workflow to run.
4. In the Schedule for field, select the days you want to run the build.
5. Specify the start time (UTC) of the build by selecting a value from the At field. Note that the build may be delayed up to 15 minutes during peak hours.
6. Click Add schedule to save the schedule.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hg5oely3ei3ol9db4vne.jpeg)

## Checking test results in Allure Testops

Now, we can observe screenshots, videos, and customised test logs in the Allure Testops UI along with a graphical report of the test results.

Here Allure UI indicates how many tests passed, which ones failed:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hploue3uw66z86f34yzx.jpeg)

In the screenshot below, we will find our passing tests along with the test logs, screenshots, and videos:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hfcsnb9vfbfwh749l1ai.jpeg)

You will also have a chance to check failing test logs and which steps the issues occurred with:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fqjxl5lztlje6g3fnifc.jpeg)

## Summary
We have gone through the steps regarding how Kaspresso manages to overcome the key issues of UI testing environments by allowing us to automate Android UI tests with flakiness protection and providing test results to Allure Testops platform to better control our manual, and automated testing through graphical analysis of test results including screenshots, videos, and all the logs. Codemagic took the code over, added an extra layer of automation, and completed the whole cycle of the automation process.