Bytecode Transformations: The Android Gradle Plugin
Tags: android, java, mobile
Published: 2021-12-14 23:05:18

By: [Roman Zavarnitsyn](https://github.com/romtsn/)

_This is the first part of a blog post series about bytecode transformations on Android. In this part we’ll cover different approaches to bytecode manipulation in Java as well as how to make it work with Android and the Android Gradle plugin. In the next two parts we’ll dive into the actual bytecode, bytecode instructions and how we can modify the bytecode and inject our own instructions, using [Room](https://developer.android.com/jetpack/androidx/releases/room) as an example. In the last part we’ll see how we can test our transformations and how it can influence Gradle build speed._

Detecting the slow spots in your app without having to write a single line of code is an intriguing idea, but is also not that easy to implement. At Sentry we wanted to provide the capability for Android users to automatically measure the execution time of database queries because oftentimes they can become a hidden bottleneck for app performance. [Room](https://developer.android.com/jetpack/androidx/releases/room) is a widely-adopted ORM solution built by Google and is a go-to library for persistence for the majority of the Android developers, so it was an obvious choice for us to start with.

If we want database operations to show up in Sentry, we need to wrap them into special objects called Spans. In short, Spans are application events that have a start and end time and some metadata like operation name, description, etc. For example, this is how the performance breakdown looks for a popular open-source app [tivi](https://github.com/chrisbanes/tivi):

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ja26ew7bbptsgaldx5bo.png)

On the screenshot above, there’s a parent span `ui.screen.interaction` that contains multiple child spans, for instance, the network request span with a `http.client` operation, duration 225ms and a request url as a description. If you want to learn more about performance @ Sentry, check [this post](https://blog.sentry.io/2021/08/23/mobile-vitals-four-metrics-every-mobile-developer-should-care-about#mobile-performance-monitoring) out.

Back to the topic, this all means we need to find a way to inject our code before and after Room executes its queries to measure their execution time. There’s a [QueryCallback](https://developer.android.com/reference/androidx/room/RoomDatabase.QueryCallback) available in the Room API, but it’s invoked only before a query gets executed, so we couldn’t really utilize it.

#Options#

Since there’s no way to know when a SQL query starts and finishes at runtime, we started looking into compile-time solutions. There are a few well-known options in the JVM world available for bytecode weaving:

  * [AspectJ](https://en.wikipedia.org/wiki/AspectJ): an AOP framework, which allows extending methods and plugging into their execution from outside of the target codebase.

  * [ASM](https://asm.ow2.io/): a bytecode manipulation framework, which allows dealing with bytecode directly. For example, it’s used by R8 and D8 on Android for [optimizing and dexing](https://jakewharton.com/digging-into-d8-and-r8/) the bytecode.

  * Other higher-level abstractions like [Javassist](https://www.javassist.org/): are all based on ASM, but have a nicer and easier-to-understand APIs to deal with.

While it would be logical to pick something higher-level, considering we had no expertise in any of those, we’ve decided to look into how we could marry those with the Android Gradle plugin (AGP), as we are aiming to transform Android apps and need to support things like differe build types, flavours and so on. A quick search revealed that we could go with:

  1. [Gradle’s TransformAction](https://docs.gradle.org/current/dsl/org.gradle.api.artifacts.transform.TransformAction.html): a plain Gradle API for transforming outputs. This is used, for example, for [dexing](https://cs.android.com/android-studio/platform/tools/base/+/mirror-goog-studio-main:build-system/gradle-core/src/main/java/com/android/build/gradle/internal/dependency/DexingTransform.kt), [jetifying](https://cs.android.com/android-studio/platform/tools/base/+/mirror-goog-studio-main:build-system/gradle-core/src/main/java/com/android/build/gradle/internal/dependency/JetifyTransform.kt), and many other things that the Android Gradle plugin does.

  2. [AGP’s Transform](https://developer.android.com/reference/tools/gradle-api/7.0/com/android/build/api/transform/Transform): an old API from AGP that gives a list of inputs to be transformed, depending on the options provided. It also handles full/incremental builds automatically. Now deprecated in favor of the new API.

  3. [AGP’s transformClassesWith](https://developer.android.com/reference/tools/gradle-api/7.0/com/android/build/api/variant/Component.html#transformclasseswith): the new API from AGP that allows registering an ASM [ClassVisitor](https://asm.ow2.io/javadoc/org/objectweb/asm/ClassVisitor.html) for visiting bytecode instructions and instrumenting `.class` files. It utilizes the aforementioned [TransformAction](https://docs.gradle.org/current/dsl/org.gradle.api.artifacts.transform.TransformAction.html) to transform dependencies and provides a Gradle task that handles full/incremental builds automatically. Available from AGP version 4.2.0 and above.

The first option would require us to manually hook into the AGP process and deal with its artifacts, so we decided to look into options 2 and 3 and compare them, as they come directly from the vendor.

Previously, in the old AGP versions (pre-4.2.0), if one would like to instrument compiled classes, they would need to register their own [Transform](https://developer.android.com/reference/tools/gradle-api/7.0/com/android/build/api/transform/Transform), traverse the input files and perform instrumentation for each of those files using [ClassWriter](https://asm.ow2.io/javadoc/org/objectweb/asm/ClassWriter.html) from ASM. For each such `Transform` AGP would register a new Gradle task, so if you happen to have 10 transforms instrumenting your application, you would end up with 10 additional Gradle tasks doing almost the same thing - iterating over the changed files, reading the bytecode, applying their own transformations and writing the bytecode back.

This is horrible for the build speed and most of that can be commonized up until the point of actually instrumenting the bytecode.

The new `transformClassesWith` API tackles exactly that by providing a single API for registering `ClassVisitors` and abstracting away file iteration and reading/writing the bytecode. It collects all visitors in a single list and then, for each file, runs all of them in order of registering, meaning there’s just a single Gradle task running for all transformations.

We’ve decided to go with ASM + `transformClassesWith` pack, deliberately supporting only the new versions of AGP.

_Note, that if you want to support bytecode transformations in lower AGP versions (below 4.2.0) you still need to use the old Transform API. However, you can perform an AGP version check at runtime and choose either a new or an old API depending on it. An example can be seen in the [Hilt](https://cs.android.com/android/platform/superproject/+/master:external/dagger2/java/dagger/hilt/android/plugin/src/main/kotlin/dagger/hilt/android/plugin/HiltGradlePlugin.kt;l=69-82) Gradle plugin._

#Using new transform APIs#

##**Registering AsmClassVisitorFactory**##

As this post is not about how to create Gradle plugins, I will skip the setup part, but in a nutshell, we have to implement the `Plugin` interface from Gradle and override a single method called `apply`, which is called when the Gradle plugin is applied to a project:

```kotlin
class SentryPlugin : Plugin<Project> {

	override fun apply(project: Project) {
		...
	}
}
```

After that we have to listen when the Android Gradle plugin is applied to the project and retrieve the new [AndroidComponentsExtension](https://developer.android.com/reference/tools/gradle-api/4.2/com/android/build/api/extension/AndroidComponentsExtension) like this:

```kotlin
project.pluginManager.withPlugin("com.android.application") {
	val androidComponentsExtension =
    project.extensions.getByType(AndroidComponentsExtension::class.java)
	...
}
```

The extension has a special `onVariants` method that configures the build variants:

```kotlin
androidComponentsExtension.onVariants { variant ->
	...
}
```

Finally we can register our custom `AsmClassVisitorFactory` for the `variant` through `transformClassesWith`:

```kotlin
variant.transformClassesWith(
  SpanAddingClassVisitorFactory::class.java,
  InstrumentationScope.ALL
) { params ->
  if (extension.tracingInstrumentation.forceInstrumentDependencies.get()) {
    params.invalidate.setDisallowChanges(System.currentTimeMillis())
  }
  params.debug.setDisallowChanges(
    extension.tracingInstrumentation.debug.get()
  )
  params.tmpDir.set(tmpDir)
}
```

`transformClassesWith` accepts 3 parameters:

  * `ClassVisitorFactory`: a factory, which provides a `ClassVisitor` implementation and defines whether this visitor is interested in instrumenting a given class

  * `InstrumentationScope`: either `ALL` or `PROJECT`. Defines whether the instrumentation applies only for project files or for project files and their dependencies (e.g. jars). In our case, we were interested in instrumenting all Room queries, regardless of their origin, so we set it to `ALL`

_If you’re using `InsrumentationScope.ALL`, beware that Gradle will cache the transformed artifacts across builds as long as the `InstrumentationParameters` do not change. This may come as a surprise while developing, as some of the classes coming from the dependencies might not show up for instrumentation. We found it useful to have a boolean parameter, which would invalidate the transform caches by simply setting `System.currentTimeMillis` and allow us to always receive all classes for instrumentation._

  * Configuration function to be applied before passing the necessary parameters for the `ClassVisitorFactory`

The [InstrumentationParameters](https://developer.android.com/reference/tools/gradle-api/7.0/com/android/build/api/instrumentation/InstrumentationParameters) are the way to pass information from the plugin to the `ClassVisitorFactory`. They are being used as Gradle inputs, this means they contribute to the up-to-date checks of the task and should be properly [annotated](https://docs.gradle.org/current/userguide/more_about_tasks.html#sec:up_to_date_checks). For example, here we are setting a `debug` boolean as well as a `tmpDir` to use this information later and stream debug output of instrumentation into a file under the `tmpDir`.

##**Implementing AsmClassVisitorFactory**##

For the `ClassVisitorFactory` it’s necessary to implement 2 methods:

  * `createClassVisitor` which provides a custom `ClassVisitor` from ASM that does the actual visiting of bytecode instructions and transformation

  * `isInstrumentable` which defines whether a given class is applicable for instrumentation or not

It is also necessary to specify an implementation of `InstrumentationParameters` as a type for `AsmVisitorFactory` or use `InstrumentationParameters.None` in case there are no params.

```kotlin
abstract class SpanAddingClassVisitorFactory :
    AsmClassVisitorFactory<SpanAddingClassVisitorFactory.SpanAddingParameters> {

    interface SpanAddingParameters : InstrumentationParameters {

        @get:Input
        @get:Optional
        val invalidate: Property<Long>

        @get:Input
        val debug: Property<Boolean>

        @get:Internal
        val tmpDir: Property<File>
    }

    override fun createClassVisitor(
        classContext: ClassContext,
        nextClassVisitor: ClassVisitor
    ): ClassVisitor =
        TODO("If we return true from the isInstrumentable below, we should return a ClassVisitor that will inject our code for measuring the execution time")

    override fun isInstrumentable(classData: ClassData): Boolean =
        TODO("Determine if we are interested in instrumenting the given ClassData. For us it would mean a class annotated with @Dao")
}
```

Inside the `isInstrumentable` method, we determine whether we are interested in instrumenting the given `ClassData` and later return our custom `ClassVisitor` from the `createClassVisitor` method in case we are. Note, however, that it’s always a good practice to fall back to `nextClassVisitor` in case there’s no `ClassVisitor` for the given class, otherwise the Gradle build will fail.

Last, let’s look at the `ClassData` structure:

```kotlin
interface ClassData {
    /**
     * Fully qualified name of the class.
     */
    val className: String

    /**
     * List of the annotations the class has.
     */
    val classAnnotations: List<String>

    /**
     * List of all the interfaces that this class or a superclass of this class implements.
     */
    val interfaces: List<String>

    /**
     * List of all the super classes that this class or a super class of this class extends.
     */
    val superClasses: List<String>
}
```

It may seem to have everything to help us determine whether a class is suitable for instrumentation or not, but there’s a setback which we’ll cover in the next post.


Using the new AGP transform APIs with ASM looks like an obvious choice for bytecode manipulation for Android as it affects the build speed almost unnoticeable (we’ll cover that later), handles full/incremental builds on its own, and offers a great API surface via ASM at the same time.

In the next part, we’ll talk about Room internals, how we collected the methods for instrumentation, and what tools are available out there for dealing with ASM.

The code is available in the [sentry-android-grade-plugin](https://github.com/getsentry/sentry-android-gradle-plugin) repo, specifically:

  * [Registering a ClassVisitorFactory](https://github.com/getsentry/sentry-android-gradle-plugin/blob/37eff955853fd395c0c6bf4c633518ce35dd508c/plugin-build/src/main/kotlin/io/sentry/android/gradle/SentryPlugin.kt#L62-L86)

  * [ClassVisitorFactory](https://github.com/getsentry/sentry-android-gradle-plugin/blob/37eff955853fd395c0c6bf4c633518ce35dd508c/plugin-build/src/main/kotlin/io/sentry/android/gradle/instrumentation/SpanAddingClassVisitorFactory.kt)

By the way, if you are already using the [Sentry Android Gradle plugin](https://docs.sentry.io/platforms/android/proguard/#gradle-configuration), give this new Room instrumentation a try in version `3.0.0-beta.1` , we would appreciate your feedback via [GitHub issues](https://github.com/getsentry/sentry-android-gradle-plugin/issues). If not, it’s time to start using Sentry — [request a demo](https://sentry.io/demo) and [try it out for free](https://sentry.io/signup/).