Gradle's leaky abstractions: Declarative(ish) shell, imperative core: Implementing a safe(ish) global configuration DSL
Tags: gradle, java, kotlin
Published: 2024-03-23 19:20:30

Gradle is [very](https://github.com/gradle/declarative-gradle/tree/main/unified-prototype) [aware](https://blog.gradle.org/declarative-gradle) [they have a complexity problem](https://blog.jetbrains.com/blog/2023/11/09/amper-improving-the-build-tooling-user-experience/). Fundamentally, the problem is that Gradle build scripts use an Actual Programming Language (either Groovy or Kotlin), and therefore provide users access to the complete Java/Groovy/Kotlin ecosystem—the JDK, the standard libraries, and all the other libraries too.

Gradle wants you to write your scripts like this:

```kotlin
// build.gradle.kts
// so declarative!
plugins {
  id("foo")
}

dependencies {
  implementation(libs.coolThing)
}

myExtension { ... }
```

but what that boils down to is just this:

```kotlin
// er, imperative?
project.pluginManager.apply("foo")

project.dependencies.add("implementation", libs.coolThing)

project.extensions.getByType(MyExtension::class.java).run { ... }
```

which in turn is this:

```kotlin
// yeah, definitely imperative
FooPlugin().apply(project) {
  ... all the code in FooPlugin.apply() ...
}

... etc ...
```

Where one of the most important takeaways here is that _build scripts are evaluated top-down, one "declarative block" after the other_. Although, even that is not necessarily true. Kotlin DSL confuses this. Here's a valid settings script:

```kotlin
// settings.gradle.kts
rootProject.name = "..."
pluginManagement { ... }
buildscript { ... }
dependencyResolutionManagement { ... }
plugins { ... }
```

Gradle will not complain if you do this, but actually some of these blocks are Special and get evaluated not in top-down order. `buildscript` is the Primordial Special block; it is always evaluated first (which makes sense, as it contributes dependencies _to the script itself_). `pluginManagement` is evaluated next, followed by `plugins`, followed by everything else. Sometimes, when I'm working on a build, I will reorder blocks to match actual evaluation order because I want users to remember this (implicitly), but mostly I've given up. Note that, in a Groovy DSL script, Gradle will actually complain and refuse to compile a script that is written out-of-order. ¯\\\_(ツ)\_/¯

### What about `afterEvaluate` and various other lazy callbacks?

This isn't a master's thesis.

## Sigh

Anyway.

## Can I create a custom extension that is configured in the root project and which configures all subprojects, and is safe for [isolated projects](https://docs.gradle.org/nightly/userguide/isolated_projects.html)?

Yes, but not easily. Here's what we want to do, from a "UI" perspective:

```kotlin
// root build.gradle.kts
plugins {
  id("my-root-plugin")
}

myExtension { ... }

// subproject/build.gradle.kts
plugins {
  id("my-sub-plugin")
}

// values set in rootProject.myExtension are
// available here, safely
```

Where "safely" means these projects aren't, uh, _unsafely_ coupled, and in particular don't access one another's mutable state. In very particular, they should follow the [Isolated Projects contract](https://docs.gradle.org/nightly/userguide/isolated_projects.html) (which is currently a WIP). I'll elaborate on "unsafely coupled" in a moment.

Naively, what we'd like to do is something like this:

```kotlin
class MySubPlugin : Plugin<Project> {
  fun apply(target: Project) {
    val myExtension = target.rootProject
      .extensions
      .getByType(MyExtension::class.java)

    // do things with values from rootProject's
    // myExtension extension.
  }
}
```

That, however, is _not safe_. A project's `ExtensionContainer` is _mutable_ and accessing another project's _mutable state_ is _not safe_ (maybe the API should forbid this 🤔).

So instead of getting access to that state directly, we will instead use one of Gradle's few blessed ways to manage global mutable state for access at configuration time: the [Shared Build Service](https://docs.gradle.org/current/userguide/build_services.html).

The data flow will be as follows:
1. Users configure the extension in the root project.
2. Method calls on the extension immediately push data into mutable objects held by the build service.
3. Subprojects query the data in the service during configuration.
4. With a little bit of cleverness, we could also have extensions in subprojects configure data just for _their_ subprojects. (This is left as an exercise for the reader.)

Important! Note that subprojects _are_ coupled to parent projects! (The root project, in this case.) But so long as we don't access the parent projects' mutable state directly, it's ok! I promise! Gradle promises to always configure parent projects before child projects, so there's always this _temporal_ coupling. (The main caveat to this is if you use an API like [`evaluationDependsOn()`](https://docs.gradle.org/current/javadoc/org/gradle/api/Project.html#evaluationDependsOn-java.lang.String-)).

> Finally, evaluate each Project by executing its "build.gradle" file, if present, against the project. The projects are evaluated in breadth-wise order, such that a project is evaluated before its child projects. This order can be overridden by calling `evaluationDependsOnChildren()` or by adding an explicit evaluation dependency using `evaluationDependsOn(String)`.

So, uh, don't do that or everything goes sideways.

### Implementing the root extension

The following demonstrates a custom DSL that's configured at the root level. The existence of an inner DSL is strictly unnecessary, but included as a demonstration because it's often useful. From the user perspective, it would be configured like so:

```kotlin
// root build.gradle.kts
myExtension {
  handler {
    foo(/* true or false */)
    bar(/* a String */)
  }
}
```

It's important that the inner handler expose methods, not properties. Methods let us do whatever we want most easily. (A Kotlin property with custom setter would also work, but I think that's unnecessarily complicated.)

```kotlin
abstract class MyExtension @Inject constructor(project: Project) {

  private val objects: ObjectFactory = project.objects

  private val service = MyService.of(project)
  private val myHandler = objects.newInstance(MyHandler::class.java, project, dslService)

  fun handler(action: Action<MyHandler>) {
    action.execute(myHandler)
  }

  internal companion object {
    fun register(project: Project): MyExtension {
      return project.extensions.create(
        "myExtension",
        MyExtension::class.java,
        project,
      )
    }
  }
}

abstract class MyHandler @Inject constructor(
  private val project: Project,
  private val service: Provider<MyService>,
) {

  private val objects = project.objects

  private val foo = objects.property(Boolean::class.java)
  private val bar = objects.property(String::class.java)

  fun foo(value: Boolean) {
    foo.set(value)
    foo.disallowChanges()

    service.get().configureHandlerFor(project) { config ->
      config.foo = value
    }
  }

  fun bar(value: String) {
    bar.set(value)
    bar.disallowChanges()

    service.get().configureHandlerFor(project) { config ->
      config.bar = value
    }
  }
}
```

> We could actually skip the `Property` creation entirely, but I like it because of the `disallowChanges()` API, which ensures the method can only ever be called once, and I want to ensure a single source of truth for most cases.

### Implementing the build service

A ("shared") build service is _kind of like_ a singleton, in that when you register one in _any_ project, it's available in _all_ projects as a single instance. (This unfortunately turns out [not to be true](https://github.com/gradle/gradle/issues/14697), in some cases, when using composite builds, but can be worked around.) An _actual_ singleton (global static instance) doesn't work at all, for the record—try it if you want to lose some sanity. Anyway, use a build service whenever you need global mutable state in your build.

> Please note that the examples in this post demonstrate a project extension having a reference to the build service. Do _not_ attempt to do this in the other direction. You don't want a globally-available object (the build service) having access to a mutable object "owned by" any individual project.

```kotlin
abstract class MyService : BuildService<BuildServiceParameters.None> {

  private val handlerConfigurations = createConfigMap<HandlerConfiguration>()

  internal fun configureHandlerFor(
    project: Project, 
    action: Action<HandlerConfiguration>
  ) {
    handlerConfigurations.merge(
      project.path,
      HandlerConfiguration().apply(action::execute)
    ) { acc, _ ->
      acc.apply(action::execute)
    }
  }

  internal fun findHandlerConfig(project: Project): HandlerConfiguration? {
    return handlerConfigurations.findConfig(project)
  }

  private fun <T> createConfigMap(): MutableMap<String, T> = mutableMapOf()

  private fun <T> Map<String, T>.findConfig(project: Project): T? {
    return get(project.rootProject.path)
  }

  internal companion object {
    fun of(project: Project): Provider<MyService> {
      return project
        .gradle
        .sharedServices
        .registerIfAbsent(
          "myService",
          MyService::class.java
        ) {}
    }
  }
}
```

`project.rootProject.path` is okay because `project.path` is _immutable state_ and safe to access from another project.

`HandlerConfig` is just a mutable value object.

```kotlin
internal class HandlerConfig(
  var foo: Boolean = false,
  var bar: String? = null
)
```

### Consuming the values in a subproject

In your convention plugin that you apply to your subprojects ([you're doing that, right?](https://developer.squareup.com/blog/herding-elephants/)), you can get a reference to the shared build service, and then query the configuration object and do whatever you like with it.

```kotlin
class MySubPlugin : Plugin<Project> {
  fun apply(target: Project) {
    val service = MyService.of(target).get()
    service.findHandlerConfig(target)?.let { config ->
      // do something with config.foo
      // do something with config.bar
    }
  }
}
```

And there we have it. If you want the ability to configure a big project from the root project, with complex data, in a way that keeps your projects mostly decoupled (sigh) from each other, I have a variant of the above running in a large production system.

## Is there another way?

Sure. You could rely entirely on [Gradle properties](https://docs.gradle.org/current/userguide/build_environment.html#sec:gradle_configuration_properties) and organize your configuration knobs as a set of key-value pairs. This is much easier to implement but much harder to use because Gradle properties don't support code completion, don't have easily accessible Javadoc, don't have any IDE support really, and it's shockingly easy to scatter your properties across your entire codebase, making them practically undiscoverable by feature engineers.