Gradle plugins and extensions: A primer for the bemused
Tags: gradle, kotlin
Published: 2021-08-21 22:28:04

---
title: Gradle plugins and extensions: A primer for the bemused
published: true
description: modeling a complex domain in Gradle
tags: gradle, kotlin
cover_image: https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wdlm15aifdr9ln7b85s5.jpg
---

_All the code for this project is on [Github](https://github.com/autonomousapps/gradle-nested-dsl). ([Github drop ICE](https://github.com/drop-ice/dear-github-2.0).)_

It's recently occurred to me that not everyone has spent their time as poorly as I have, and therefore may not know how to created custom nested DSLs in Gradle plugins. While surprisingly useful, this is, more importantly, _very_ aesthetic.

```groovy
// app/build.gradle
theState {
  theDeepState {
    theDeepestState {
      undermine 'the will of the people'
    }
  }
}
```
*Proof that the IDE kinda-almost provides type hints even for good ol' Groovy DSL (more on that in a bit)*
![Screencap of Groovy DSL demonstrating IDE hints for our custom DSL](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/k2y2q1czt2ejp6ofgnjs.png)

*Proof that the IDE understands Kotlin DSL better than the Groovy version*
![Screencap of Kotlin DSL demonstrating IDE hints for our custom DSL](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/x2zttk14y0i74jlfpzif.png)

> Soon after I took those screencaps, I upgraded this project's version of Gradle from 7.1.1 to 7.2, and my IDE (IntelliJ IDEA Ultimate) got confused and no longer gives me DSL hints for Groovy scripts. ¯\\\_(ツ)\_/¯

Leaving aside why we'd want to undermine the will of the people (I mean, isn't it obvious?), how do we do this?

## Who this is for

<a name="source-1" />This is for anyone looking for non-trivial examples for one of the fundamental building blocks of Gradle plugin design. I wouldn't go so far as to say they're production-ready (sure as hell I'm not going to be writing any tests!), but I _am_ currently using techniques  like these for building a 2+ million LOC application, so&#8230;[<sup>1</sup>](#endnotes)

## A domain-specific language for the secret government bureaucracy controlling our lives

We'll start by looking at the extension itself, then work backwards to how it is configured and used, and finally how to declare and build it.

```kotlin
// TheStateExtension.kt
package mutual.aid.gradle

import org.gradle.api.Action
import org.gradle.api.Project
import org.gradle.api.model.ObjectFactory
import javax.inject.Inject

open class TheStateExtension @Inject constructor(
  objects: ObjectFactory
) {

  /** Configure the inner DSL object, [TheDeepStateHandler]. */
  val theDeepState: TheDeepStateHandler = objects.newInstance(TheDeepStateHandler::class.java)

  /** Configure the inner DSL object, [TheDeepStateHandler]. */
  fun theDeepState(action: Action<TheDeepStateHandler>) {
    action.execute(theDeepState)
  }

  companion object {
    fun Project.theState(): TheStateExtension {
      return extensions.create("theState", TheStateExtension::class.java)
    }
  }
}

/**
 * An inner DSL object.
 */
open class TheDeepStateHandler @Inject constructor(
  objects: ObjectFactory
) {

  /** Configure the innermost DSL object, [TheDeepestStateHandler]. */
  val theDeepestState: TheDeepestStateHandler = objects.newInstance(TheDeepestStateHandler::class.java)

  /** Configure the innermost DSL object, [TheDeepestStateHandler]. */
  fun theDeepestState(action: Action<TheDeepestStateHandler>) {
    action.execute(theDeepestState)
  }
}

/**
 * An even-more inner-er DSL object.
 */
open class TheDeepestStateHandler {

  private val whoToUndermine = mutableListOf<String>()
  internal val victims: List<String> get() = whoToUndermine.toList()

  /** Tells the app who - or which groups - it should undermine. */
  fun undermine(who: String) {
    whoToUndermine.add(who)
  }
}
```

Some of the salient points:
1. I like to name the outermost extension class `FooExtension`, and the inner DSL objects `BarHandler`. Having a convention like that makes it easier to navigate in a large code base.
2. You can inject all of these types with a [variety of services](https://docs.gradle.org/current/userguide/custom_gradle_types.html#service_injection) (like [`ObjectFactory`](https://docs.gradle.org/current/javadoc/org/gradle/api/model/ObjectFactory.html)), as well as completely arbitrary objects you supply. Just remember to `@Inject` that constructor!
3. For something this deep into Gradle territory, let the Gradle APIs do the work for you. Don't try to get creative with Groovy closures or Kotlin lambdas-with-receivers &mdash; just use the `ObjectFactory` and the `Action<T>` interface. I'll elaborate more on this in a moment.
4. You can expose the handlers directly (as I have in the example), as well as exposing them via a function, and this lets your users use both dot-notation and DSL-like syntax with curly braces.

## Instantiating the extension

Now we know how to create simple inner DSL objects. How do we create and configure the outer-most extension?

```kotlin
// ThePluginOfOppression.kt
package mutual.aid.gradle

import mutual.aid.gradle.TheStateExtension.Companion.theState
import org.gradle.api.Plugin
import org.gradle.api.Project
import org.gradle.api.tasks.JavaExec

class ThePluginOfOppression : Plugin<Project> {

  override fun apply(project: Project): Unit = project.run {
    val theState = theState()
  }
}
```
Recall that `TheStateExtension.theState()` is a companion function that is simply `project.extensions.create("theState", TheStateExtension::class.java)`. I like keeping that function with the class itself for encapsulation, as a factory method. It's also important to note that, even though I'm not using the `theState` instance I create, I still need to create it here so that it can be accessed in a build script when this plugin is applied. Let's go ahead and see how that works, before circling back to actually _using_ the potential user-provided config.

## Applying the plugin and configuring the extension in a build script

```groovy
// app/build.gradle
plugins {
  id 'mutual.aid.oppression-plugin'
}

// 1: DSL-like
theState {
  theDeepState {
    theDeepestState {
      undermine 'the will of the people'
    }
  }
}

// 2: With dot-notation for the laconic
theState
  .theDeepState
  .theDeepestState
  .undermine 'the will of the people'

// 3: Mix and match
theState.theDeepState.theDeepestState {
  undermine 'the will of the people'
}
```
Easy-peasy. Apply the plugin and configure the extension. Now's a good time to talk about those `Action<T>` functions that enable the DSL syntax. As a reminder, here's what one looks like:

```kotlin
import org.gradle.api.Action

fun theDeepState(action: Action<TheDeepStateHandler>) {
  action.execute(theDeepState)
}
```

<a name="source-2" />I keep including the import statements in these code snippets because it's important to note the precise API we're using here &mdash; the [`org.gradle.api`](https://docs.gradle.org/current/javadoc/org/gradle/api/package-summary.html) API! [<sup>2</sup>](#endnotes) Gradle has special handling for these types. At (build) runtime, Gradle _rewrites your build code on the fly_ (with [ASM](https://asm.ow2.io/)), such that the method signature `theDeepState(action: Action<T>)` becomes effectively `theDeepState(action: T.() -> Unit)`. Actually, it is more accurate to say you get both. In my Groovy DSL script, I could also use `it.` liberally if I preferred (I don't).

Now we know why the IDE struggles with this with its type hints: it sees the source code, which specifies a standard SAM interface; it doesn't see the lambda-with-receiver that is provided on the fly. 

<a name="source-3" />It's unclear why it looks better with the Kotlin DSL.[<sup>3</sup>](#endnotes) If you explore the [generated type-safe accessors](https://docs.gradle.org/current/userguide/kotlin_dsl.html#type-safe-accessors), they also use `Action<T>`. I guess we'll never know.

## Making use of user-provided configuration: who should we oppress today?

Let's go back to our plugin definition, which has now been expanded to _use_ the information provided by our user in our custom DSL.

```kotlin
class ThePluginOfOppression : Plugin<Project> {

  override fun apply(project: Project): Unit = project.run {
    // 1: Apply additional plugins    
    pluginManager.apply("org.jetbrains.kotlin.jvm")
    pluginManager.apply("application")

    // 2: Create our extension
    val theState = theState()

    // 3: Wait for the DSL to be evaluated, and use the information provided
    afterEvaluate {
      tasks.named("run", JavaExec::class.java) {
        it.args = theState.theDeepState.theDeepestState.victims
      }
    }
  }
}
```
**1. Apply additional plugins**. It isn't strictly necessary to apply these other plugins by our plugin, but it showcases the versatility of [convention plugins](https://dev.to/autonomousapps/easy-modularity-keeping-your-gradle-build-scripts-clean-and-eliminating-duplication-in-your-multi-module-projects-3pa), and also helps keep our example more encapsulatd.
**2. Create our extension**. Same as before.
**3. Make use of user-provided data**. Sometimes it is not possible to use the [`Provider`](https://docs.gradle.org/current/javadoc/org/gradle/api/provider/Provider.html) API and it's necessary to wait for user data &mdash; this is what `afterEvaluate` was made for. In our case, we're pushing the data &mdash; which victims to oppress &mdash; into a standard [`JavaExec` task](https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/JavaExec.html).

Let's run the program and see what happens:
```bash
$ ./gradlew -q app:run
Now undermining: the will of the people
```

Oppression achieved!

## Domain object containers

If you're an Android developer, you'll be familiar with this bit of Gradle configuration:

```groovy
android {
  buildTypes {
    release { ... }
    debug { ... }
    myCustomBuildType { ... }
  }
}
```

Where do those build types come from? We now know how to generate and use nested DSL objects, but these values are user-provided! The situation becomes slightly more clear when one looks at the Kotlin DSL version of the above:

```kotlin
android {
  buildTypes {
    getByName("release") { ... }
    getByName("debug") { ... }
    create("myCustomBuildType") { ... }
  }
}
```
`buildTypes` is a _function_ that provides a `NamedDomainObjectContainer<BuildType>`. Groovy-flavored Gradle has syntactic sugar that converts `debug {}` into `getByName("debug") {}` _OR_ `create("debug") {}` if that named type has not yet been created. In Kotlin, you have to be explicit. This is also, btw, how I learned that there's no default instance named "release" for `signingConfig`.

We now know, in rough terms, what a [`NamedDomainObjectContainer`](https://docs.gradle.org/current/javadoc/org/gradle/api/NamedDomainObjectContainer.html) is. How do we create one? How do we get new instances from one? How do we use it? How do our users use it?

## Using domain object containers

For this next and final example, let's switch it up. Oppression is boring; how can we help instead?

Let's start with a new extension, `ThePeopleExtension`:

```kotlin
package mutual.aid.gradle.people

import org.gradle.api.Action
import org.gradle.api.Named
import org.gradle.api.NamedDomainObjectContainer
import org.gradle.api.Project
import org.gradle.api.model.ObjectFactory
import org.gradle.api.provider.Property
import javax.inject.Inject

open class ThePeopleExtension @Inject constructor(objects: ObjectFactory) {

  val problems = objects.domainObjectContainer(ProblemHandler::class.java)

  fun problems(action: Action<NamedDomainObjectContainer<ProblemHandler>>) {
    action.execute(problems)
  }

  companion object {
    internal fun Project.thePeople(): ThePeopleExtension =
      extensions.create("thePeople", ThePeopleExtension::class.java)
  }
}

open class ProblemHandler @Inject constructor(
  private val name: String,
  objects: ObjectFactory
) : Named {

  override fun getName(): String = name

  internal val description: Property<String> = objects.property(String::class.java)
  val solutions = objects.domainObjectContainer(SolutionHandler::class.java)

  fun solutions(action: Action<NamedDomainObjectContainer<SolutionHandler>>) {
    action.execute(solutions)
  }

  fun description(description: String) {
    this.description.set(description)
    this.description.disallowChanges()
  }
}

open class SolutionHandler @Inject constructor(
  private val name: String,
  objects: ObjectFactory
) : Named {

  override fun getName(): String = name

  internal val action: Property<String> = objects.property(String::class.java)
  internal val description: Property<String> = objects.property(String::class.java)
  internal val rank: Property<Int> = objects.property(Int::class.java)

  fun action(action: String) {
    this.action.set(action)
    this.action.disallowChanges()
  }

  fun description(description: String) {
    this.description.set(description)
    this.description.disallowChanges()
  }

  fun rank(rank: Int) {
    this.rank.set(rank)
    this.rank.disallowChanges()
  }
}
```
Let's talk about a few of the patterns here before we continue.

First, note that the types that are meant to be in a `NamedDomainObjectContainer` all implement the `Named` interface. This isn't strictly necessary, but it _is_ necessary that the types have a `getName(): String` function, or they can't go into a _named_ domain object container.

Second, we create such a container with the method `ObjectFactory.domainObjectContainer(Class<T>)`.

The final interesting pattern in the above is this:

```kotlin
fun description(description: String) {
  this.description.set(action)
  this.description.disallowChanges()
}
```
As `description` is a `Property<String>`, I prefer to keep those values `internal` and expose them via a function. Users then have a nice DSL like `description 'my description` (in Groovy), or `description("my description")` (in Kotlin). Encapsulating those fields also lets me to do extra stuff like call `disallowChanges()`, which I think is important to prevent violating the [principle of least astonishment](https://en.wikipedia.org/wiki/Principle_of_least_astonishment). Without that, users could call the `description()` method repeatedly, from multiple locations, and it would be hard to tell where the data were really coming from. When we do this, and someone attempts to call the method more than once, the build will fail.

Let's continue. How does this DSL look "in action"?

```groovy
// app/build.gradle
thePeople {
  problems {
    climateChange {
      description 'There is no question of cost, because the cost of doing nothing is everything.'
      solutions {
        cleanEnergy {
          description 'We cannot burn any more fossil energy'
          action 'Replace all fossil sources with clean solutions like wind, solar, and geothermal'
          rank 1
        }
        massTransit {
          description 'Single-occupant vehicles are a major source of carbon pollution'
          action 'Increase density in urban environments and build free public transit for all'
          rank 2
        }
        stopEatingAnimals {
          description 'Animal agriculture is one of the top contributors to carbon pollution'
          action 'Most people can thrive on a plant-based diet and do not need animal protein, and could make such a choice with immediate effect'
          rank 3
        }
        antiRacism {
          description 'People of Western European descent (\'white people\') have been the primary beneficiaries of burning fossil carbon'
          action 'White people should should bear the responsibility of paying for climate change mitigation'
          rank 4
        }
        seizeGlobalCapital {
          description 'The costs of climate change are inequitably distributed'
          action 'The costs of climate change mitigation should be born primarily by the wealthiest'
          rank 5
        }
        lastResort {
          description 'If the rich and the powerful refuse to get out of the way of legislative reforms of the system killing us all, there is, unfortunately, always a last resort'
          action 'It starts with \'g\' and rhymes with \'poutine\''
          rank 6
        }
      }
    }
  }
}
```

I think that's fairly readable, given the complexity of the domain we're trying to model.

But now how do we react to that in our plugin? As always, I think learning is best done by example, so let's tie it all together by looking at how a new plugin, `ThePluginOfThePeople`, configures tasks based on this user-provided data.

```kotlin
class ThePluginOfThePeople : Plugin<Project> {

  override fun apply(project: Project): Unit = project.run {
    val thePeople = thePeople()

    thePeople.problems.all { problem ->
      tasks.register("listSolutionsFor${problem.name.capitalize()}", ListSolutionsTask::class.java) {
        it.problem.set(problem)
      }
    }
  }
}

abstract class ListSolutionsTask : DefaultTask() {

  init {
    group = "People"
    description = "Prints list of solutions for a given problem"
  }

  @get:Input
  abstract val problem: Property<ProblemsHandler>

  @TaskAction fun action() {
    val problem = problem.get()

    val msg = buildString {
      appendLine(problem.name.capitalize())
      appendLine(problem.description.get())
      appendLine()
      appendLine("Solutions:")
      problem.solutions.sortedBy { it.rank.get() }.forEachIndexed { i, sol ->
        appendLine("${i + 1}. ${sol.name}")
        appendLine("   ${sol.description.get()}")
        appendLine("   ${sol.action.get()}")
      }
    }

    logger.quiet(msg)
  }
}
```

We can see all the tasks that our new plugin registered quite easily:

```bash
$ ./gradlew app:tasks --group people -q

------------------------------------------------------------
Tasks runnable from project ':app'
------------------------------------------------------------

People tasks
------------
listSolutionsForClimateChange - Prints list of solutions for a given problem
```

In our plugin, we use `thePeople.problems.all(Action<T>)` to react to user-provided configuration. `all(Action<T>)` executes the provided action against all elements of the given collection, as well as all _future_ elements that may be added; in this sense, it is lazy. For us, it is useful because the plugin's `apply()` method runs immediately when the plugin is applied (in the `plugins` block), which means that the user data is not yet available to react on. `all()` elegantly solves this problem without recourse to, say, `afterEvaluate`.

Within our `problems.all` block, we register a single task &mdash; one task per problem &mdash; and configure that task by setting its one input as the given `ProblemHandler`, on a `Provider<ProblemHandler>`. This is fully serializable, and so is a valid `@Input` property, as well as being compatible with the experimental [configuration cache](https://docs.gradle.org/current/userguide/configuration_cache.html).

Our task definition is straightforward. It's an abstract class, letting us used [managed types](https://docs.gradle.org/current/userguide/custom_gradle_types.html#managed_types) (our `@Input abstract val problem`), and has a simple action. The biggest footgun here is remembering to call `get()` on the various `Provider<String>` instances, else we'll get  funny output like `property 'description$fancy_plugin'`.

Finally, let's run one of the generated tasks like so:

```bash
$ ./gradlew app:listSolutionsForClimateChange
Configuration cache is an incubating feature.
Calculating task graph as configuration cache cannot be reused because file 'app/build.gradle' has changed.

> Task :app:listSolutionsForclimateChange
ClimateChange
There is no question of cost, because the cost of doing nothing is everything.

Solutions:
1. cleanEnergy
   We cannot burn any more fossil energy
   Replace all fossil sources with clean solutions like wind, solar, and geothermal
2. massTransit
   Single-occupant vehicles are a major source of carbon pollution
   Increase density in urban environments and build free public transit for all
3. stopEatingAnimals
   Animal agriculture is one of the top contributors to carbon pollution
   Most people can thrive on a plant-based diet and do not need animal protein, and could make such a choice with immediate effect
4. antiRacism
   People of Western European descent ('white people') have been the primary beneficiaries of burning fossil carbon
   White people should should bear the responsibility of paying for climate change mitigation
5. seizeGlobalCapital
   The costs of climate change are inequitably distributed
   The costs of climate change mitigation should be born primarily by the wealthiest
6. lastResort
   If the rich and the powerful refuse to get out of the way of legislative reforms of the system killing us all, there is, unfortunately, always a last resort
   It starts with 'g' and rhymes with 'poutine'
```

## Wrapping up

In this post, we learned how to use Gradle to model a complex domain with a nested domain-specific language, or DSL, and how to accommodate custom user data in such a DSL using the `NamedDomainObjectContainer`. I encourage you to explore the complete sample on Github, which includes build scripts and project layout decisions that were left out of this post, for simplicity.

{% github autonomousapps/gradle-nested-dsl no-readme %} 

<a name="endnotes"/>
## Endnotes
<sup>1</sup> Watch this space for me lamenting being fired for my terrible technique. [<small>up</small>](#source-1)
<sup>2</sup> I highly recommend exploring these packages in depth. [<small>up</small>](#source-2)
<sup>3</sup> Jetbrains is the creator of the Kotlin language. [<small>up</small>](#source-3)