fixing a (transitive) dependency mess with gradle
Tags: gradle, java
Published: 2020-06-09 09:02:18

# the transitive dependency mess

While extracting openapi-processor-core from [*openapi-processor-spring*](https://hauner.github.io/openapi-processor/spring/index.html) I noticed duplicate dependencies with different and sometimes unexpected versions in the external libraries section o the project view (in IDEA).

In this case jackson packages.

Running `gradle dependencies` to see where they are coming from prints the following output (look for `jackson`):

_gradle dependencies (simplified)_

```text
+--- com.fasterxml.jackson.module:jackson-module-kotlin:2.11.0
|    +--- com.fasterxml.jackson.core:jackson-databind:2.11.0 <1>
|    |    +--- com.fasterxml.jackson.core:jackson-annotations:2.11.0
|    |    \--- com.fasterxml.jackson.core:jackson-core:2.11.0
|    \--- com.fasterxml.jackson.core:jackson-annotations:2.11.0
+--- org.openapi4j:openapi-parser:1.0
|    \--- org.openapi4j:openapi-core:1.0
|         +--- com.fasterxml.jackson.core:jackson-databind:2.9.10 -> 2.11.0 (*) <2>
|         \--- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.9.10 -> 2.10.2 <3>
|              \--- com.fasterxml.jackson.core:jackson-core:2.10.2 -> 2.11.0 <4>
+--- io.swagger.parser.v3:swagger-parser:2.0.20
     +--- io.swagger.parser.v3:swagger-parser-v3:2.0.20
          +--- io.swagger.core.v3:swagger-models:2.1.2
          |    \--- com.fasterxml.jackson.core:jackson-annotations:2.10.1 -> 2.11.0 <5>
          +--- io.swagger.core.v3:swagger-core:2.1.2
          |    +--- com.fasterxml.jackson.core:jackson-annotations:2.10.1 -> 2.11.0
          |    +--- com.fasterxml.jackson.core:jackson-databind:2.10.1 -> 2.11.0 (*)
          |    +--- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.10.1 -> 2.10.2 (*) <6>
          |    +--- com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.10.1 <7>
          |         +--- com.fasterxml.jackson.core:jackson-annotations:2.10.1 -> 2.11.0
          |         +--- com.fasterxml.jackson.core:jackson-core:2.10.1 -> 2.11.0
          |         \--- com.fasterxml.jackson.core:jackson-databind:2.10.1 -> 2.11.0 (*)
          +--- com.fasterxml.jackson.core:jackson-annotations:2.10.2 -> 2.11.0
          +--- com.fasterxml.jackson.core:jackson-databind:2.10.2 -> 2.11.0 (*)
          \--- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.10.2 (*) <8>
```

There is a bit of a mess regarding the jackson packages and their versions. Here are a couple of entries:

<1> some with version 2.11.0
<2> upgrade from 2.9.10 to 2.11.0
<3> upgrade from 2.9.10 to 2.10.2
<4> upgrade from 2.10.2 to 2.11.0
<5> upgrade from 2.10.1 to 2.11.0
<6> upgrade from 2.10.1 to 2.10.2
<7> version 2.10.1
<8> version 2.10.2

The 3 top-level dependencies are direct dependencies (as gradle calls them) of my project and all of them use different jackson versions:

. `com.fasterxml.jackson.module:jackson-module-kotlin:2.11.0`
. `org.openapi4j:openapi-parser:1.0`
. `io.swagger.parser.v3:swagger-parser:2.0.20`

The first one is simple. Because `com.fasterxml.jackson.module:jackson-module-kotlin` is a jackson package itself, gradle selects **2.11.0** for the other transitive jackson dependencies.

_jackson-module-kotlin (simplified)_
```text
    +--- com.fasterxml.jackson.module:jackson-module-kotlin:2.11.0
         +--- com.fasterxml.jackson.core:jackson-annotations:2.11.0
         +--- com.fasterxml.jackson.core:jackson-core:2.11.0
         \--- com.fasterxml.jackson.core:jackson-databind:2.11.0
```

Next one is `org.openapi4j:openapi-parser:1.0`. This one depends on jackson **2.9.10** and gradle upgrades everything to **2.11.0** because that is the newest version it found.

_openapi-parser (simplified)_
```text
+--- org.openapi4j:openapi-parser:1.0
     +--- com.fasterxml.jackson.core:jackson-core:2.10.2 -> 2.11.0
     +--- com.fasterxml.jackson.core:jackson-databind:2.9.10 -> 2.11.0 (*)
     \--- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.9.10 -> 2.10.2
```

Ok, not for all jackson packages. It doesn't use **2.11.0** for `jackson-dataformat-yaml`.

It is not using **2.11.0** because gradle has not seen a newer version of `jackson-dataformat-yaml`than **2.9.10**. So why is it not **2.9.10**? It is not because the third direct dependency `io.swagger.parser.v3:swagger-parser:2.0.20` is using **2.10.1** and **2.10.2**.

_swagger-parser (simplified)_

```text
+--- io.swagger.parser.v3:swagger-parser:2.0.20
     +--- com.fasterxml.jackson.core:jackson-annotations:2.10.1 -> 2.11.0
     +--- com.fasterxml.jackson.core:jackson-core:2.10.1 -> 2.11.0
     +--- com.fasterxml.jackson.core:jackson-databind:2.10.1 -> 2.11.0
     +--- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.10.1 -> 2.10.2
     \--- com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.10.1
```

**2.10.2** for `jackson-dataformat-yaml`. This is the highest version for this package so gradle picks it.

Gradle finally selects the following versions:

package (stripped com.fasterxml.jackson.) | version
--- | ---
`core:jackson-annotations`| 2.11.0
`core:jackson-core`       | 2.11.0
`core:jackson-databind`   | 2.11.0
`dataformat:jackson-dataformat-yaml`| 2.10.2   => want 2.11.0
`datatype:jackson-datatype-jsr310`| 2.10.1 => want 2.11.0


Gradle uses the highest version it does find for each package. It doesn't know that the jackson packages belong together and that all of them should use the same version.

There are two possible solutions to make all jackson packages use the expected version.

## solution

According to the gradle documentation the preferred solution is to add a `constraints` block to the `dependencies`. Adding a direct dependency is not the correct solution:

[gradle-constraints](https://docs.gradle.org/current/userguide/dependency_constraints.html)

> Often developers incorrectly fix transitive dependency issues by adding direct dependencies. To avoid this, Gradle provides the concept of dependency constraints.

Modifying the dependency resolution rules is not an option either:

[gradle dependency resolve rules](https://docs.gradle.org/current/userguide/resolution_rules.html#sec:dependency_resolve_rules)

> If you are authoring a library, you should always prefer dependency constraints as they are published for your consumers.

### constraints

The project is a library and the constraints solution look like this in the `build.gradle`:

```groovy
ext {
    jacksonVersion = '2.11.0'
}

dependencies {
    // .. dependencies ..

    constraints {
        implementation("com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:$jacksonVersion") {
            because 'use the same version for all jackson packages'
        }
        implementation("com.fasterxml.jackson.datatype:jackson-datatype-jsr310:$jacksonVersion") {
            because 'use the same version for all jackson packages'
        }
    }

}
```

Ok, this fixes the issue. :-)

Too bad each single package requires its own constraint (the closure with `because` ist not required). It would be nice to use wildcards to match all jackson packages with a single constraint:

```groovy
ext {
    jacksonVersion = '2.11.0'
}

dependencies {
    // .. dependencies ..

    constraints {
        // does NOT work
        implementation("com.fasterxml.jackson.*:*:$jacksonVersion") {
            because 'use the same version for all jackson packages'
        }
    }

}
```

That would be a bit simpler but unfortunately it does not work.

### platform

I found another solution in the gradle documentation that even uses jackson as an example:

[gradle dependency version alignment](https://docs.gradle.org/current/userguide/dependency_version_alignment.html).

First step is to create a metadata rule that joins all jackson packages into a *platform*. Jackson has a platform bom that lists all packages belonging to a platform version, and it is used to create the platform rule:

```groovy
class JacksonPlatformRule implements ComponentMetadataRule {
    void execute (ComponentMetadataContext ctx) {
        ctx.details.with {
            if (id.group.startsWith ("com.fasterxml.jackson")) {
                belongsTo ("com.fasterxml.jackson:jackson-bom:${id.version}", false)
            }
        }
    }
}
```

I simply added the class at the end of the `build.gradle` for testing, but it will probably move to gradles `buildSrc` folder.

Second step is to activate the rule in the `dependencies` block:

```groovy
dependencies {
    components.all(JacksonPlatformRule)

    // .. dependencies ..
}
```

By grouping the packages gradle now selects the jackson **platform** with the highest version and uses the versions listed in the bom for any jackson package.

This is also visible in the output of `gradle dependencies` (starting at line 4). Gradle selects the **2.11.0** platform module and upgrades all jackson packages to the platform version:

_gradle dependencies (simplified)_
```text
+--- com.fasterxml.jackson.module:jackson-module-kotlin:2.11.0
|    +--- com.fasterxml.jackson.core:jackson-databind:2.11.0
|    |    +--- com.fasterxml.jackson.core:jackson-annotations:2.11.0
|    |    |    \--- com.fasterxml.jackson:jackson-bom:2.11.0
|    |    |         +--- com.fasterxml.jackson:jackson-bom:2.11.0 (*)
|    |    |         +--- com.fasterxml.jackson.core:jackson-annotations:2.11.0 (c)
|    |    |         +--- com.fasterxml.jackson.core:jackson-core:2.11.0 (c)
|    |    |         +--- com.fasterxml.jackson.core:jackson-databind:2.11.0 (c)
|    |    |         +--- com.fasterxml.jackson.module:jackson-module-kotlin:2.11.0 (c)
|    |    |         +--- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.11.0 (c)
|    |    |         \--- com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.11.0 (c)
|    |    +--- com.fasterxml.jackson.core:jackson-core:2.11.0
|    |    |    \--- com.fasterxml.jackson:jackson-bom:2.11.0 (*)
|    |    \--- com.fasterxml.jackson:jackson-bom:2.11.0 (*)
|    +--- com.fasterxml.jackson.core:jackson-annotations:2.11.0 (*)
|    \--- com.fasterxml.jackson:jackson-bom:2.11.0 (*)
+--- org.openapi4j:openapi-parser:1.0
|    \--- org.openapi4j:openapi-core:1.0
|         +--- com.fasterxml.jackson.core:jackson-databind:2.9.10 -> 2.11.0 (*)
|         \--- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.9.10 -> 2.11.0
|              +--- com.fasterxml.jackson.core:jackson-databind:2.11.0 (*)
|              +--- org.yaml:snakeyaml:1.26
|              +--- com.fasterxml.jackson.core:jackson-core:2.11.0 (*)
|              \--- com.fasterxml.jackson:jackson-bom:2.11.0 (*)
+--- io.swagger.parser.v3:swagger-parser:2.0.20
     +--- io.swagger.parser.v3:swagger-parser-v3:2.0.20
          +--- io.swagger.core.v3:swagger-models:2.1.2
          |    \--- com.fasterxml.jackson.core:jackson-annotations:2.10.1 -> 2.11.0 (*)
          +--- io.swagger.core.v3:swagger-core:2.1.2
          |    +--- com.fasterxml.jackson.core:jackson-annotations:2.10.1 -> 2.11.0 (*)
          |    +--- com.fasterxml.jackson.core:jackson-databind:2.10.1 -> 2.11.0 (*)
          |    +--- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.10.1 -> 2.11.0 (*)
          |    \--- com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.10.1 -> 2.11.0
          |         +--- com.fasterxml.jackson.core:jackson-annotations:2.11.0 (*)
          |         +--- com.fasterxml.jackson.core:jackson-core:2.11.0 (*)
          |         +--- com.fasterxml.jackson.core:jackson-databind:2.11.0 (*)
          |         \--- com.fasterxml.jackson:jackson-bom:2.11.0 (*)
          +--- com.fasterxml.jackson.core:jackson-annotations:2.10.2 -> 2.11.0 (*)
          +--- com.fasterxml.jackson.core:jackson-databind:2.10.2 -> 2.11.0 (*)
          \--- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.10.2 -> 2.11.0 (*)
```

## conclusion

The `constraints` solution is easy to use on single dependencies. The platform rule is the way to go for groups of packages like jackson.

That's it.