JMH with Gradle — From Easy to Simple
Tags: jmh, gradle, benchmarking, jvm
Published: 2018-04-16 17:24:39

---
title: JMH with Gradle — From Easy to Simple
published: true
tags: jmh,gradle,benchmarking, jvm
canonical_url: 
---

[JMH](http://openjdk.java.net/projects/code-tools/jmh/) is _a Java harness for building, running, and analyzing nano/micro/milli/macro benchmarks written in Java and other languages targeting the JVM_.

[Gradle](https://gradle.org/) is my current build tool of choice.

This post tells how to setup JMH in gradle.

Starting with the simple approach. Just include and use this plugin: [https://github.com/melix/jmh-gradle-plugin](https://github.com/melix/jmh-gradle-plugin).

I used it quite a while and it worked out of the box for smaller projects. On medium sized ones i run into some problems, but could resolve them by making use of its configurations (like **zip64 = true** ). For a larger project with a complex dependency setup however, i run into some additional classpath problems.

So i took a step back, looked what the project was doing, also cast a glance at [http://gvsmirnov.ru/blog/tech/2014/03/10/keeping-your-benchmarks-separate.html](http://gvsmirnov.ru/blog/tech/2014/03/10/keeping-your-benchmarks-separate.html) and figured… it’s not as complex… and can be even less complex… E.g. i don’t see why i need that shadow jar stuff!

#### Step by Step

Common practice is to have the JMH benchmarks separated, like instead of putting them e.g. in **src/test/java** , put them into **src/jmh/java**. Do that in gradle by adding another source-set:

```
sourceSets { 
  jmh 
}
```

In case your benchmarks using classes from your tests, do it like that:

```
sourceSets { 
    jmh { 
        compileClasspath += sourceSets.test.runtimeClasspath 
        runtimeClasspath += sourceSets.test.runtimeClasspath 
        } 
}
```

Next you wanna setup the dependencies for that new source-set called **jmh**. So everything your benchmarks depend on, which is usually the project itself (with all its dependencies) and the JMH jars:

```
dependencies { 
    jmhCompile project 
    jmhCompile 'org.openjdk.jmh:jmh-core:1.18' 
    jmhAnnotationProcessor 'org.openjdk.jmh:jmh-generator-annprocess:1.18' 
}
```

Note: **jmhCompile** is bound to the **jmh** source-set definition. You can choose the source-set name freely. If you call it e.g. **benchmark** , use **benchmarkCompile** in the dependencies section!

Anyone still using Eclipse ? This piece helps to set it up correctly:

```
eclipse { 
    classpath { 
        plusConfigurations.add(configurations.jmhCompile) 
        defaultOutputDir = file('build/classes-jmh-ide') 
    } 
}
```

The final piece is about the execution of the benchmark. In its simplest version it looks like that:

```
task jmh(type: JavaExec, description: 'Executing JMH benchmarks') { 
    classpath = sourceSets.jmh.runtimeClasspath 
    main = 'org.openjdk.jmh.Main' 
}
```

But almost certainly you wanna enhance that version, making some of JMH’s parameter, parameter of the Gradle task. E.g. the regex to select the Benchmarks + writing the result to a JSON file:

```
task jmh(type: JavaExec, description: 'Executing JMH benchmarks') { 
    classpath = sourceSets.jmh.runtimeClasspath 
    main = 'org.openjdk.jmh.Main' 

    def include = project.properties.get('include', ''); 
    def format = project.properties.get('format', 'json'); 
    def resultFile = file("build/reports/jmh/result.${format}") 
    resultFile.parentFile.mkdirs() 

    args include 
    args '-rf', format 
    args '-rff', resultFile 
}
```

Voila, now you can run your benchmarks with:

```
gw jmh2 -Pinclude=“.\*MyBenchmark.\*”
```

Note: _There are probably still more parameters you wanna pass through… see the below complete version for some more i found useful._

#### Complete version

Putting everything together and adding some sugar and spice (jmhHelp task, more parameters, etc..) leaves us with that:

{% gist https://gist.github.com/jzillmann/754bf625d43b91fcff295935cbfd1bff %}

#### Closing Story

Few month back i discovered another Gradle plugin — think it was about annotation processing — and told my dear friend and colleague [Mascha](https://mascha.me) about it. I was enthusiastic, he was like ’yeah, i would check what the plugin is doing really, i’m preferring to do simple things on my own lately’.

I virtually raised my eyebrows in a way he couldn’t see it (i like re-using things), but didn’t confront.

Now weeks later, i figured he is right! This time… ;)

#### Final Words

I hope you enjoyed the post! Let me know if you think it’s worth making this a plugin… ;)

Actually, there seems to be a plugin doing the JMH integration in a similar way: [https://github.com/blackboard/jmh-gradle-plugin](https://github.com/blackboard/jmh-gradle-plugin). Checked it out, i haven’t!

* * *