Designing a Gradle plugin for J2CL
Tags: j2cl, gwt, bazel, java
Published: 2020-07-28 10:24:10

---
title: Designing a Gradle plugin for J2CL
canonical_url: https://blog.ltgt.net/designing-gradle-j2cl-plugin/
published: true
series: gradle-j2cl-plugin
tags: j2cl,gwt,bazel,java
---

<p>In the <a href="https://dev.to/tbroyer/reverse-engineering-j2cl-bazel-integration-1o1f">previous post</a>,
I explored how J2CL is used in Bazel.
Starting with this post, with this knowledge,
I'll try to design a Gradle plugin for J2CL.</p>
<h2 id="building-blocks">Building blocks</h2>
<p>Let's start with the low-level building blocks.</p>
<p>All the J2CL (<code>GwtIncompatibleStrip</code> and <code>J2clTranspile</code>)
and Closure compiler (<code>CommandLineRunner</code>) tools
can easily be called from Java processes
(the Bazel workers actually call different, sometimes internal, APIs),
so they could be called from <a href="https://docs.gradle.org/6.5.1/userguide/custom_tasks.html#worker_api">Gradle workers</a>.
It also shouldn't be a problem to call the J2CL tools incrementally,
only processing changed files;
well, actually, it won't be a problem for the <code>GwtIncompatibleStrip</code>
which processes files one by one in isolation;
the <code>J2clTranspile</code> however, just like <code>javac</code>,
would also need to reprocess other Java files referencing the changed files,
so making it incremental would mean knowing about those dependencies;
let's leave this optimization work for later
(FWIW, the way Bazel deals with it is to <strong>not</strong> do any incremental/partial processing of any kind,
but instead using small sets of files, generally at the Java package level).</p>
<p>That means we'd have to declare 3 <code>configuration</code>s (for the 3 tools' dependencies),
and we could create corresponding tasks with proper inputs and outputs.</p>

![Inputs and outputs of the GwtIncompatibleStrip task](https://dev-to-uploads.s3.amazonaws.com/i/15sofbzegk162hsmi07e.png)

![Inputs and outputs of the J2clTranspile task](https://dev-to-uploads.s3.amazonaws.com/i/ci3dcogzbfb0x6uddv0d.png)

![Inputs and outputs of the ClosureCompile task](https://dev-to-uploads.s3.amazonaws.com/i/mn5b2ss7gyqnbehuhst6.png)

<p>That would work for project sources though, but not external dependencies.
For those, we could probably use <a href="https://docs.gradle.org/6.5.1/userguide/artifact_transforms.html">artifact transforms</a>,
but that excludes compiling the source files with <code>javac</code>,
at least if we want to leverage the Gradle standard <code>JavaCompile</code> task
(I've been floating the idea of an ASM-based <code>GwtIncompatibleStrip</code>,
which would solve the problem then,
as an artifact transform of the binary JAR).
Let's keep external dependencies for later though.</p>
<p>Speaking of the <code>JavaCompile</code> tasks,
for J2CL we'd want to configure their <a href="https://docs.gradle.org/6.5.1/dsl/org.gradle.api.tasks.compile.CompileOptions.html#org.gradle.api.tasks.compile.CompileOptions:bootstrapClasspath"><code>bootstrapClasspath</code></a> to the Java Runtime Emulation JAR.
Unfortunately, <code>-bootclasspath</code> <a href="https://docs.oracle.com/en/java/javase/11/tools/javac.html#GUID-AEEC9F07-CB49-4E96-8BC7-BCC2C7F725C9__GUID-38BC1737-2F22-4288-8DC9-933C37D471AB">can only be used</a> when compiling for Java 8,
so that rules out using any Java 9+ syntax: private methods in interfaces, <code>var</code> for type inference, etc.
For those more recent Java versions, we'd want to use <a href="https://docs.oracle.com/en/java/javase/11/tools/javac.html#GUID-AEEC9F07-CB49-4E96-8BC7-BCC2C7F725C9__GUID-96B805B4-3A9B-45B7-887B-84D0897C9CA2"><code>--system</code></a>,
but that's an entirely different format,
and one that's not even supported by Bazel yet, let alone produced by J2CL
(though maybe it's not that hard to create from the Java Runtime Emulation JAR).
Fwiw, Google also faces the same issue for Android for adding Java 9+ syntax support,
as well as J2ObjC,
so we can be sure they'll find a solution
(they're actually already <a href="https://github.com/google/j2cl/issues/97">working on it</a>).</p>
<h2 id="tests">Tests</h2>
<p>For tests, the annotation processor and its <code>@J2clTestInput</code> have been designed as implementation details,
hidden behind a <code>j2cl_test</code> rule in Bazel.
Contrary to Bazel where one <code>j2cl_test</code> rule only runs a single test class (which could actually be a test suite),
with a <code>gen_j2cl_tests</code> macro to generate them automatically from source files using a naming convention,
in Gradle we'll have a single task for the whole <code>src/test</code>.
We could however have a task that generates a JUnit suite, annotated with <code>@J2clTestInput</code>,
and referencing the test classes, using a naming convention,
and then processes it with the annotation processor.</p>
<p>This will generate some Java code to be transpiled with J2CL.
This phase can reuse the <code>J2clTranspile</code> task from above,
using the <code>src/test/java</code> and the generate Java code all at once.</p>
<p>The <code>test_summary.json</code> file then needs to be processed
and fan out one Closure compilation per JS entrypoint
(one per non-suite test class, with <code>.testsuite</code> file extension).
This cannot reuse the <code>ClosureCompile</code> task though:
we want a single task driving multiple Closure compilations.
The result will thus be several JS applications;
we'll put them into separate directories (named after the original Java test),
and generate an additional HTML page to run them.
The task could be made incremental, only recompiling tests that have changed,
but that would need dependency information between files
(that can be extracted using Closure, with some additional work;
or possibly even just analyzing the <code>goog.provide</code>/<code>goog.require</code>).
BTW, that knowledge could possibly also be used by the <code>ClosureCompile</code> task
to skip compilation if a file has changed that's not needed by anything.</p>
<p>Finally, we'll need to run those generated tests in browsers.
The best way to do that is through Selenium WebDriver.
We'll thus want a task taking those directories of compiled tests,
that starts an HTTP server to serve them,
and loads each of them in a web browser through WebDriver,
generating reports (that helps skipping the task if no input has changed).</p>

![Inputs and outputs of the GenerateTests task](https://dev-to-uploads.s3.amazonaws.com/i/nl3bdlzydozgfj4vod08.png)

![Inputs and outputs of the ClosureCompileTests task](https://dev-to-uploads.s3.amazonaws.com/i/hck74q6yzx9vtn73k5n6.png)

![Inputs and outputs of the J2clTest task](https://dev-to-uploads.s3.amazonaws.com/i/jfr0qvm7u35k2wpbgl9n.png)

<h2 id="putting-it-all-to-work">Putting it all to work</h2>
<p>For an application like the HelloWorld sample from the Bazel repository,
wiring all those tasks together would lead to a graph like the following:</p>

![Directed graph of tasks and their input/output files in the HelloWorld sample](https://dev-to-uploads.s3.amazonaws.com/i/lxdpl3z1lbull79zz8zc.png)

<!--
digraph {
    subgraph clustermain {
        "src/main" -> stripGwtIncompatible [label="**/*.java"]
        stripGwtIncompatible [shape=box]
        stripGwtIncompatible -> "<stripped sources>"
        "<stripped sources>" -> compileJava
        compileJava [shape=box]
        compileJava -> "<compiled classes>"
        compileJava -> "<generated sources>"
        "src/main" -> transpileJ2cl [label="**/*.native.js"]
        "<stripped sources>" -> transpileJ2cl
        "<generated sources>" -> transpileJ2cl
        transpileJ2cl [shape=box]
        transpileJ2cl -> "<transpiled sources>"
        "src/main" -> compileClosure [label="**/*.js !**/*.native.js"]
        compileClosure [shape=box]
        "<transpiled sources>" -> compileClosure
    }

    subgraph clustertest {
        "src/test" -> stripTestGwtIncompatible [label="**/*.java"]
        stripTestGwtIncompatible [shape=box]
        stripTestGwtIncompatible -> "<stripped test sources>" 
        "<stripped test sources>" -> compileTestJava
        "<compiled classes>" -> compileTestJava
        compileTestJava [shape=box]
        compileTestJava -> "<compiled test classes>"
        compileTestJava -> "<generated test sources>"
        { rank="same"; "<compiled test classes>" "<generated test sources>"}
        "<compiled classes>" -> generateTests
        "<compiled test classes>" -> generateTests
        generateTests [shape=box]
        generateTests -> "<generated tests>"
        "<generated tests>"
        "<generated tests>" -> compileTestClosure [label="**/test_summary.json **/*.testsuite"]
        "<generated tests>" -> transpileTestJ2cl [label="**/*.java"]
        "src/test" -> transpileTestJ2cl [label="**/*.native.js"]
        "<compiled classes>" -> transpileTestJ2cl
        "<stripped test sources>" -> transpileTestJ2cl
        "<generated test sources>" -> transpileTestJ2cl
        transpileTestJ2cl [shape=box]
        transpileTestJ2cl -> "<transpiled test sources>"
        "<transpiled sources>" -> compileTestClosure
        "<transpiled test sources>" -> compileTestClosure
        "src/main" -> compileTestClosure [label="**/*.js !**/*.native.js"]
        "src/test" -> compileTestClosure [label="**/*.js !**/*.native.js"]
        compileTestClosure [shape=box]
        compileTestClosure -> "<closure compiled tests>"
        "<closure compiled tests>"
        "<closure compiled tests>" -> test
        test [shape=box]
    }
}
-->
<p>The code for these tasks and sample project is available <a href="https://github.com/tbroyer/gradle-j2cl-plugin">on Github</a>.</p>
{% github tbroyer/gradle-j2cl-plugin no-readme %}
<h2 id="next-steps">Next steps</h2>
<p>Now that we have the building blocks and are able to wire them together for a simple example,
the next steps will be:</p>
<ul>
<li>handling external dependencies (stripping and transpiling them on-the-fly)</li>
<li>handling project dependencies (a library subproject would expose its transpiled sources to an application subproject)</li>
<li>defining conventions to make things as simple as applying a Gradle plugin (a J2CL application would be the easiest, as libraries could target J2CL, GWT, the JVM, J2ObjC, etc.)</li>
</ul>