Config Gradle to generate Java code from Protobuf
Tags: grpc, java, tutorial, beginners
Published: 2020-02-29 10:08:47

In the last lecture, we have finished writing our protocol buffer messages and generate Go codes from them. Today we're going to do the same for Java.

Here's the link to the [full gRPC course playlist on Youtube](https://www.youtube.com/playlist?list=PLy_6D98if3UJd5hxWNfAqKMr15HZqFnqf)
Github repository: [pcbook-go](https://github.com/techschool/pcbook-go) and [pcbook-java](https://github.com/techschool/pcbook-java)
Gitlab repository: [pcbook-go](https://gitlab.com/techschool/pcbook) and [pcbook-java](https://gitlab.com/techschool/pcbook-java)

{% youtube nB7J6D_IXrA %}

We will create a new Gradle project, setup some plugins and config them to automatically generate Java code whenever the project is built. We will also learn how to use option to customise the generated codes.

OK, let's start!

## New Gradle project

First, make sure you already have Java Development Kit and IntelliJ IDEA installed on your computer.

![New Gradle project](https://dev-to-uploads.s3.amazonaws.com/i/xcir5xyrmbckohnnsymc.png)

Open IntelliJ IDEA and create a new project. We will use Gradle because Google has an official Gradle plugin for protocol buffer. For the project SDK, make sure that the correct Java version is selected. It should be at least Java 8. Then click Next.

![Fill in GroupID and ArtifactID](https://dev-to-uploads.s3.amazonaws.com/i/cx2evhmilq8ftm9xwbpk.png)

Fill in the GroupID, the ArtifactID. Check the project name and its location to be exactly what you want. Then click Finish. It might take a few seconds for IntelliJ IDEA to setup the project.

![Fill in project name and location](https://dev-to-uploads.s3.amazonaws.com/i/ukd0w7sdfmfue2e07ifo.png)

## Config protobuf-gradle plugin

Now we will open the `build.gradle` file to setup some plugins. The first one is `protobuf-gradle` plugins from Google.

![Protobuf Gradle plugin](https://dev-to-uploads.s3.amazonaws.com/i/lt6sozgmmn72isarrqm5.png)

Open [this Github page](https://github.com/google/protobuf-gradle-plugin), scroll down a bit, copy the plugins block and paste it to our `build.gradle file`:

```gradle
plugins {
    id "com.google.protobuf" version "0.8.10"
    id "java"
}

group 'com.gitlab.techschool'
version '1.0-SNAPSHOT'

sourceCompatibility = 1.8

repositories {
    mavenCentral()
}

dependencies {
    testCompile group: 'junit', name: 'junit', version: '4.12'
}
```

## Add protobuf-java dependency

Next, we need to add a dependency: `protobuf-java` to our project. Let's open [this maven repository page](https://mvnrepository.com/artifact/com.google.protobuf/protobuf-java). 

It's the `protobuf-java` artifact of `com.google.protobuf` package. Let's select the latest version: `3.10.0`

![Add protobuf-java dependency](https://dev-to-uploads.s3.amazonaws.com/i/3pexossjwt6uelr8ywpr.png)

Click on the Gradle tab and copy the setting, then paste it into the dependencies block of our `build.gradle` file:

```gradle
plugins {
    id "com.google.protobuf" version "0.8.10"
    id "java"
}

group 'com.gitlab.techschool'
version '1.0-SNAPSHOT'

sourceCompatibility = 1.8

repositories {
    mavenCentral()
}

dependencies {
    testCompile group: 'junit', name: 'junit', version: '4.12'

    // https://mvnrepository.com/artifact/com.google.protobuf/protobuf-java
    compile group: 'com.google.protobuf', name: 'protobuf-java', version: '3.10.0'
}
```

## Add grpc-all dependency

We will need a package to work with gRPC as well. So let's go back to the maven repository and search for `grpc-all`. It should be found at [this page](https://mvnrepository.com/artifact/io.grpc/grpc-all).

![Add grpc-all dependency](https://dev-to-uploads.s3.amazonaws.com/i/ium78ng5jgg7eebhprla.png)

Select the latest version (`1.25.0` in my case). Similar as before, open the Gradle tab and copy the setting, then paste it to the dependencies block of our `build.gradle` file. IntelliJ IDEA will automatically detect and configure it for us.

```gradle
plugins {
    id "com.google.protobuf" version "0.8.10"
    id "java"
}

group 'com.gitlab.techschool'
version '1.0-SNAPSHOT'

sourceCompatibility = 1.8

repositories {
    mavenCentral()
}

dependencies {
    testCompile group: 'junit', name: 'junit', version: '4.12'

    // https://mvnrepository.com/artifact/com.google.protobuf/protobuf-java
    compile group: 'com.google.protobuf', name: 'protobuf-java', version: '3.10.0'

    // https://mvnrepository.com/artifact/io.grpc/grpc-all
    compile group: 'io.grpc', name: 'grpc-all', version: '1.25.0'

}
```

## Setup protobuf compiler

Now we will setup protobuf compiler. By default, the `protobuf-gradle-plugin` will search for `protoc` executable in the system. We've already installed it in previous lecture with Homebrew. However, if you come here directly for Java, I will show you another way to get the pre-compiled `protoc`.

First, go to the maven repository and look for `protoc`. It can be found in [this page](https://mvnrepository.com/artifact/com.google.protobuf/protoc).

Select the latest version (`3.10.1` in my case), but don't add it to the dependencies block as before. Instead, we will config it in a separate block.

Let's go back to the `protobuf-gradle-plugin` github page, and copy this setting block:

![Config protoc](https://dev-to-uploads.s3.amazonaws.com/i/lh7akvkdbi2068xfkqmb.png)

Paste it in our `build.gradle` file, replace the version with the latest one that we've found above, and IntelliJ will take care of the rest.

```gradle
plugins {
    id "com.google.protobuf" version "0.8.10"
    id "java"
}

group 'com.gitlab.techschool'
version '1.0-SNAPSHOT'

sourceCompatibility = 1.8

repositories {
    mavenCentral()
}

dependencies {
    testCompile group: 'junit', name: 'junit', version: '4.12'

    // https://mvnrepository.com/artifact/com.google.protobuf/protobuf-java
    compile group: 'com.google.protobuf', name: 'protobuf-java', version: '3.10.0'

    // https://mvnrepository.com/artifact/io.grpc/grpc-all
    compile group: 'io.grpc', name: 'grpc-all', version: '1.25.0'
}

protobuf {
    protoc {
        artifact = 'com.google.protobuf:protoc:3.10.1'
    }
}
```

## Tell protoc to use gRPC plugin

Now, a very important thing that we should do is to tell `protoc` to use the gRPC plugin when generating Java code. We can do that by getting back to the github page of `protobuf-gradle-plugin` and look for this plugins block.

![Tell protoc to use gRPC plugin](https://dev-to-uploads.s3.amazonaws.com/i/emq0iz47lfs0ssswqswf.png)

There's an artifact for `protoc-gen-grpc-java`, we can search for its latest version on the [maven repository](https://mvnrepository.com/artifact/io.grpc/protoc-gen-grpc-java).

It's 1.25.0 in my case, so let's copy & paste the config block to `build.gradle` file, and change the version to 1.25.0:


```gradle
plugins {
    id "com.google.protobuf" version "0.8.10"
    id "java"
}

group 'com.gitlab.techschool'
version '1.0-SNAPSHOT'

sourceCompatibility = 1.8

repositories {
    mavenCentral()
}

dependencies {
    testCompile group: 'junit', name: 'junit', version: '4.12'

    // https://mvnrepository.com/artifact/com.google.protobuf/protobuf-java
    compile group: 'com.google.protobuf', name: 'protobuf-java', version: '3.10.0'

    // https://mvnrepository.com/artifact/io.grpc/grpc-all
    compile group: 'io.grpc', name: 'grpc-all', version: '1.25.0'
}

protobuf {
    protoc {
        artifact = 'com.google.protobuf:protoc:3.10.1'
    }

    plugins {
        grpc {
            artifact = 'io.grpc:protoc-gen-grpc-java:1.25.0'
        }
    }

    generateProtoTasks {
        all()*.plugins {
            grpc {}
        }
    }
}
```

We should also add the `generateProtoTasks` settings to guide `protoc` to use the gRPC plugin.

I know it looks complicated and there are too many stuffs, but it will help us speed up a lot in the development phase.

## Where the generated codes are located

There's one more thing left before we're good to go, that's to tell IntelliJ IDEA where our generated codes will be located. That way it can easily and correctly do code analysis and code suggestion for us later.

To do that, we use the `sourceSets` block. And inside this block, we specify 2 source directories: 1 for gRPC, and the other for normal protobuf messages:

```gradle
plugins {
    id "com.google.protobuf" version "0.8.10"
    id "java"
}

group 'com.gitlab.techschool'
version '1.0-SNAPSHOT'

sourceCompatibility = 1.8

repositories {
    mavenCentral()
}

dependencies {
    testCompile group: 'junit', name: 'junit', version: '4.12'

    // https://mvnrepository.com/artifact/com.google.protobuf/protobuf-java
    compile group: 'com.google.protobuf', name: 'protobuf-java', version: '3.10.0'

    // https://mvnrepository.com/artifact/io.grpc/grpc-all
    compile group: 'io.grpc', name: 'grpc-all', version: '1.25.0'
}

sourceSets {
    main {
        java {
            srcDirs 'build/generated/source/proto/main/grpc'
            srcDirs 'build/generated/source/proto/main/java'
        }
    }
}

protobuf {
    protoc {
        artifact = 'com.google.protobuf:protoc:3.10.1'
    }

    plugins {
        grpc {
            artifact = 'io.grpc:protoc-gen-grpc-java:1.25.0'
        }
    }

    generateProtoTasks {
        all()*.plugins {
            grpc {}
        }
    }
}
```

OK, now we are really good to go!

## Generate Java codes

Let's create a new proto folder inside `src/main`. Then copy all the proto files that we have written in previous lectures to this folder.

![Copy proto files](https://dev-to-uploads.s3.amazonaws.com/i/0vk6dmqfid9smqvk4f30.png)

Now comes the interesting part. Once we click the Build icon, IntelliJ IDEA will start some background tasks to generate codes for us. When they finished, we can find the generated codes in this folder:

```bash
build/generated/source/proto/main/java
```

The `grpc` folder is empty now because we haven't written any RPC yet. In the `java` folder, there are 6 Java files, one for each of the messages.

![Generated Java code](https://dev-to-uploads.s3.amazonaws.com/i/t276otoqssmm8kr64h3n.png)

There are a lot of codes in 1 single file. And the generated package name is the same as the protobuf package: `techschool.pcbook`. We can easily change this package name similar to what we did for Go by this setting:

```protobuf
option java_package="com.gitlab.techschool.pcbook.pb"
```

We can also tell protoc to split the codes into smaller files  instead of putting them inside 1 single big file. It's pretty simple, just set:

```protobuf
option java_multiple_files = true;
```

These options should be added to all proto files. For example, the `laptop_message.proto` file will be like this:

```protobuf
syntax = "proto3";

package techschool.pcbook;

option go_package = "pb";
option java_package = "com.gitlab.techschool.pcbook.pb";
option java_multiple_files = true;

import "processor_message.proto";
import "memory_message.proto";
import "storage_message.proto";
import "screen_message.proto";
import "keyboard_message.proto";
import "google/protobuf/timestamp.proto";

message Laptop {
    string id = 1;
    string brand = 2;
    string name = 3;
    CPU cpu = 4;
    Memory ram = 5;
    repeated GPU gpus = 6;
    repeated Storage storages = 7;
    Screen screen = 8;
    Keyboard keyboard = 9;
    oneof weight {
        double weight_kg = 10;
        double weight_lb = 11;
    }
    double price_usd = 12;
    uint32 release_year = 13;
    google.protobuf.Timestamp updated_at = 14;
}
```

Now if I delete the old generated codes and rebuild the project, there will be many java files, and the package name will be changed to what we want.

![Multiple Java files and custom package name](https://dev-to-uploads.s3.amazonaws.com/i/bbgv3qyryexq10plpmmb.png)

And that's it! We're done! 

In the next lecture, we will start writing codes in Go and Java to serialise protobuf messages to binary and JSON.

Thank you for reading and see you then!

---

If you like the article, please [subscribe to our Youtube channel](http://bit.ly/techschoolguru) and [follow us on Twitter](https://twitter.com/techschoolguru) for more tutorials in the future.

---
If you want to join me on my current amazing team at Voodoo, check out [our job openings here](https://bit.ly/voodoojobs). Remote or onsite in Paris/Amsterdam/London/Berlin/Barcelona with visa sponsorship.