Gradle Remote Cache with Redis and Circle CI
By: Iñaki Villar
Tags: Redis, Gradle, Ci

I would say Remote Cache is one of the top features to optimize the build speed across the Development Team using Gradle as Build System.

Today, we have different solutions and resources helping to create, manage and implement Remote Caching. Gradle Enterprise Build Cache is the most powerful solution, offering a remote and replicated cache, we are using it at Tinder and we are thrilled with the results in a distributed team.

There are other solutions like Remote Cache in AWS S3 (here and here), or you can implement in your custom infrastructure with the official Docker image for a remote Node. For targeting specific caching systems like Redis, you also have different plugins (here and here). And from the Gradle perspective, you can extend the build cache functionality with the BuildCacheConfiguration API.

These days of massive working from home, new scenarios have raised due to this new reality. Nelson Osacky is writing an excellent series of posts about the trade-of of using remote caching with slow connections

But this article is just a quick test to compare for a specific Cache System(Redis), which Kubernetes infrastructure provider(EKS/GKE) suits better for a Circle CI.

Simple Local Example

To validate the idea, we are going to use a simple project and Redis. We can create a Redis instance locally with Docker:

docker run --name redis-build-cache -d redis

For communication with Redis, we will use one of the plugins mentioned before(https://github.com/tehlers/gradle-redis-build-cache). In your Gradle project, you need to include the plugin:

And then, set up the RedisBuildCache Service in the buildCache configuration:

Finally, we can execute the target task adding the parameter --scan to retrieve the build scan. After the second execution, we double-check in the build scan generated the correct configuration(performance > build cache):

Experimenting with K8s Providers

Now that we know the local configuration works, it’s time to move to the cloud.

Project

We are going to use a fork of Tivi, a project by Chris Banes to use as a showcase of different features of Android. The main debug task contains 30 projects and 903 tasks.

CI Configuration

Once we have forked the project, we have to set up the CircleCI configuration. We will create a new configuration file ( .circleci/config.yml ), including the main debug task:

- run:

name: build

command: ./gradlew app:assembleDebug --scan

We are using the --scan parameter to generate the build scan like the local example. Don’t forget to add the buildScan configuration block to add the license acceptance.

Deploying Redis Instance

We are going to use a Redis Instance in Kubernetes with EKS and GKE. The creation of the cluster is out of the scope of this article, but in case you need some guidance, for GKE you have:

And for EKS, I find that the Bitnami article is one of the best resources:

When you create a cluster, you need to choose a zone/region. To have better results, we need to select according to the CircleCI infrastructure. They mentioned on the website:

CircleCI builds are currently run mainly from AWS East and West as well as Google Cloud Platform East.

In this example, we are going to base both infrastructures in the same zones (us-east).

Once you have your cluster up, it’s the moment to deploy the Redis instance. You can use the charts provided by Helm or the option that suits in your cluster. In case you want to apply something quick, you can execute in your cluster:

The last step is going to provide a public IP, this IP it’s linked with the Redis instance. To get the IP, you need to execute kubectl get svc :

Once the IP is ready(for GKE is not immediate), you can proceed to update the build.gradle configuration with the new URL in the same configuration explained in the previous point.

And that it is! Now is time to trigger the builds in Circle CI and collect the results.

Results

GKE

Performance > Task Execution:

Performance > Build Cache