Migrating to Bazel from Maven or Gradle? Part 3 — How to optimize local dev experience
By: Natan Silnitsky
Tags: Docker, Windows, Mac, Bazel

Bazel’s abilities vary considerably on different operating systems, with Linux being superior to windows/MacOs.

In the following sections, I will describe how crucial bazel features for fast and reproducible builds are implemented on the different OSs.

I will also describe the trade-offs of a virtualization solution for local development with Bazel.

Reproducible Builds with Sandboxing

One of the most important features of Bazel is sandboxing.

Sandboxing allows for hermetic builds and safe caching of build outputs. It restricts access to files that were not explicitly defined in the build configuration. It also restricts access to the outside network.

Bazel implement sandboxing in Linux using User Namespaces that offers kernel support for file system and network isolation, and on macs using a feature called “SeatBelt” (sandbox-exec).

There is currently no windows support for Sandboxing.

There is an experimental sandboxing feature called sandboxfs that offers platform independent sandboxing that utilizes the FUSE file system. Preliminary performance results show faster performance for certain use cases, but it is still very early days for sandboxfs.

Parallel Tests

Another important feature of bazel is that it runs test actions in a highly parallel fashion.

Thus your integration tests that contain network requests must choose random ports or otherwise have to run in isolation from one another.

Bazel solves the isolation requirement using the same sandboxing techniques mentioned above.

While working well on Linux, port isolation is not supported on macOS. Even worse — binding to localhost is not supported on mac sandbox.

No matter which platform you’re running on, expect to have to adapt your code for sandboxing (e.g. stop using InetAddress.getLocalHost).

Remote Caching

A very cool feature bazel has is the possibility to locally utilize a remote cache (populated by servers or other developers). This way you never have to run a clean build, even on a brand new computer!

Unfortunately if your CI server are running on Linux agents, you will not be able to utilize the CI cache results on macs or windows, only on Linux machines. This is because bazel considers the platform as part of the hash function input for caching build targets.

Nevertheless you can still create another remote cache server for your local development environment. You need to decide whether to populate it using a dedicated CI server or developer machines or both.

There is a danger of cache poisoning in case of cache population from developer machines running bazel with default configurations. For instance a machine with a different JDK version can send incorrect builds results to the shared remote cache.

You can setup a specific jdk installation link using special bazel flags, but the best way to avoid surprises is to employ strict configuration management (like jamf for macs) on all dev machines.

Remote Execution

If you would like to employ remote execution, i.e. send build actions to a remote workers farm based on Linux, you have to be running locally on the same platform, in this case, Linux.

Even on Linux, running build actions remotely can have a considerable latency overhead that may outweigh the benefits of a larger degree of parallelization. Google remote execution service currently only supports Linux.

A new and experimental feature of Bazel called dynamic scheduling can help you enjoy both worlds by sending the actions to the remote execution service while simultaneously running the actions locally, It will cancel the slower action once the first one finishes.

IDEs

There is a bazel plugin for both intellij and Eclipse. It delegates the work to bazel by invoking it in command line, so these plugins are platform-independent.

But there are known debugging issues for Intellij bazel plugin for windows.

On the one hand, the delegation of work to bazel cli by the plugin insures correctness and a single source of truth. But on the other hand, it requires to call bazel build every time a developer wants to update Intellij knowledge of the code structure.

On macs, setting this sync to be automatic will sometimes stop the developer from being able to work in intellij for the duration of the build. The upcoming Bazel 0.24 which will utilize Darwin’s QoS service may resolve this issue.