Episode 2: “The Holy Grail” — GraalVM (Building “Super Optimum Microservices Architecture” Series)
By: A B Vijay Kumar
Tags: Microservices, Java, Containers, Graalvm, Cloud Computing

Graal & Truffle

Truffle is a language interpreter framework. It's a framework, that the language programmers implement and let their languages run on GraalVM, leveraging the optimizations that GraalVM provides.

To understand this better, if I am a language programmer, what I need to do is to write an interpreter using “Tuffle Language Implementation Framework”, and package them as components, that I can run on GraalVM, not worrying about writing complicated optimizations. GraalVM will take care of running the code optimally. I can choose to compile “Ahead-Of-Time” to Graal Native Image or “Just-In-Time”.

Truffle also provides a framework called “Truffle Instrument API” for building tools. Instruments provide fine-grained VM level runtime events, which can be used to built profiling, tracing analyzing, debugging tools. The best part is, if you have your language interpreters written with Truffle, you can use the ecosystem of Truffle instruments already built. (example VisualVM, Chrome Debugger, GraalVM Visual Studio Code Extension etc)

Graal JIT

Graal can compile the code Just-In-Time (JIT, like C2) or Ahead of Time (AOT) directly to native image.

There is a general myth that AOT is faster, which is very true in the first few runs, but there is a possibility that the JIT might outperform the AOTs, as JIT is constantly optimizing (Graal VM/C2) based on the feedback it gets from profiling. JIT normally has a larger footprint, than AOT.

For Serverless — it makes more sense to go towards AOT, while for long-running container-based/VM based deployments, JIT might make more sense.

The diagram below shows how the Graal JIT compiler works (you might find it very close to C2 compilers in what we covered in Episode 1

Graal flow

Abstract Syntax Tree is the intermediate representation. AST provides a very optimum way to represent the syntactic structure of the language, where typically the parent node is the operator, and the children node represents the operands or operators (based on cardinality).

In this statement, a, b, c can be any variables (for loosely typed languages). The interpreter starts assuming “Generics”, based on the profiling, of various executions, it starts assuming the specifics & then will optimize the code, using partial evaluation.

Partial Evaluation & De-Optimization: Truffle (language interpreters written with Truffle) runs as an interpreter, and Graal JIT kicks in to start identifying optimizations in the code. Here are some of the optimizations

Inlining : The compiler identifies small methods, and inline them into the code. This makes this method calls inexpensive.

: The compiler identifies small methods, and inline them into the code. This makes this method calls inexpensive. Escape Analysis : The compiler detects local variables, and allocates them in CPU registers for faster access.

: The compiler detects local variables, and allocates them in CPU registers for faster access. Dead Code: JIT only compiles the code that is used.

The above optimizations are based on speculation, and eventually, if the speculation is proven wrong at runtime, the JIT will re-optimize and recompile (as shown in the diagram above). Re-optimization & recompiling is an expensive task.

Partial Evaluation creates an intermediate representation of the language, from the code and the data, and as it learns, and identifies new data types, it will deoptimize to AST Interpreter, applies optimizations, and does the node-rewriting and recompiles. After a certain point, it would have the most optimum representation.

Illustration of how Graal works

Graal AOT

Graal Ahead of Time compilation is a very powerful way to create native images, for a particular targeted OS/Architecture. For cloud-native workloads and serverless, this is a very powerful option, for lower footprint, faster startups, and more importantly embeddable runtimes (providing immutability). Imagine you Tekton/Jenkins pipelines build your code with this option, you already get a super optimum native image (assuming we are running them on a specific container runtime platform — which in most cases is built on Linux)

Graal AOT

Graal Polyglot Interoperability

GraalVM supporting and allowing applications written in multiple languages, it also provides ways to pass data between these applications, that are written using different languages. Truffle provides “Polyglot Interoperability Protocol”. The Interoperability protocol defines the message, that each language needs to implement, to support the passing of data between the polyglot applications.

Sulong

Sulong is a high-performance LLVM bitcode runtime that is built on Graal VM. Sulong uses Truffle and Graal as dynamic compiler.

LLVM is an open-source project which is a collection of modular, reusable compilers and toolchains. There are a lot of languages (C, C++, Fortran, Rust, Swift etc) compilers that are built on LLVM, where LLVM provides the intermediate representation (LLVM-IR).

Sulong pipeline looks different from what we looked at other language compilers, that are running on Truffle., please refer to the diagram above, on how C/C++ code gets compiled.