# Julia meets the FPGA: Higher-level synthesis methodology for heterogeneous hardware and software architectures

Gaëtan Lounes, Robin Gerzaguet, Matthieu Gautier

## ► To cite this version:

Gaëtan Lounes, Robin Gerzaguet, Matthieu Gautier. Julia meets the FPGA: Higher-level synthesis methodology for heterogeneous hardware and software architectures. Conference on the Julia programming language (JuliaCon), Jul 2024, Eindhoven, Netherlands. pp.1, 2024, 10.1007/11532378_2. hal-04907355

HAL Id: hal-04907355

https://hal.science/hal-04907355v1

Submitted on 22 Jan 2025

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
## Introduction

Julia language has acknowledged great results in the heterogeneous compilation place due to the flexibility of the compilation toolchain and thanks to extension of this compiler. For instance GPU and IPU can be targeted smoothly. FPGA are another class of interesting target which has been little-explored [1], they aim to operate in energy efficient and high throughput applications. **We propose Judias: A Julia MLIR frontend for HLS.**

## Glossary

**AST:** Abstract Syntax Tree

**Brutus:** Brutus is a research project to bring MLIR to Julia (https://github.com/JuliaLabs/brutus)

**Clang:** LLVM backend C-like compiler (https://clang.llvm.org/)

**DSE:** Design Space Exploration

**DSL:** Design Specific Language

**FPGA:** Field Programmable Gate Array

**HDL:** Hardware Description Language

**HLS:** High Level Synthesis

**IR:** Intermediate Representation

**LLVM:** Low Level Virtual Machine, State of the art compiler backend (https://llvm.org/) [2]

**MLIR:** Multi Layer IR (https://mlir.llvm.org/) [3]

**SSA:** Static Single Assignment

## Julia compiler heterogeneous hardware usages

To support new target, the GPUCompiler.jl¹ package is used. It reuses the plain Julia compiler up to the translate step. Then in the backend, each target defines a new `LLVM` pipeline for their specific needs in order to generate specialized `LLVMIR`.

<figure><img src="image_2.png" /><figcaption>Figure 1: Simplified Julia toolchain using GPUCompiler.jl</figcaption></figure>

## HLS

FPGA are configured using `HDL` language such as VHDL or Verilog. But this form is not ideal because it requires solid hardware knowledge and algorithm development is unwieldy. `HLS` have been introduced to overcome these limitations. State of the art, `HLS` tools, such as Vitis from AMD, uses a `DSL` based on C (using a custom `Clang`) which is then transformed to a `LLVMIR` containing various annotations to model hardware semantics. The closed-source nature of this `IR` usage and semantic make its integration with new frontend cumbersome.

<figure><img src="image_3.png" /><figcaption>Figure 2: Simplified HLS toolchain of Vitis</figcaption></figure>

## A new compiler framework: MLIR

An alternative method used by many `HLS` users doesn't use the `DSL` or `LLVM IR` directly; they introduced a new toolchain using `MLIR` [3]. The idea behinds `MLIR` is to generalize the concept of `IR` and ways to used and mutated it. They introduced new `IR` (called dialects in `MLIR` terminology) which are analyzed and changed to fit an `IR` which is then sent to `HLS` tools.

### MLIR key principles

* Expressively: Dialect / Attributes / Operator / Type / Region

* Reusability: Traits / Pass / Compiler

* Transformable: Progressive Lowering / Rewrite Patterns

## Brutus: a MLIR Julia backend

> "Brutus" is a research project that uses MLIR to implement code-generation and optimisations for Julia."
> 
> **Brutus** backend has some singularities:

* Julia `IR` is the input of `Brutus`. Contrary to GPUCompiler.jl which process `LLVMIR`.

* Aim to improve backend compilations pass.

<figure><img src="image_4.png" /><figcaption>Figure 3: A simple Brutus toolchain</figcaption></figure>

## Conclusion

Julia can be used with `FPGA` through `HLS`. Notably, Julia `IR` singularities (typed `IR` and compactness) are brought to `MLIR` using the powerness of Julia compilation.

One major extend is to bring a `DSE` into the playground: using Julia interpreter capacities and `MLIR` lively semantics to achieve a rich and fast design space analysis. One idea is to exploit Graph Neural Network on `MLIR` representation.

## Judias

In a similar manner as Brutus prototype tool, we propose using Julia `IR` as `MLIR` frontend.

The key difference lies in the usage of `MLIR`, it's not for improving backend compilation performance but to be used as a frontend of a `HLS` user. In this case: ScaleHLS [4]. It is a `HLS` framework which uses `MLIR` extensively to represent and transform `HLS` designs at different levels of abstraction. Ultimately, the final representation created by the framework is transformed in order to generate code usable by Vitis.

<figure><img src="image_5.png" /><figcaption>Figure 4: Judias toolchain</figcaption></figure>

## Julia compiler example

Listing 1 shows a simple Julia function containing arithmetic, local definition and a for loop. This program goes through the Julia toolchain and Listing 2 is the resulting form just before translate to `LLVM` form.

<figure><img src="image_6.png" /><figcaption>Listing 1: a Julia program</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Listing 2: IR generated</figcaption></figure>

### Julia IR remarks

* Julia `IR` contains standard structure (mainly shared with `LLVM`):

* SSA

* phinode

* basic blocks

* strongly typed

* Explicit loop information is erased

* Non-standard compilation toolchain: Inliner is changed to keep track of some functions calls.

## Judias usage

The powerful framework `MLIR` enables the write of a dialect for Julia. Julia `IR` is first changed to fit with this dialect:

* phinodes are replaced by block with arguments

* Julia type and operation behaviors are mapped to the `MLIR` dialect

* instructions are translated to `MLIR` operations

Listing 3 shows the result of passes which was executed to transform the dialect `IR`, for instance to retrieve the spatial loop informations and to canonicalize the `IR`.

<figure><img src="image_8.png" /><figcaption>Listing 3: MLIR using JIR custom dialect</figcaption></figure>

<figure><img src="image_9.png" /></figure>

### Judias remarks

* Loop spatial information is retrieved during rising pass and is explicitly defined in the dialect

* The JIR dialect contains runtime information from Julia interpreter such as types or function call

## To standard dialect and Hardware

Listing 3 can be lowered to standard dialect (Affine, Arith, Function, Index) to obtain Listing 4, then in this form it can be provided to the `HLS` framework. ScaleHLS and then Vitis `HLS` generated a `HDL` representation using Table 1 ressources and which can be represented as Figure 5.

<figure><img src="image_10.png" /><figcaption>Listing 4: MLIR using standard dialect</figcaption></figure>

<table><caption>Table 1: RTL ressource usage</caption><thead><tr><td>Ressource</td><td>Quantity</td></tr></thead><tbody><tr><td>LUT</td><td>228</td></tr><tr><td>FF</td><td>69</td></tr><tr><td>DSP</td><td>4</td></tr><tr><td>BRAM</td><td>2</td></tr><tr><td>URAM</td><td>0</td></tr></tbody></table>

<figure><img src="image_11.png" /><figcaption>Figure 5: final RTL view from Vivado</figcaption></figure>

## Bibliography

[1] B. Biggs, I. McInerney, E. C. Kerrigan, and G. A. Constantinides, "High-Level Synthesis Using the Julia Language," no. arXiv:2201.11522. arXiv, Feb. 2022.

[2] C. Lattner and V. Adve, "The LLVM Compiler Framework and Infrastructure Tutorial," in *Languages and Compilers for High Performance Computing*, R. Eigenmann, Z. Li, and S. P. Midkiff, Eds., in Lecture Notes in Computer Science. Berlin, Heidelberg: Springer, 2005, pp. 15-16. doi: 10.1007/11532378_2.

[3] C. Lattner et al., "MLIR: A Compiler Infrastructure for the End of Moore's Law," no. arXiv:2002.11054. arXiv, Feb. 2020.

[4] H. Ye et al., "ScaleHLS: A New Scalable High-Level Synthesis Framework on Multi-Level Intermediate Representation," no. arXiv:2107.11673. arXiv, Dec. 2021.

<figure><img src="image_12.png" /></figure>

<figure><img src="image_13.png" /></figure>

<figure><img src="image_14.png" /></figure>