<figure><img src="image_1.png" /></figure>

HAL
open science

# An Open-Source Framework for Efficient Numerically-Tailored Computations

Louis Ledoux, Marc Casas

► To cite this version:

Louis Ledoux, Marc Casas. An Open-Source Framework for Efficient Numerically-Tailored Computations. BSCSymposium23, May 2023, Barcelona (Online), Spain. 10th International BSC Severo Ochoa Doctoral Symposium 2023, 2023. [hal-04094835](https://hal.science/hal-04094835)

HAL Id: hal-04094835

https://hal.science/hal-04094835

Submitted on 11 May 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# AN OPEN-SOURCE FRAMEWORK FOR EFFICIENT NUMERICALLY-TAILORED COMPUTATIONS

Louis Ledoux¹,² and Marc Casas¹,²

¹Barcelona Supercomputing Center
²Universitat Politècnica de Catalunya

## 1. Scientific HPC: The Mathematical Aspect

* Many scientific problems are reduced to Matrix-Matrix Multiplications (MMM)
* Arouse the interest in GEMM kernels present in the BLAS libraries

### Contrasting Numerical Requirements

* Ill-conditioned linear systems require high-precision arithmetic to ensure correct and reproducible results [1].
* Emerging workloads such as DNNs, which can have millions up to billions of parameters, have shown resilience to arithmetic tinkering [2] and precision lowering [3].

### Current Limitations

* General purpose arithmetic units and computer formats such as the IEEE754 standard naturally underperform in this vast land of scenarios
* We propose the generation of numerically tailored circuits where the necessary and sufficient internal precision is generated to improve numerical quality while reducing the energy cost

## 2. Open Source SW/HW Co-designed Framework

* Our framework [4] can be used with any software language. Below is depicted how its two phases work
  **Runtime Execution Flow**
  * 1 malloc(): Allocates space for matrices to be multiplied
  * 2 gemm(): BLAS kernels calls, either directly, or via higher-level code such as pytorch
  * 3 cast(): Convert computer formats and pad matrices if required
  * 4 DMA(): performs virtual addresses translations; runs at 20GB/s; uses shared memory
  **A priori Hardware Generation Flow**
  * A make image: With a set of functional and performance specifications calls all following steps
  * B flopoco [5]: Automated Pipelined datapath of custom arithmetic operators for any (frequency, target) couples; Design of custom Fused-Dot-Product (FDP) unit
  * C Systolic Array: Local and Distributed control; HSSD (Half Speed Sink Down) mechanism to allow overlaps of in- and out-going matrices to saturate OpenCAPI link
  * D AXI-Stream: Wrapping with well known buses/protocols to communicate with HOST

## 3.A. Evaluation: Sea Surface Height Computation

<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /></figure>

<figure><img src="image_5.png" /></figure>

## 3.B. Evaluation: Artificial Intelligence

<figure><img src="image_6.png" /></figure>

### Results

* Our 91-bit $\langle ovf : 30, msb : 30, lsb : 30 \rangle$ FDP always maintains reproducibility unlike IEEE754-64 and -128 FMAs.
* Our proposed FDP consistently exhibits 52 correct bits, which is at least 5× and 27.7× more than quad-precision and double-precision
* On VU3P-2 FPGA at 200MHz, the units draw 0.266, 0.549, and 0.491 Watts for the IEEE754-64, -128 FMAs, and the 91-bit FDP, respectively
* For all evaluated sizes, the 91-bit FDP yields at least 5.6× and 15.1× more correct bits for the same wattage as 128-bit and 64-bit FMAs, respectively.

<figure><img src="image_7.png" /></figure>

## 4. References

* [1] D.Bailey et al. "High-Precision Computation and Mathematical Physics", PoS(ACAT08).
* [2] J.Johnson "Rethinking floating point for deep learning" arXiv2018
* [3] M.Courbariaux et al. "Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1" arXiv2016
* [4] L.Ledoux et al. "A Generator of Numerically-Tailored and High-Throughput Accelerators for Batched GEMMs" FCCM2022
* [5] M.Istoan et al. "Automating the pipeline of arithmetic datapaths" DATE2017

### Acknowledgements

* Marc Casas is supported by Grant RYC-2017-23269 funded by MCIN/AEI/ 10.13039/501100011033 and by "ESF Investing in your future".

### Biography

<figure><img src="image_8.png" /></figure>

Louis Ledoux received his BSc degree in 2016 in Computer Science from Université de Rennes1, France. The following years, he pursued his MSc degree in parallel with an Engineer diploma from École Supérieure d'Ingénieurs de Rennes (ESIR). He concluded in 2018 his studies in Rennes with a position of Hardware Engineer at b<>com, a national research laboratory. This position allowed him to experiment with the first FPGAs in the cloud and their virtualizations. Since 2018, he has been a PhD candidate at the Computer Architecture departments of Barcelona Supercomputing Center (BSC) and Universitat Politècnica de Catalunya (UPC), Spain.

## 5. Conclusions

* Our work highlights the importance of numerically tailored accumulators for reproducibility
* Our results provide valuable insights into the trade-offs between power consumption and accuracy, and we believe that our results have the potential to inform the design of scientific computing systems
* In light of this, we encourage other researchers to explore the possibilities of custom precision accumulators using our open-source framework