# Multifacets of lossy compression for scientific data in the Joint-Laboratory of Extreme Scale Computing

Franck Cappello, Sheng Di, Robert Underwood, Dingwen Tao, Jon Calhoun, Yoshii Kazutomo, Kento Sato, Amarjit Singh, Luc Giraud, Emmanuel Agullo, et al.

► To cite this version:

Franck Cappello, Sheng Di, Robert Underwood, Dingwen Tao, Jon Calhoun, et al. Multifacets of lossy compression for scientific data in the Joint-Laboratory of Extreme Scale Computing. Future Generation Computer Systems, 2024, 10.1016/j.future.2024.05.022. hal-04618060

HAL Id: hal-04618060

https://inria.hal.science/hal-04618060

Submitted on 19 Jun 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Multifacets of lossy compression for scientific data in the Joint-Laboratory of Extreme Scale Computing

Franck Cappello, Sheng Di, Robert Underwood, Dingwen Tao, Jon Calhoun, Yoshii Kazutomo, Kento Sato, Amarjit Singh, Luc Giraud, Emmanuel Agullo, et al.

► To cite this version:

Franck Cappello, Sheng Di, Robert Underwood, Dingwen Tao, Jon Calhoun, et al. Multifacets of lossy compression for scientific data in the Joint-Laboratory of Extreme Scale Computing. Future Generation Computer Systems, 2024, 10.1016/j.future.2024.05.022. hal-04618060

HAL Id: hal-04618060

https://inria.hal.science/hal-04618060v1

Submitted on 19 Jun 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Journal Pre-proof

Multifacets of lossy compression for scientific data in the
Joint-Laboratory of Extreme Scale Computing

Franck Cappello, Sheng Di, Robert Underwood, Dingwen Tao,
Jon Calhoun, Yoshii Kazutomo, Kento Sato, Amarjit Singh,
Luc Giraud, Emmanuel Agullo, Xavier Yepes, Mario Acosta, Sian Jin,
Jiannan Tian, Frédéric Vivien, Boyuan Zhang, Kentaro Sano,
Tomohiro Ueno, Thomas Grützmacher, Hartwig Anzt

<figure><img src="image_2.png" /></figure>

PII: S0167-739X(24)00255-3

DOI: https://doi.org/10.1016/j.future.2024.05.022

Reference: FUTURE 7323

To appear in: *Future Generation Computer Systems*

Received date : 10 January 2024

Revised date : 9 May 2024

Accepted date : 14 May 2024

Please cite this article as: F. Cappello, S. Di, R. Underwood et al., Multifacets of lossy compression for scientific data in the Joint-Laboratory of Extreme Scale Computing, *Future Generation Computer Systems* (2024), doi: https://doi.org/10.1016/j.future.2024.05.022.

This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

© 2024 Published by Elsevier B.V.
Journal Pre-proof

Multifacets of lossy compression for scientific data in the
Joint-Laboratory of Extreme Scale Computing

Franck Cappello, Sheng Di, Robert Underwood, Dingwen Tao,
Jon Calhoun, Yoshii Kazutomo, Kento Sato, Amarjit Singh,
Luc Giraud, Emmanuel Agullo, Xavier Yepes, Mario Acosta, Sian Jin,
Jiannan Tian, Frédéric Vivien, Boyuan Zhang, Kentaro Sano,
Tomohiro Ueno, Thomas Grützmacher, Hartwig Anzt

<figure><img src="image_2.png" /></figure>

PII: S0167-739X(24)00255-3

DOI: https://doi.org/10.1016/j.future.2024.05.022

Reference: FUTURE 7323

To appear in: *Future Generation Computer Systems*

Received date : 10 January 2024

Revised date : 9 May 2024

Accepted date : 14 May 2024

Please cite this article as: F. Cappello, S. Di, R. Underwood et al., Multifacets of lossy compression for scientific data in the Joint-Laboratory of Extreme Scale Computing, *Future Generation Computer Systems* (2024), doi: https://doi.org/10.1016/j.future.2024.05.022.

This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

© 2024 Published by Elsevier B.V.
# Multifacets of lossy compression for scientific data in the Joint-Laboratory of Extreme
Scale Computing

Franck Cappello, Mario Acosta, Emmanuel Agullo, Hartwig Anzt, Sheng Di, Luc Giraud, Thomas Grützmacher, Sian Jin, Kento Sato, Amarjit Singh, Dingwen Tao, Jiannan Tian, Tomohiro Ueno, Robert Underwood, Frédéric Vivien, Xavier Yepes-Arbós, Kazutomo Yoshii, Boyuan Zhang

## Abstract

The Joint Laboratory on Extreme-Scale Computing (JLESC) was initiated at the same time lossy compression for scientific data became an important topic for the scientific communities. The teams involved in the JLESC played and are still playing an important role in developing the research, techniques, methods, and technologies making lossy compression for scientific data a key tool for scientists and engineers. In this paper, we present the evolution of lossy compression for scientific data from 2015, describing the situation before the JLESC started, the evolution of this discipline in the past 8 years (until 2023) through the prism of the JLESC collaborations on this topic and some of the remaining open research questions.

## 1. Introduction

In this paper, we will describe the compression work mainly developed under the collaborations of JLESC organizations, including Argonne National Laboratory, Barcelona Supercomputing Center (BSC), National Center for Supercomputing Applications (NCSA), Rikagaku Kenkyusho Institute (RIKEN), National Institute for Research in Digital Science and Technology (INRIA), Juelich Supercomputing Center, and the University of Tennessee. The paper presents a collection of compression research in many different scientific domains and several use cases. While more details about the presented works can be found in the corresponding publications, we believe that the overview presented in this paper will introduce readers to the breadth of lossy compression applicability to scientific data. It should be noted that the paper does not cover all scientific domains and possible use cases and, in fact, as the past height years show, the application of lossy compression for scientific data is rapidly expanding to more scientific domains and more use cases.

### 1.1. Before 2015

Before 2015, scientific data reduction was initially limited to a few specific use cases where generated data was exceeding—sometimes by several orders of magnitude—the communication, storage, and analysis capabilities of that time. The first use case was experiments producing critical data for scientific discoveries. Examples of such experiments are the detectors at the Large Hadron Collider (LHC) [52] and the NASA Mars rovers [87]. In the case of the LHC, detectors are equipped with real-time analysis systems to select the most interesting proton-proton (pp) collisions and reduce the data by up to 1/100000 [52]. For the NASA Mars rovers, the communication bandwidth is constrained by the deep-space communications channel between Mars and Earth [87]: for the Curiosity rover, the

data rate direct-to-Earth was 32 kb/s, and the data rate through the Odyssey orbiter was 256 kb/s. To reduce image data, NASA Mars rovers used different image compressors: ICER [110] and JPEG [20]. The second main use case was a visualization of scientific data [58, 96, 96]. The need for lossy compression for scientific visualization was identified in the 1980s [2]. For many other scientific experiments and simulations, data reduction was unnecessary: data resolution and generation rate of instrument detectors and numerical simulation were compatible with contemporary communication and storage technologies. The interest in a broader use of scientific data reduction and compression increased circa 2000, with compression schemes such as ISABELA [90], ISOBAR [130], FPC [24], and fpzip [104].

### 1.2. From 2015 to 2023

The first successful compressor for scientific data was ZFP from Lawrence Livermore National Laboratory. The first ZFP paper was published in December 2014 [103]. A few months later and independently, the SZ project started at Argonne National Laboratory, and the first version of SZ was published in June 2016 [36]. The rapid success of these two compressors, combined with a realization from the broader scientific community of the necessity to reduce data effectively, inspired many other teams to develop scientific data compression schemes. One difficulty for the early adopters of lossy compressors was to understand the performance of the different compressors from the published papers. Many publications were using different datasets and different metrics to showcase the better performance of their compression schemes. This situation created a problem concerning the scientific methodology: how to fairly compare different compression schemes if comparisons do not use community-established common datasets and metrics. The
# Multifacets of lossy compression for scientific data in the Joint-Laboratory of Extreme
Scale Computing

Franck Cappello, Mario Acosta, Emmanuel Agullo, Hartwig Anzt, Sheng Di, Luc Giraud, Thomas Grützmacher, Sian Jin, Kento Sato, Amarjit Singh, Dingwen Tao, Jiannan Tian, Tomohiro Ueno, Robert Underwood, Frédéric Vivien, Xavier Yepes-Arbós, Kazutomo Yoshii, Boyuan Zhang

## Abstract

The Joint Laboratory on Extreme-Scale Computing (JLESC) was initiated at the same time lossy compression for scientific data became an important topic for the scientific communities. The teams involved in the JLESC played and are still playing an important role in developing the research, techniques, methods, and technologies making lossy compression for scientific data a key tool for scientists and engineers. In this paper, we present the evolution of lossy compression for scientific data from 2015, describing the situation before the JLESC started, the evolution of this discipline in the past 8 years (until 2023) through the prism of the JLESC collaborations on this topic and some of the remaining open research questions.

## 1. Introduction

In this paper, we will describe the compression work mainly developed under the collaborations of JLESC organizations, including Argonne National Laboratory, Barcelona Supercomputing Center (BSC), National Center for Supercomputing Applications (NCSA), Rikagaku Kenkyusho Institute (RIKEN), National Institute for Research in Digital Science and Technology (INRIA), Juelich Supercomputing Center, and the University of Tennessee. The paper presents a collection of compression research in many different scientific domains and several use cases. While more details about the presented works can be found in the corresponding publications, we believe that the overview presented in this paper will introduce readers to the breadth of lossy compression applicability to scientific data. It should be noted that the paper does not cover all scientific domains and possible use cases and, in fact, as the past height years show, the application of lossy compression for scientific data is rapidly expanding to more scientific domains and more use cases.

### 1.1. Before 2015

Before 2015, scientific data reduction was initially limited to a few specific use cases where generated data was exceeding—sometimes by several orders of magnitude—the communication, storage, and analysis capabilities of that time. The first use case was experiments producing critical data for scientific discoveries. Examples of such experiments are the detectors at the Large Hadron Collider (LHC) [52] and the NASA Mars rovers [87]. In the case of the LHC, detectors are equipped with real-time analysis systems to select the most interesting proton-proton (pp) collisions and reduce the data by up to 1/100000 [52]. For the NASA Mars rovers, the communication bandwidth is constrained by the deep-space communications channel between Mars and Earth [87]: for the Curiosity rover, the

data rate direct-to-Earth was 32 kb/s, and the data rate through the Odyssey orbiter was 256 kb/s. To reduce image data, NASA Mars rovers used different image compressors: ICER [110] and JPEG [20]. The second main use case was a visualization of scientific data [58, 96, 96]. The need for lossy compression for scientific visualization was identified in the 1980s [2]. For many other scientific experiments and simulations, data reduction was unnecessary: data resolution and generation rate of instrument detectors and numerical simulation were compatible with contemporary communication and storage technologies. The interest in a broader use of scientific data reduction and compression increased circa 2000, with compression schemes such as ISABELA [90], ISOBAR [130], FPC [24], and fpzip [104].

### 1.2. From 2015 to 2023

The first successful compressor for scientific data was ZFP from Lawrence Livermore National Laboratory. The first ZFP paper was published in December 2014 [103]. A few months later and independently, the SZ project started at Argonne National Laboratory, and the first version of SZ was published in June 2016 [36]. The rapid success of these two compressors, combined with a realization from the broader scientific community of the necessity to reduce data effectively, inspired many other teams to develop scientific data compression schemes. One difficulty for the early adopters of lossy compressors was to understand the performance of the different compressors from the published papers. Many publications were using different datasets and different metrics to showcase the better performance of their compression schemes. This situation created a problem concerning the scientific methodology: how to fairly compare different compression schemes if comparisons do not use community-established common datasets and metrics. The
SDRBench<fnref n="1" /> [162] repository was created to address this problem. SDRBench provides 14 different scientific datasets from different scientific disciplines that compression scheme developers can use to evaluate the performance of their design and compare it with other compressors. In parallel with this effort, the Z-checker [140, 158] software was created to provide users and developers of lossy compressors with a tool offering many (more than 30) different metrics to evaluate the impact of compression schemes on scientific data.

From 2015 to 2023, the need for scientific data reduction and compression increased rapidly, with many different applications and use-cases [26]. This expansion of the compression usage for scientific data is still increasing, with new use cases in artificial intelligence (AI) (e.g., federated learning communication compression, compression of scientific data for training, compression of AI models weights and activations). Several important evolutions of existing compressors or new compression schemes have appeared, increasing sometimes dramatically the performance of previous compressors. For example, MGARD was proposed for lossy compression of scientific data while respecting quantities of interest accuracy [54]. TTHRESH, which relies on high-order singular value decomposition for decorrelation, was designed to reach very high compression ratios on datasets with three or more dimensions [18]. SZ2 was refactored to become a modular and highly customizable lossy compression framework [101]. This redesign is instrumental for the implementation of specialized compressors such as ROIBIN-SZ [150] and MDZ [163]. Bit Grooming [159], Digit Rounding [34], and SPERR [95] were optimized for climate data lossy compression. From 2000, several groups focused on lossy compression using AI models [117], perhaps with less success than initially expected, which motivates further research in this area.

In parallel with a better understanding of compression schemes and their applications to scientific data, users started to express more complex data quality preservation criteria. Beyond preserving quantities of interest, researchers in cosmology stressed the need to preserve the accuracy of analysis results. For example, they required compressors to generate compressed data leading to a maximum deviation on the power spectrum analysis of 1%. They had similar requirements concerning the halo mass distribution analysis. For crystallography, the requirement was to keep intact (no lossy compression) regions of interest surrounding bright peaks and compress the remaining data significantly. For molecular dynamics, users demanded the precise preservation of the radial distribution function. In many cases, these difficult requirements could not be directly translated through mathematical analysis into controls of the quantization (approximation) stage of the lossy compressors. In some cases, such formal analysis of the compression errors could be used to design lossy compressor error controls preserving important data characteristics. Examples are the preservation of critical points in vector fields [99] and the preservation of linear and nonlinear data characteristics [54, 75, 154].

The widening interest in and adoption of lossy compression schemes by the scientific community have also encouraged researchers to investigate the design, implementation, and effects of “ultra-fast lossy compressors” for compressing communications in parallel simulations and even data between the processors (CPUs, GPUs, accelerators) and the memory (internal or external) [42, 41].

The following sections present examples of applications and use cases of lossy compression for scientific data, specifically developed by collaborators within the JLESC.

## 2. Compression for Climate Simulations

In this section, we describe the lossy compression research related to climate simulations that was conducted by researchers from Argonne National Laboratory (ANL) and Barcelona Supercomputing Center (BSC).

### 2.1. Research Background

The execution scales of climate simulations such as the Community Earth System Model (CESM) [85, 72] or EC-Earth [38] keep growing because of ever-increasing research problems to study. More and more model output data is being produced, and how to efficiently store and transfer it has become a critical issue for researchers. For example, when running large climate ensembles of high-fidelity 1 km × 1 km simulations where each instance simulates 15 years, 260 TB of data could be generated for estimating only one ensemble member per simulated day [37]. For the climate simulations with the Coupled Model Comparison Project Phase 6 [112], each model may produce nearly 10 PB of data. More examples of large volumes of data that are handled by climate research computing centers can be found in [15, 17, 71, 89, 159].

### 2.2. Research Problem Formulation

In this subsection we introduce a pair of Earth system models (ESMs) commonly used in the climate modeling community, the research objective in using lossy compression for these models, and evaluation methods in the context of climate data.

The climate research community has two well-known models. One of them is EC-Earth, developed by a European consortium of national meteorological services and research institutes. Its latest version currently being developed, EC-Earth4, couples various components, including OpenIFS 43R3 as the atmospheric component, NEMO 4.0.1 as the ocean component, and other components coupled using OASIS3-MCT, to understand and predict climate variability and climate change.

The other well-known ESM is the Community Earth System Model (CESM) [85], which is a fully coupled global climate model developed in collaboration with many partners in the research community. CESM provides effective simulations of the past, present, and future climate states of the Earth using hundreds of different analysis fields, including potential temperature, salinity, in situ density, sea surface height, solar short-wave heat flux, abiotic surface pH, and vertical submeso velocity.

<footnote n="1">¹https://sdrbench.github.io</footnote>
SDRBench<fnref n="1" /> [162] repository was created to address this problem. SDRBench provides 14 different scientific datasets from different scientific disciplines that compression scheme developers can use to evaluate the performance of their design and compare it with other compressors. In parallel with this effort, the Z-checker [140, 158] software was created to provide users and developers of lossy compressors with a tool offering many (more than 30) different metrics to evaluate the impact of compression schemes on scientific data.

From 2015 to 2023, the need for scientific data reduction and compression increased rapidly, with many different applications and use-cases [26]. This expansion of the compression usage for scientific data is still increasing, with new use cases in artificial intelligence (AI) (e.g., federated learning communication compression, compression of scientific data for training, compression of AI models weights and activations). Several important evolutions of existing compressors or new compression schemes have appeared, increasing sometimes dramatically the performance of previous compressors. For example, MGARD was proposed for lossy compression of scientific data while respecting quantities of interest accuracy [54]. TTHRESH, which relies on high-order singular value decomposition for decorrelation, was designed to reach very high compression ratios on datasets with three or more dimensions [18]. SZ2 was refactored to become a modular and highly customizable lossy compression framework [101]. This redesign is instrumental for the implementation of specialized compressors such as ROIBIN-SZ [150] and MDZ [163]. Bit Grooming [159], Digit Rounding [34], and SPERR [95] were optimized for climate data lossy compression. From 2000, several groups focused on lossy compression using AI models [117], perhaps with less success than initially expected, which motivates further research in this area.

In parallel with a better understanding of compression schemes and their applications to scientific data, users started to express more complex data quality preservation criteria. Beyond preserving quantities of interest, researchers in cosmology stressed the need to preserve the accuracy of analysis results. For example, they required compressors to generate compressed data leading to a maximum deviation on the power spectrum analysis of 1%. They had similar requirements concerning the halo mass distribution analysis. For crystallography, the requirement was to keep intact (no lossy compression) regions of interest surrounding bright peaks and compress the remaining data significantly. For molecular dynamics, users demanded the precise preservation of the radial distribution function. In many cases, these difficult requirements could not be directly translated through mathematical analysis into controls of the quantization (approximation) stage of the lossy compressors. In some cases, such formal analysis of the compression errors could be used to design lossy compressor error controls preserving important data characteristics. Examples are the preservation of critical points in vector fields [99] and the preservation of linear and nonlinear data characteristics [54, 75, 154].

The widening interest in and adoption of lossy compression schemes by the scientific community have also encouraged researchers to investigate the design, implementation, and effects of “ultra-fast lossy compressors” for compressing communications in parallel simulations and even data between the processors (CPUs, GPUs, accelerators) and the memory (internal or external) [42, 41].

The following sections present examples of applications and use cases of lossy compression for scientific data, specifically developed by collaborators within the JLESC.

## 2. Compression for Climate Simulations

In this section, we describe the lossy compression research related to climate simulations that was conducted by researchers from Argonne National Laboratory (ANL) and Barcelona Supercomputing Center (BSC).

### 2.1. Research Background

The execution scales of climate simulations such as the Community Earth System Model (CESM) [85, 72] or EC-Earth [38] keep growing because of ever-increasing research problems to study. More and more model output data is being produced, and how to efficiently store and transfer it has become a critical issue for researchers. For example, when running large climate ensembles of high-fidelity 1 km × 1 km simulations where each instance simulates 15 years, 260 TB of data could be generated for estimating only one ensemble member per simulated day [37]. For the climate simulations with the Coupled Model Comparison Project Phase 6 [112], each model may produce nearly 10 PB of data. More examples of large volumes of data that are handled by climate research computing centers can be found in [15, 17, 71, 89, 159].

### 2.2. Research Problem Formulation

In this subsection we introduce a pair of Earth system models (ESMs) commonly used in the climate modeling community, the research objective in using lossy compression for these models, and evaluation methods in the context of climate data.

The climate research community has two well-known models. One of them is EC-Earth, developed by a European consortium of national meteorological services and research institutes. Its latest version currently being developed, EC-Earth4, couples various components, including OpenIFS 43R3 as the atmospheric component, NEMO 4.0.1 as the ocean component, and other components coupled using OASIS3-MCT, to understand and predict climate variability and climate change.

The other well-known ESM is the Community Earth System Model (CESM) [85], which is a fully coupled global climate model developed in collaboration with many partners in the research community. CESM provides effective simulations of the past, present, and future climate states of the Earth using hundreds of different analysis fields, including potential temperature, salinity, in situ density, sea surface height, solar short-wave heat flux, abiotic surface pH, and vertical submeso velocity.

<footnote n="1">¹https://sdrbench.github.io</footnote>
The research objective is to explore or develop an efficient error-controlled lossy compressor, which is expected to get high compression ratios on climate datasets while respecting the fidelity very well in climate research. There are three ways to assess the quality of the reconstructed data in general, all of which have been widely used in the community.

*   **Statistical Metric.** Quite a few statistical metrics have been used in prior studies [17, 14, 13, 121, 64, 148], including structural similarity index measure (SSIM), peak signal-to-noise ratio (PSNR), and KS test.

*   **Visualization.** Many existing studies [14, 148] conducted by both compression developers and climate scientists assess the quality of the lossy reconstructed data by visualization of the lossy reconstructed data versus original uncompressed data.

*   **Feature Analysis.** Some works [56] have focused on the preservation of specific features of climate data (such as during the lossy compression in particular).

## 2.3. Related State of the Art

In this subsection, we comprehensively discuss the related works, which mainly include the different lossy compression methods developed and evaluated for climate simulation datasets.

Multiple works have evaluated the effectiveness or impact of lossy compression methods on climate simulation datasets. In 2014, Baker et al. [17] published a paper in which they evaluated the compression ratios for multiple lossless compressors (such as FPC [24]) and lossy compressors (such as FPZIP [105], ISABELA [90], and JPEG2000 [141]), as well as the impact of lossy compressors on climate data analysis. Many statistical analysis metrics were used in that investigation, including compression ratio, PSNR, normalized root mean squared error, and normalized maximum pointwise error. However, much more efficient error-bounded lossy compressors were not yet developed in 2014, so that work missed many important compressors such as SZ [36, 139, 98, 161, 101], ZFP [103], and Digit Rounding [34]. Thereafter, a few papers were published that investigated the effectiveness of error-bounded lossy compressors on climate data, while focusing mainly on two lossy compressors—SZ and ZFP. For example, in [121, 64, 14, 13], the climate researchers provided a series of analyses for lossily compressed climate model data based on some lossy compressors including SZ1.4.13 [139], ZFP0.5.3 [103], and FPZIP [105]. However, these compressors are now relatively outdated because more effective lossy compression algorithms were devised thereafter. For example, the latest version of SZ has been upgraded to 3.1, which adopts a different data prediction method in the SZ compression pipeline, so that the compression quality and results are substantially different [161]. Recently, Underwood et al. [148] performed a comprehensive investigation of the compression quality on climate datasets for 11 state-of-the-art error-bounded lossy compressors including MGARDx [100], SZ3 [101], ZFP 0.5.5 [103], and TTHRESH [18]. This study included many important metrics, such as the KS test and

SSIM [16]. The authors concluded that either SZ3 or ZFP has the best compression ratios/qualities in most cases.

In addition to the specific analysis of the effectiveness of lossy compressors for climate datasets, climate data have also been widely used as classic scientific datasets for the design and evaluation of new error-bounded lossy compressors. CESM datasets, for example, have often been used to evaluate SZ-series compressors (from the early version 1.4 [139] to 2.1 [98], 3.1 [161], QoZ [106], and FAZ [107]), as well as GPU data compressors [144, 143]) during the development of SZ or performance optimization.

## 2.4. XIOS and OpenIFS

The use case used by the Barcelona Supercomputing Center to test SZ involved XIOS and OpenIFS. OpenIFS is an atmospheric general circulation model developed and maintained by the European Centre for Medium-Range Weather Forecasts (ECMWF). It is derived from the Integrated Forecasting System (IFS) [19, 40], which is the operational global meteorological forecasting model and data assimilation system of ECMWF. OpenIFS is used as the atmospheric component of the new EC-Earth4 model.

One of the main bottlenecks of OpenIFS was the old sequential I/O scheme used, where all 2D and 3D fields were gathered by the master Message Passing Interface (MPI) process, and the data was then sequentially written onto the storage system. The approach did not scale at high resolutions. To overcome this issue, XIOS was integrated into OpenIFS [155]. The XML Input/Output Server (XIOS) [84, 66, 109] is an asynchronous MPI parallel I/O server used by Earth system models to avoid I/O contention. It focuses on very high scalability with support for high-resolution output. XIOS is developed by the Institute Pierre Simon Laplace. Figure 1 shows the integration between OpenIFS and XIOS. The OpenIFS processes execute the XIOS clients that asynchronously send data to the XIOS servers. These servers are run on independent nodes and are in charge of writing the data into the storage system.

<figure><img src="image_3.png" /><figcaption>Figure 1: Diagram of the OpenIFS-XIOS integration</figcaption></figure>

Since XIOS uses netCDF4 as the output data format, it is possible to enable lossless compression via HDF5. It uses gzip,
The research objective is to explore or develop an efficient error-controlled lossy compressor, which is expected to get high compression ratios on climate datasets while respecting the fidelity very well in climate research. There are three ways to assess the quality of the reconstructed data in general, all of which have been widely used in the community.

*   **Statistical Metric.** Quite a few statistical metrics have been used in prior studies [17, 14, 13, 121, 64, 148], including structural similarity index measure (SSIM), peak signal-to-noise ratio (PSNR), and KS test.

*   **Visualization.** Many existing studies [14, 148] conducted by both compression developers and climate scientists assess the quality of the lossy reconstructed data by visualization of the lossy reconstructed data versus original uncompressed data.

*   **Feature Analysis.** Some works [56] have focused on the preservation of specific features of climate data (such as during the lossy compression in particular).

## 2.3. Related State of the Art

In this subsection, we comprehensively discuss the related works, which mainly include the different lossy compression methods developed and evaluated for climate simulation datasets.

Multiple works have evaluated the effectiveness or impact of lossy compression methods on climate simulation datasets. In 2014, Baker et al. [17] published a paper in which they evaluated the compression ratios for multiple lossless compressors (such as FPC [24]) and lossy compressors (such as FPZIP [105], ISABELA [90], and JPEG2000 [141]), as well as the impact of lossy compressors on climate data analysis. Many statistical analysis metrics were used in that investigation, including compression ratio, PSNR, normalized root mean squared error, and normalized maximum pointwise error. However, much more efficient error-bounded lossy compressors were not yet developed in 2014, so that work missed many important compressors such as SZ [36, 139, 98, 161, 101], ZFP [103], and Digit Rounding [34]. Thereafter, a few papers were published that investigated the effectiveness of error-bounded lossy compressors on climate data, while focusing mainly on two lossy compressors—SZ and ZFP. For example, in [121, 64, 14, 13], the climate researchers provided a series of analyses for lossily compressed climate model data based on some lossy compressors including SZ1.4.13 [139], ZFP0.5.3 [103], and FPZIP [105]. However, these compressors are now relatively outdated because more effective lossy compression algorithms were devised thereafter. For example, the latest version of SZ has been upgraded to 3.1, which adopts a different data prediction method in the SZ compression pipeline, so that the compression quality and results are substantially different [161]. Recently, Underwood et al. [148] performed a comprehensive investigation of the compression quality on climate datasets for 11 state-of-the-art error-bounded lossy compressors including MGARDx [100], SZ3 [101], ZFP 0.5.5 [103], and TTHRESH [18]. This study included many important metrics, such as the KS test and

SSIM [16]. The authors concluded that either SZ3 or ZFP has the best compression ratios/qualities in most cases.

In addition to the specific analysis of the effectiveness of lossy compressors for climate datasets, climate data have also been widely used as classic scientific datasets for the design and evaluation of new error-bounded lossy compressors. CESM datasets, for example, have often been used to evaluate SZ-series compressors (from the early version 1.4 [139] to 2.1 [98], 3.1 [161], QoZ [106], and FAZ [107]), as well as GPU data compressors [144, 143]) during the development of SZ or performance optimization.

## 2.4. XIOS and OpenIFS

The use case used by the Barcelona Supercomputing Center to test SZ involved XIOS and OpenIFS. OpenIFS is an atmospheric general circulation model developed and maintained by the European Centre for Medium-Range Weather Forecasts (ECMWF). It is derived from the Integrated Forecasting System (IFS) [19, 40], which is the operational global meteorological forecasting model and data assimilation system of ECMWF. OpenIFS is used as the atmospheric component of the new EC-Earth4 model.

One of the main bottlenecks of OpenIFS was the old sequential I/O scheme used, where all 2D and 3D fields were gathered by the master Message Passing Interface (MPI) process, and the data was then sequentially written onto the storage system. The approach did not scale at high resolutions. To overcome this issue, XIOS was integrated into OpenIFS [155]. The XML Input/Output Server (XIOS) [84, 66, 109] is an asynchronous MPI parallel I/O server used by Earth system models to avoid I/O contention. It focuses on very high scalability with support for high-resolution output. XIOS is developed by the Institute Pierre Simon Laplace. Figure 1 shows the integration between OpenIFS and XIOS. The OpenIFS processes execute the XIOS clients that asynchronously send data to the XIOS servers. These servers are run on independent nodes and are in charge of writing the data into the storage system.

<figure><img src="image_3.png" /><figcaption>Figure 1: Diagram of the OpenIFS-XIOS integration</figcaption></figure>

Since XIOS uses netCDF4 as the output data format, it is possible to enable lossless compression via HDF5. It uses gzip,
but this default compression filter cannot offer high compression ratios and can take a considerable amount of time. Keeping low compression times is important to avoid increasing the execution time of the model, which would considerably impact the computational efficiency. Also crucial is the high compression of the data without affecting the accuracy needed to perform the scientific studies. This capability will become even more important in the upcoming years when new exascale machines will allow us to run the models at an unprecedented level of resolution that will output a huge amount of data. To explore lossy compression tools, therefore, we integrated SZ into XIOS and evaluated the results.

## 2.5. XIOS-SZ Integration

The first step was to study how to use SZ from XIOS to minimize the code changes. SZ is registered as a third-party filter of HDF5 [50], so it can be easily enabled from the netCDF4 API.

We developed the HDF5 filter for the SZ compressor (namely, H5Z SZ filter) based on the HDF5 filtering mechanism [48]. The design architecture is illustrated in Figure 2. As shown in the figure, the H5Z SZ filter comprises three key components—H5Z_SZ_Init, H5Z_sz_set_local, and H5Z_filter_sz. They are in charge of the initialization (such as memory allocation), HDF5 format analysis and SZ parameter configuration, and execution of the SZ compressor, respectively. The datasets stored in the original HDF5 file would be extracted one by one and passed to the SZ filter for compression processing. After that, another HDF5 formatted file with compressed data stored instead is generated, while the original hierarchical structure of the HDF5 file is not changed at all.

<figure><img src="image_4.png" /><figcaption>Figure 2: Design architecture of the H5Z SZ filter</figcaption></figure>

Figure 3 shows the usage of the H5Z SZ filter in XIOS: when XIOS creates a new netCDF4 file, the netCDF4 library internally uses HDF5 to structure the different datasets. This feature allows us to use the SZ compressor as a third-party library. In addition, the H5Z SZ filter allows us to define independent compression parameters per field. This is an important feature because in climate modeling the needed accuracy between fields can widely vary. For example, the precision bits for total precipitation are different concerning vorticity as compared to that of vorticity.

The XIOS code was also refactored to make use of the H5Z SZ filter and support independent compression parameters. In

<figure><img src="image_5.png" /><figcaption>Figure 3: Usage of the H5Z SZ filter in XIOS through the netCDF4 library</figcaption></figure>

XIOS, the output configuration is set up through XML files, which now are also used to specify different compression parameters, as shown in the following listing.

<caption>Listing 1: Example of an XML definition for independent compression parameters</caption>

```xml
<file id="x" compression_type="lossy"
      error_bound_mode="abs" error_bound="0.001">
  <field field_ref="t" error_bound_mode="rel"
         error_bound="0.1" />
  <field field_ref="q" error_bound="0.00001" />
  <field field_ref="cc" compression_type="lossless"
         compression_level="4" />
</file>
```

On the other hand, XIOS was also adapted to perform compression in parallel, for both gzip and SZ. This feature is particularly relevant because compression in parallel mode was not available in HDF5 until version 1.10.2 when support was added. It is an interesting feature because for large netCDF files the increase in time due to serialization is high. This can be minimized by enabling compression because it is therefore cheaper in time to write smaller netCDF files. Figure 14 illustrates the two types of writing and compressing files in XIOS through HDF5. XIOS has two different writing modes as illustrated in Figure 4: The pool of servers (a) can be independently used to write the data (multiple file mode), where each server writes an independent netCDF file with a subset of the data; or they (b) can jointly and synchronously write the data (one file mode), where all servers write a single netCDF file containing all the data. XIOS originally could only use compression with the multiple file mode because it is an inherited feature of HDF5. However, from version 1.10.2, HDF5 started to support parallel compression, although it might not be well-tuned yet. Taking advantage of this, XIOS was also adapted to enable compression in the one file mode, for both gzip and SZ. This feature is important to try to mitigate one of the main performance issues of XIOS in using the one-file mode. This mode requires synchronization among all the servers to correctly write into the same file. As a consequence, this serialization implies a degradation that gets worse for large volumes of data. If data is compressed before writing in this mode, the size reduction might considerably help in mitigating the performance degradation.

## 2.6. Performance Evaluation

The SZ compressor in XIOS was evaluated on MareNos-trum4, the supercomputer of the BSC. Its computing nodes consist of two sockets of an Intel Xeon Platinum 8160 CPU with 24
but this default compression filter cannot offer high compression ratios and can take a considerable amount of time. Keeping low compression times is important to avoid increasing the execution time of the model, which would considerably impact the computational efficiency. Also crucial is the high compression of the data without affecting the accuracy needed to perform the scientific studies. This capability will become even more important in the upcoming years when new exascale machines will allow us to run the models at an unprecedented level of resolution that will output a huge amount of data. To explore lossy compression tools, therefore, we integrated SZ into XIOS and evaluated the results.

## 2.5. XIOS-SZ Integration

The first step was to study how to use SZ from XIOS to minimize the code changes. SZ is registered as a third-party filter of HDF5 [50], so it can be easily enabled from the netCDF4 API.

We developed the HDF5 filter for the SZ compressor (namely, H5Z SZ filter) based on the HDF5 filtering mechanism [48]. The design architecture is illustrated in Figure 2. As shown in the figure, the H5Z SZ filter comprises three key components—H5Z_SZ_Init, H5Z_sz_set_local, and H5Z_filter_sz. They are in charge of the initialization (such as memory allocation), HDF5 format analysis and SZ parameter configuration, and execution of the SZ compressor, respectively. The datasets stored in the original HDF5 file would be extracted one by one and passed to the SZ filter for compression processing. After that, another HDF5 formatted file with compressed data stored instead is generated, while the original hierarchical structure of the HDF5 file is not changed at all.

<figure><img src="image_4.png" /><figcaption>Figure 2: Design architecture of the H5Z SZ filter</figcaption></figure>

Figure 3 shows the usage of the H5Z SZ filter in XIOS: when XIOS creates a new netCDF4 file, the netCDF4 library internally uses HDF5 to structure the different datasets. This feature allows us to use the SZ compressor as a third-party library. In addition, the H5Z SZ filter allows us to define independent compression parameters per field. This is an important feature because in climate modeling the needed accuracy between fields can widely vary. For example, the precision bits for total precipitation are different concerning vorticity as compared to that of vorticity.

The XIOS code was also refactored to make use of the H5Z SZ filter and support independent compression parameters. In

<figure><img src="image_5.png" /><figcaption>Figure 3: Usage of the H5Z SZ filter in XIOS through the netCDF4 library</figcaption></figure>

XIOS, the output configuration is set up through XML files, which now are also used to specify different compression parameters, as shown in the following listing.

<caption>Listing 1: Example of an XML definition for independent compression parameters</caption>

```xml
<file id="x" compression_type="lossy"
      error_bound_mode="abs" error_bound="0.001">
  <field field_ref="t" error_bound_mode="rel"
          error_bound="0.1" />
  <field field_ref="q" error_bound="0.00001" />
  <field field_ref="cc" compression_type="lossless"
          compression_level="4" />
</file>
```

On the other hand, XIOS was also adapted to perform compression in parallel, for both gzip and SZ. This feature is particularly relevant because compression in parallel mode was not available in HDF5 until version 1.10.2 when support was added. It is an interesting feature because for large netCDF files the increase in time due to serialization is high. This can be minimized by enabling compression because it is therefore cheaper in time to write smaller netCDF files. Figure 14 illustrates the two types of writing and compressing files in XIOS through HDF5. XIOS has two different writing modes as illustrated in Figure 4: The pool of servers (a) can be independently used to write the data (multiple file mode), where each server writes an independent netCDF file with a subset of the data; or they (b) can jointly and synchronously write the data (one file mode), where all servers write a single netCDF file containing all the data. XIOS originally could only use compression with the multiple file mode because it is an inherited feature of HDF5. However, from version 1.10.2, HDF5 started to support parallel compression, although it might not be well-tuned yet. Taking advantage of this, XIOS was also adapted to enable compression in the one file mode, for both gzip and SZ. This feature is important to try to mitigate one of the main performance issues of XIOS in using the one-file mode. This mode requires synchronization among all the servers to correctly write into the same file. As a consequence, this serialization implies a degradation that gets worse for large volumes of data. If data is compressed before writing in this mode, the size reduction might considerably help in mitigating the performance degradation.

## 2.6. Performance Evaluation

The SZ compressor in XIOS was evaluated on MareNos-trum4, the supercomputer of the BSC. Its computing nodes consist of two sockets of an Intel Xeon Platinum 8160 CPU with 24
<figure><img src="image_6.png" /><figcaption>Figure 4: Diagram (a): partial compression where each I/O server compresses its netCDF file; diagram (b): the new compression in parallel where all I/O servers write into a single compressed netCDF file. Diagram (a): multiple file mode where each I/O server writes its netCDF file; diagram (b): one file mode where all I/O servers write into a single netCDF file</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Figure 5: Comparison between Uncompressed raw and SZ reconstructed data (error bound = 0.001) using temperature and relative humidity fields at 1000 hPa</figcaption></figure>

cores each for a total of 48 cores per node and 96 GB of main memory distributed with 12x 8 GB 2667 MHz DIMM. The tests use 80 MPI processes for OpenIFS, six OpenMP threads in each MPI task, and two XIOS I/O servers in an exclusive node. For the evaluation, the tests included a simulation of a ten-day forecast running a Tco511L91 configuration, which corresponds to a horizontal resolution of 23 km and with a time step of 900 seconds. We extracted 10+ datasets (climate fields) from the climate data files (.nc) and evaluated the compression ratios of different error-bounded lossy compressors (including SZ, SZx, and ZFP) with the same error bounds as well as Gzip lossless compressor. Since the datasets are all in unstructured format, we can only treat them as 1D arrays.

We first assess the quality of the reconstructed data generated by the error-bounded lossy compression against the original uncompressed raw data to ensure the appropriateness of the error-bound setting we are using. Figure 5 demonstrates the visualization of the two 3D fields, temperature, and relative humidity, at the pressure level of 1000 hPa, based on the raw data and lossy compressed data (using SZ with relative error bound of 0.001). The figure demonstrates a fairly high visual quality of the decompressed data under the relative error bound of 0.001. Thus, we use the same setting in the following test for other lossy compressors on the climate datasets.

We evaluate the compression ratios for different lossy compressors on the climate datasets. As shown in Figure 6, SZ exhibits the highest compression ratio from among all compressors in our evaluation. In absolute terms, SZ's compression ratio is about 1.5-20× and 2-10× as high as that of the lossy compressor ZFP and SZx, respectively.

Figure 7 further shows different tests comparing gzip against SZ in different scenarios: no compression and compressing 19 3D fields combined with one and multiple file modes, which corresponds to the two modes illustrated in Figure 4. For these

<figure><img src="image_8.png" /><figcaption>Figure 6: Comparison of compression ratios among different lossy compressors with the same relative error bound of 0.001</figcaption></figure>

tests a compression level of 5 was used for gzip and a relative error bound of 0.001 for SZ. Figure 7 shows different tests comparing no compression and SZ compression of 19 3D fields combined with one and multiple file modes of XIOS, which corresponds to the two modes illustrated before in Figure 4. For these tests, a relative error bound of 0.001 was used for SZ.

In Figure 7 we can observe that the compression ratio of SZ is much higher than that of gzip and that SZ takes less time as well: SZ can offer a compression ratio of 20.5 whereas gzip's ratio is only 1.94. The overhead of enabling the SZ compression for the multiple file mode is negligible, but in gzip there is an extra cost of 142%. This becomes even worse with the one-file mode because the serialization due to the needed synchronization of multiple I/O servers is very high when writing large volumes of data. This hugely impacts gzip but not SZ at the same level: while SZ has an overhead of 71.1%, gzip needs up to 520.3% more time. In Figure 7 we can observe that the compression ratio of SZ is high (around 20.5) for both one and multiple file modes of XIOS. However, whereas the execution
<figure><img src="image_6.png" /><figcaption>Figure 4: Diagram (a): partial compression where each I/O server compresses its netCDF file; diagram (b): the new compression in parallel where all I/O servers write into a single compressed netCDF file. Diagram (a): multiple file mode where each I/O server writes its netCDF file; diagram (b): one file mode where all I/O servers write into a single netCDF file</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Figure 5: Comparison between Uncompressed raw and SZ reconstructed data (error bound = 0.001) using temperature and relative humidity fields at 1000 hPa</figcaption></figure>

cores each for a total of 48 cores per node and 96 GB of main memory distributed with 12x 8 GB 2667 MHz DIMM. The tests use 80 MPI processes for OpenIFS, six OpenMP threads in each MPI task, and two XIOS I/O servers in an exclusive node. For the evaluation, the tests included a simulation of a ten-day forecast running a Tco511L91 configuration, which corresponds to a horizontal resolution of 23 km and with a time step of 900 seconds. We extracted 10+ datasets (climate fields) from the climate data files (.nc) and evaluated the compression ratios of different error-bounded lossy compressors (including SZ, SZx, and ZFP) with the same error bounds as well as Gzip lossless compressor. Since the datasets are all in unstructured format, we can only treat them as 1D arrays.

We first assess the quality of the reconstructed data generated by the error-bounded lossy compression against the original uncompressed raw data to ensure the appropriateness of the error-bound setting we are using. Figure 5 demonstrates the visualization of the two 3D fields, temperature, and relative humidity, at the pressure level of 1000 hPa, based on the raw data and lossy compressed data (using SZ with relative error bound of 0.001). The figure demonstrates a fairly high visual quality of the decompressed data under the relative error bound of 0.001. Thus, we use the same setting in the following test for other lossy compressors on the climate datasets.

We evaluate the compression ratios for different lossy compressors on the climate datasets. As shown in Figure 6, SZ exhibits the highest compression ratio from among all compressors in our evaluation. In absolute terms, SZ's compression ratio is about 1.5-20× and 2-10× as high as that of the lossy compressor ZFP and SZx, respectively.

Figure 7 further shows different tests comparing gzip against SZ in different scenarios: no compression and compressing 19 3D fields combined with one and multiple file modes, which corresponds to the two modes illustrated in Figure 4. For these

<figure><img src="image_8.png" /><figcaption>Figure 6: Comparison of compression ratios among different lossy compressors with the same relative error bound of 0.001</figcaption></figure>

tests a compression level of 5 was used for gzip and a relative error bound of 0.001 for SZ. Figure 7 shows different tests comparing no compression and SZ compression of 19 3D fields combined with one and multiple file modes of XIOS, which corresponds to the two modes illustrated before in Figure 4. For these tests, a relative error bound of 0.001 was used for SZ.

In Figure 7 we can observe that the compression ratio of SZ is much higher than that of gzip and that SZ takes less time as well: SZ can offer a compression ratio of 20.5 whereas gzip's ratio is only 1.94. The overhead of enabling the SZ compression for the multiple file mode is negligible, but in gzip there is an extra cost of 142%. This becomes even worse with the one-file mode because the serialization due to the needed synchronization of multiple I/O servers is very high when writing large volumes of data. This hugely impacts gzip but not SZ at the same level: while SZ has an overhead of 71.1%, gzip needs up to 520.3% more time. In Figure 7 we can observe that the compression ratio of SZ is high (around 20.5) for both one and multiple file modes of XIOS. However, whereas the execution
<figure><img src="image_9.png" /><figcaption>Figure 7: Comparison of the execution time and the output size between gzip and SZ using both normal and parallel I/O Execution time and output size using no compression and the SZ compressor in XIOS for both one and multiple file modes</figcaption></figure>

time does not increase for the multiple file mode, there is considerable performance degradation for the one file mode. This is a preliminary study, so we plan to further analyze the source of this degradation in the future. It might be related to the new parallel compression feature added in HDF5. It might also be caused by the performance issue of the one file mode for large netCDF files described in Section 2.5. However, it is possible to improve this performance degradation by adding more XIOS servers to make use of more computational resources, and therefore, speeding up the processing of events, compression, and netCDF encoding. For instance, instead of using 2 XIOS servers as illustrated in Figure 7, with 6 XIOS servers the execution time is reduced from 5561 to 4696 seconds.

It is also important to note that the compression ratio between the one and multiple file modes is slightly different. This is related to the metadata of the netCDF files. In the multiple file mode metadata is replicated among the different partial netCDF files, whereas in the one file mode metadata is outputted once, in the single netCDF file generated.

## 2.7. Conclusions and Future Work

We have presented an overview of the integration of the SZ compressor in XIOS, which involved developing the H5Z SZ filter as a third-party filter of HDF5 and refactoring the XIOS source code to make use of SZ as well as defining independent compression parameters per field. This last point is crucial to produce enough accurate data to not invalidate scientific results.

The performance evaluation of SZ is positive compared with the default lossless compression of XIOS offered by gzip. The SZ compressor is faster, and it achieves much higher compression ratios, which are even more outstanding when using the one-file mode of XIOS. The performance evaluation of SZ is positive when using the multiple-file mode of XIOS because the performance remains the same and it achieves a high compression ratio. In addition, the one file mode can be improved when adding extra XIOS servers to make use of more computational resources. However, the performance numbers presented

in this paper might vary because we used the same compression parameters for all the fields, so in an experiment using scientifically validated compression parameters these can be different.

An important need in the future is to determine with climate scientists the different acceptable error ranges per field to set the compression parameters accordingly. In addition, further work is needed on the parallel compression offered by the one-file mode of XIOS. In that sense, of interest is the testing of the latest versions of HDF5 to see the impact of the performance improvements introduced in the experimental version 1.13.1.

## 3. Compression for Instrument Data

In this subsection, we delve into our collaborative research efforts between RIKEN and Argonne National Laboratory on data compression techniques for large experimental scientific instruments such as particle colliders, distributed radio telescopes, and synchrotron radiation facilities. These instruments are pivotal in deepening our understanding of the universe, leading to significant advancements in fields such as semiconductors, medicine, and materials, to highlight just a few areas of impact. To facilitate scientific discoveries, capturing more intricate details both spatially and temporally is essential, which consequently leads to an exponential surge in the volume of data generated. Figure 8 depicts a typical data pipeline configuration of scientific instruments and consists of two distinctive edge computing stages: upstream and downstream. With the ever-increasing volume of data generated by those instruments, transmitting raw data downstream for analysis becomes exceedingly challenging, cost-ineffective, or impossible if the data rate surpasses the capabilities of the network hardware that bridges between stages. Our goal is to understand the nature of data at each stage and investigate optimal compression techniques.

Synchrotron radiation facilities play a crucial role in studying the structures of various materials, from physical to biological domains. These facilities use intense X-rays to capture structural changes over time, providing insights into phenomena such as protein function. As these facilities enhance their capabilities with improved X-ray sources and advanced X-ray imaging detectors, they will produce even more data. For instance, the SPring-8 facility will upgrade its beamlines with the upcoming CITIUS generation detector. In 2025, SPring-8 beamlines are expected to generate a massive 1.3 exabytes of raw data annually. Likewise, with the completion of the upgrades at the Advanced Photon Source at Argonne National Laboratory and the Linac Coherent Light Source at SLAC National Accelerator Center in the United States, these facilities are expected to generate 1 TB/s.

### 3.1. Research Problem Formulation

First, the necessity for data compression or reduction inside or near the data source has become paramount in order to transfer data efficiently into downstream edge computing. Compressing data directly within the detector ASICs is one of the potential solutions. However, the challenge lies in developing efficient hardware compressor algorithms and devising a design
<figure><img src="image_9.png" /><figcaption>Figure 7: Comparison of the execution time and the output size between gzip and SZ using both normal and parallel I/O Execution time and output size using no compression and the SZ compressor in XIOS for both one and multiple file modes</figcaption></figure>

time does not increase for the multiple file mode, there is considerable performance degradation for the one file mode. This is a preliminary study, so we plan to further analyze the source of this degradation in the future. It might be related to the new parallel compression feature added in HDF5. It might also be caused by the performance issue of the one file mode for large netCDF files described in Section 2.5. However, it is possible to improve this performance degradation by adding more XIOS servers to make use of more computational resources, and therefore, speeding up the processing of events, compression, and netCDF encoding. For instance, instead of using 2 XIOS servers as illustrated in Figure 7, with 6 XIOS servers the execution time is reduced from 5561 to 4696 seconds.

It is also important to note that the compression ratio between the one and multiple file modes is slightly different. This is related to the metadata of the netCDF files. In the multiple file mode metadata is replicated among the different partial netCDF files, whereas in the one file mode metadata is outputted once, in the single netCDF file generated.

## 2.7. Conclusions and Future Work

We have presented an overview of the integration of the SZ compressor in XIOS, which involved developing the H5Z SZ filter as a third-party filter of HDF5 and refactoring the XIOS source code to make use of SZ as well as defining independent compression parameters per field. This last point is crucial to produce enough accurate data to not invalidate scientific results.

The performance evaluation of SZ is positive compared with the default lossless compression of XIOS offered by gzip. The SZ compressor is faster, and it achieves much higher compression ratios, which are even more outstanding when using the one-file mode of XIOS. The performance evaluation of SZ is positive when using the multiple-file mode of XIOS because the performance remains the same and it achieves a high compression ratio. In addition, the one file mode can be improved when adding extra XIOS servers to make use of more computational resources. However, the performance numbers presented

in this paper might vary because we used the same compression parameters for all the fields, so in an experiment using scientifically validated compression parameters these can be different.

An important need in the future is to determine with climate scientists the different acceptable error ranges per field to set the compression parameters accordingly. In addition, further work is needed on the parallel compression offered by the one-file mode of XIOS. In that sense, of interest is the testing of the latest versions of HDF5 to see the impact of the performance improvements introduced in the experimental version 1.13.1.

## 3. Compression for Instrument Data

In this subsection, we delve into our collaborative research efforts between RIKEN and Argonne National Laboratory on data compression techniques for large experimental scientific instruments such as particle colliders, distributed radio telescopes, and synchrotron radiation facilities. These instruments are pivotal in deepening our understanding of the universe, leading to significant advancements in fields such as semiconductors, medicine, and materials, to highlight just a few areas of impact. To facilitate scientific discoveries, capturing more intricate details both spatially and temporally is essential, which consequently leads to an exponential surge in the volume of data generated. Figure 8 depicts a typical data pipeline configuration of scientific instruments and consists of two distinctive edge computing stages: upstream and downstream. With the ever-increasing volume of data generated by those instruments, transmitting raw data downstream for analysis becomes exceedingly challenging, cost-ineffective, or impossible if the data rate surpasses the capabilities of the network hardware that bridges between stages. Our goal is to understand the nature of data at each stage and investigate optimal compression techniques.

Synchrotron radiation facilities play a crucial role in studying the structures of various materials, from physical to biological domains. These facilities use intense X-rays to capture structural changes over time, providing insights into phenomena such as protein function. As these facilities enhance their capabilities with improved X-ray sources and advanced X-ray imaging detectors, they will produce even more data. For instance, the SPring-8 facility will upgrade its beamlines with the upcoming CITIUS generation detector. In 2025, SPring-8 beamlines are expected to generate a massive 1.3 exabytes of raw data annually. Likewise, with the completion of the upgrades at the Advanced Photon Source at Argonne National Laboratory and the Linac Coherent Light Source at SLAC National Accelerator Center in the United States, these facilities are expected to generate 1 TB/s.

### 3.1. Research Problem Formulation

First, the necessity for data compression or reduction inside or near the data source has become paramount in order to transfer data efficiently into downstream edge computing. Compressing data directly within the detector ASICs is one of the potential solutions. However, the challenge lies in developing efficient hardware compressor algorithms and devising a design
<figure><img src="image_10.png" /><figcaption>Figure 8: A data pipeline configuration of scientific instruments</figcaption></figure>

strategy for hardware compressor algorithms that not only meet
performance and resource requirements but also minimize de-
velopment costs.

Hundreds to thousands of terabytes of instrument data are processed, archived, visualized, and analyzed each year at light sources [150, 93]. At this volume, data instrument data presents significant *data compression of instrument data* presents a significant challenge due to the complexity and volume of data involved operational challenges to move and store. Compression offers a potential solution, but lossless compression is insufficiently effective to use as the primary means of compression which leads to the consideration of lossy compression. However, with lossy compression, it becomes critical to preserve the scientific integrity of the images which itself is complicated by the complexity and acquisition rate of the data. This data consists of observations of complex materials at short intervals with significant noise. Compression methods which are capable of separating the true data from the noise is critical in order to achieve high compression ratios while preserving the scientific content of the images while meeting operational constraints. We provide details on how each compression approach tackles this differently in Section 3.3 Without preserving the scientific integrity of the images, lossy compression techniques will not be adopted.

Another significant issue is the variability of instrument data, which differ in their structure or data type due to the different kinds of experiments performed at light sources. Serial crystallography, tomography, ptychography, X-ray photon correlation spectroscopy, and coherent surface scattering imaging represent just a fraction of the kinds of methodologies used at light sources, and each produces vastly different structures of data. Compounding the challenges raised by the various experimental methodologies are the different types of detectors with different features including gain-switching sensitivity, various capture rates, and different output precision, leading to myriad data structures that need to be compressed. Therefore, one-type-fits-all compression solutions are not appropriate.

Additionally, the continuous generation of real-time instru-
ment data introduces the need for real-time compression and
decompression processes, further complicating the task. Detec-
tors have limited storage capacity as they capture images con-
tinuously. If the data is not read off the detector quickly, it will
be overwritten by the next frame, resulting in uncontrolled data
loss that will inhibit the capture of scientific phenomena. Thus,
compression pipelines located at the edge need to operate in
real-time, placing constraints on the bandwidth of the compres-

sor.

In order to address these challenges effectively, domain-
specific knowledge and innovative adaptations of compression
techniques are required to ensure that instrument data remains
accessible and valuable for scientific and industrial purposes.

*Why It Is Important.* Data from light sources informs our un-
derstanding of the world around us. Domains as far spread as
agriculture, biology, neuroscience, electrochemistry, materials
science, and microelectronics leverage this data so that we can
better understand the structure and properties and can design
better solutions to problems all around us. Improvements in
detector technology give us greater resolution allowing us to
study progressively smaller phenomena, a larger field of view
allowing us to study larger samples, and a higher rate allowing
us to study increasingly ephemeral phenomena. However, these
capabilities further increase the volume of data required.

To understand why compression for instruments is impor-
tant, one must understand the consequences of either not using
compression at all or using only lossless compression. Storing
nearly 1 TB/s continuously would exhaust all of the available
storage at Argonne in less than 3 days and all of the storage at
Riken in less than 2 days, rendering each system at these sites
ineffective for most tasks since they require at least some stor-
age. Lossless compression is of limited effectiveness because
of the high entropy of light source data, giving a compression
ratio of less than 2, which grants only a brief reprieve from
the deluge of data by extending our available capacity to just
under a week. However, lossy compression systems such as
ROIBIN-SZ can achieve much higher compression ratios—up
to 196x—extending the time available to process data from a
week to nearly a year by allowing collected data to be stored
in a compressed form prior to processing drastically decreasing
the rate at which storage capacity is consumed, allowing time to
sort through the data that is collected and to transfer only impor-
tant data to off-site/archival storage, leaving storage capacity for
other usages of the machines and reducing costs [150]. There-
fore, lossy compression is an essential part of any sustainable
storage system for processing data from light sources.

For lossy compression to be a complete solution, however,
we need to establish the integrity of the results, the capability
to meet bandwidth requirements, and the suitability for a wide
variety of experiments.

*Potential Impact.* Efficiently managing data in large-scale fa-
cilities such as synchrotron radiation centers is essential to keep
storage costs in check and streamline data transfer operations.
Advanced data compression techniques and predictive models
can significantly reduce the volume of data needing storage
and transmission. Beyond cost reductions from reduced hard-
ware requirements, reducing the volume of data also facilitates
real-time analysis by providing more margin to complete tasks
within deadlines with available resources. These concrete im-
provements underscore the transformative impact of these meth-
ods on the way data is handled and leveraged in the pursuit of
scientific knowledge.
<figure><img src="image_10.png" /><figcaption>Figure 8: A data pipeline configuration of scientific instruments</figcaption></figure>

strategy for hardware compressor algorithms that not only meet
performance and resource requirements but also minimize de-
velopment costs.

Hundreds to thousands of terabytes of instrument data are processed, archived, visualized, and analyzed each year at light sources [150, 93]. At this volume, data instrument data presents significant *data compression of instrument data* presents a significant challenge due to the complexity and volume of data involved operational challenges to move and store. Compression offers a potential solution, but lossless compression is insufficiently effective to use as the primary means of compression which leads to the consideration of lossy compression. However, with lossy compression, it becomes critical to preserve the scientific integrity of the images which itself is complicated by the complexity and acquisition rate of the data. This data consists of observations of complex materials at short intervals with significant noise. Compression methods which are capable of separating the true data from the noise is critical in order to achieve high compression ratios while preserving the scientific content of the images while meeting operational constraints. We provide details on how each compression approach tackles this differently in Section 3.3 Without preserving the scientific integrity of the images, lossy compression techniques will not be adopted.

Another significant issue is the variability of instrument data, which differ in their structure or data type due to the different kinds of experiments performed at light sources. Serial crystallography, tomography, ptychography, X-ray photon correlation spectroscopy, and coherent surface scattering imaging represent just a fraction of the kinds of methodologies used at light sources, and each produces vastly different structures of data. Compounding the challenges raised by the various experimental methodologies are the different types of detectors with different features including gain-switching sensitivity, various capture rates, and different output precision, leading to myriad data structures that need to be compressed. Therefore, one-type-fits-all compression solutions are not appropriate.

Additionally, the continuous generation of real-time instru-
ment data introduces the need for real-time compression and
decompression processes, further complicating the task. Detec-
tors have limited storage capacity as they capture images con-
tinuously. If the data is not read off the detector quickly, it will
be overwritten by the next frame, resulting in uncontrolled data
loss that will inhibit the capture of scientific phenomena. Thus,
compression pipelines located at the edge need to operate in
real-time, placing constraints on the bandwidth of the compres-

sor.

In order to address these challenges effectively, domain-
specific knowledge and innovative adaptations of compression
techniques are required to ensure that instrument data remains
accessible and valuable for scientific and industrial purposes.

*Why It Is Important.* Data from light sources informs our un-
derstanding of the world around us. Domains as far spread as
agriculture, biology, neuroscience, electrochemistry, materials
science, and microelectronics leverage this data so that we can
better understand the structure and properties and can design
better solutions to problems all around us. Improvements in
detector technology give us greater resolution allowing us to
study progressively smaller phenomena, a larger field of view
allowing us to study larger samples, and a higher rate allowing
us to study increasingly ephemeral phenomena. However, these
capabilities further increase the volume of data required.

To understand why compression for instruments is impor-
tant, one must understand the consequences of either not using
compression at all or using only lossless compression. Storing
nearly 1 TB/s continuously would exhaust all of the available
storage at Argonne in less than 3 days and all of the storage at
Riken in less than 2 days, rendering each system at these sites
ineffective for most tasks since they require at least some stor-
age. Lossless compression is of limited effectiveness because
of the high entropy of light source data, giving a compression
ratio of less than 2, which grants only a brief reprieve from
the deluge of data by extending our available capacity to just
under a week. However, lossy compression systems such as
ROIBIN-SZ can achieve much higher compression ratios—up
to 196x—extending the time available to process data from a
week to nearly a year by allowing collected data to be stored
in a compressed form prior to processing drastically decreasing
the rate at which storage capacity is consumed, allowing time to
sort through the data that is collected and to transfer only impor-
tant data to off-site/archival storage, leaving storage capacity for
other usages of the machines and reducing costs [150]. There-
fore, lossy compression is an essential part of any sustainable
storage system for processing data from light sources.

For lossy compression to be a complete solution, however,
we need to establish the integrity of the results, the capability
to meet bandwidth requirements, and the suitability for a wide
variety of experiments.

*Potential Impact.* Efficiently managing data in large-scale fa-
cilities such as synchrotron radiation centers is essential to keep
storage costs in check and streamline data transfer operations.
Advanced data compression techniques and predictive models
can significantly reduce the volume of data needing storage
and transmission. Beyond cost reductions from reduced hard-
ware requirements, reducing the volume of data also facilitates
real-time analysis by providing more margin to complete tasks
within deadlines with available resources. These concrete im-
provements underscore the transformative impact of these meth-
ods on the way data is handled and leveraged in the pursuit of
scientific knowledge.
## 3.2. Related State of the Art

In this segment, we deliver an overview of the previous state of the art that is highly relevant to our work in the context of image and video compression.

Recently, researchers have expressed increasing interest in using convolutional neural networks (CNNs) and recurrent neural networks for data compression. Some of these methods build on the established HEVC (H.265) standard and target specific modules, such as intraprediction [31], [46], [94], interpretation [33], [164], quantization [8], entropy encoding [122], and loop filtering [74]. These efforts have generally led to improvements over the HEVC codec, ranging from 0.5 percent to 5 percent in terms of compression efficiency.

Numerous studies have delved into lossy compression techniques exploring a range of video standards such as JPEG, MPEG-4 (H.264), and HEVC (H.265). Other endeavors have involved deep neural networks operating independently of the HEVC framework. For example, Chen et al. [28] introduced DeepCoder, which achieved results on a par with the x264 decoder, and pixel motion (PixelMotionCNN) extension and hybrid prediction [28], which demonstrated comparable performance to the H.264 codec [133]. Some scientists have investigated the utilization of long short-term memory (LSTM) networks to learn video representations, while others have combined CNN and LSTM [108] to predict future frames. Although these approaches are aimed at improving existing video compression methods, they predominantly belong to the realm of lossy compression. Notably, their bitrate savings over the x264 encoder have generally averaged below 10%. Lossy compression techniques such as SZ and ZFP primarily concentrated on enhancing the compression of data by adhering to an error margin. SZ [36], for example, represents an error-bound lossy compression approach tailored for actual data compression, ZFP [100] stands as a specialized library for high-throughput compression of floating-point arrays, while ZFP [103] is primarily a specialized library for high-throughput compression of floating-point arrays.

TEZip offers cutting-edge solutions that can transform the instrument data. TEZip [127] is addressing vast instrument datasets. PredNet is an example of a deep convolutional recurrent neural network used for this purpose. This innovative approach to temporal relationships within instrument image data offers robust support for integer, floating-point value prediction in datasets. This advancement represents a significant step in the state of the art for data compression, enabling more effective handling of complex, time-varying image data.

When considering compression for light sources specifically, a different line of compressors has been considered. Prior to ROIBIN-SZ and TEZip, there were two primary state-of-the-art approaches: general-purpose error-bounded lossy compressors such as SZ and non-hit rejection. Hadiean-Jazi et al. [62] considered the use of so-called non-hit rejection techniques. Non-hit rejection observes that some frames contain no detected regions of interest. This occurs in serial crystallography because while injecting microcrystals before the detector, it is possible that no crystals are in the field of view of the detector. When such a frame is detected, the storage of the frame can be re-

jected, reducing the storage and bandwidth needs of the compression. On datasets that we study, this can reduce the volume of data by between 3.22× and 5.81×.

Two decades earlier, in 1998, Ferrer et al. [47] studied compression schemes derived from wavelet transforms (the basis of the modern-day SPERR [95] compressor) and cosine transforms (used by JPEG-style compression algorithms and similar to what is done in ZFP). Since that work, compression designs now support error bounds on the error introduced by compression, which allows for greater confidence in the reconstructed data.

More recently, Leonarski et al. [93] studied the use of SZ. This work found that while large compression ratios are possible, they can be affected substantially by small local intensity fluctuations when using large error bounds, thus limiting the effective compression ratios that can be used in practice with general-purpose error-bounded lossy compressors.

## 3.3. Overview of Methods Developed in the JLESC

Researchers at RIKEN and Argonne have developed three distinct methods within the JLESC, sharing progress and findings to improve compression for instruments. StreamPressor [157] is a stream compressor hardware generator written in the Chisel hardware construction langauge [12] for evaluating various designs of streaming hardware compressor that has combinations of predefined hardware compressor primitives as well as user-defined primitives, which allows researchers to evaluate efficient hardware compressor algorithms for future computing architecture and detector systems. Riken's compressor TEZip[127] (time evolutionary zip) uses a deep convolutional recurrent neural network to exploit the similarity between consecutive images in instrument data in order to achieve extremely high compression ratios at high speeds separating signal (differences between frames) from noise (similarities between frames). Argonne's compressor ROIBIN-SZ[150] instead relies upon more traditional computing approaches but leverages domain-specific knowledge from serial crystallography to separate the noisy background from features of interest while achieving high compression ratios at extremely high speeds while preserving scientific quality. Together these approaches serve distinct roles in the data life cycle at light sources—ROIBIN-SZ is better suited for extremely high-rate applications early in the life cycle when the data is collected, and TEZip is better suited for extremely high compression applications for archival purposes later in the life cycle.

*Streaming Hardware Compressor Generator:* Research and development of on-chip hardware compressors for detector application-specific integrated circuits (ASICs) is a mid to long-term investment, similar to any other custom chip development. Due to the strict timing and resource constraints in detectors, hardware implementation of general-purpose compression algorithms, such as LZ77, is not feasible. Hardware compressors need to process data without relying on a large buffer, ideally with all processing occurring in a data-flow manner, eliminating the need for any buffer. Additionally, the unique characteristics of instrument data, such as sparsity and unusual data precision, may not
## 3.2. Related State of the Art

In this segment, we deliver an overview of the previous state of the art that is highly relevant to our work in the context of image and video compression.

Recently, researchers have expressed increasing interest in using convolutional neural networks (CNNs) and recurrent neural networks for data compression. Some of these methods build on the established HEVC (H.265) standard and target specific modules, such as intraprediction [31], [46], [94], interpretation [33], [164], quantization [8], entropy encoding [122], and loop filtering [74]. These efforts have generally led to improvements over the HEVC codec, ranging from 0.5 percent to 5 percent in terms of compression efficiency.

Numerous studies have delved into lossy compression techniques exploring a range of video standards such as JPEG, MPEG-4 (H.264), and HEVC (H.265). Other endeavors have involved deep neural networks operating independently of the HEVC framework. For example, Chen et al. [28] introduced DeepCoder, which achieved results on a par with the x264 decoder, and pixel motion (PixelMotionCNN) extension and hybrid prediction [28], which demonstrated comparable performance to the H.264 codec [133]. Some scientists have investigated the utilization of long short-term memory (LSTM) networks to learn video representations, while others have combined CNN and LSTM [108] to predict future frames. Although these approaches are aimed at improving existing video compression methods, they predominantly belong to the realm of lossy compression. Notably, their bitrate savings over the x264 encoder have generally averaged below 10%. Lossy compression techniques such as SZ and ZFP primarily concentrated on enhancing the compression of data by adhering to an error margin. SZ [36], for example, represents an error-bound lossy compression approach tailored for actual data compression, ZFP [100] stands as a specialized library for high-throughput compression of floating-point arrays, while ZFP [103] is primarily a specialized library for high-throughput compression of floating-point arrays.

TEZip offers cutting-edge solutions that can transform the instrument data. TEZip [127] is addressing vast instrument datasets. PredNet is an example of a deep convolutional recurrent neural network used for this purpose. This innovative approach to temporal relationships within instrument image data offers robust support for integer, floating-point value prediction in datasets. This advancement represents a significant step in the state of the art for data compression, enabling more effective handling of complex, time-varying image data.

When considering compression for light sources specifically, a different line of compressors has been considered. Prior to ROIBIN-SZ and TEZip, there were two primary state-of-the-art approaches: general-purpose error-bounded lossy compressors such as SZ and non-hit rejection. Hadiean-Jazi et al. [62] considered the use of so-called non-hit rejection techniques. Non-hit rejection observes that some frames contain no detected regions of interest. This occurs in serial crystallography because while injecting microcrystals before the detector, it is possible that no crystals are in the field of view of the detector. When such a frame is detected, the storage of the frame can be re-

jected, reducing the storage and bandwidth needs of the compression. On datasets that we study, this can reduce the volume of data by between 3.22× and 5.81×.

Two decades earlier, in 1998, Ferrer et al. [47] studied compression schemes derived from wavelet transforms (the basis of the modern-day SPERR [95] compressor) and cosine transforms (used by JPEG-style compression algorithms and similar to what is done in ZFP). Since that work, compression designs now support error bounds on the error introduced by compression, which allows for greater confidence in the reconstructed data.

More recently, Leonarski et al. [93] studied the use of SZ. This work found that while large compression ratios are possible, they can be affected substantially by small local intensity fluctuations when using large error bounds, thus limiting the effective compression ratios that can be used in practice with general-purpose error-bounded lossy compressors.

## 3.3. Overview of Methods Developed in the JLESC

Researchers at RIKEN and Argonne have developed three distinct methods within the JLESC, sharing progress and findings to improve compression for instruments. StreamPressor [157] is a stream compressor hardware generator written in the Chisel hardware construction langauge [12] for evaluating various designs of streaming hardware compressor that has combinations of predefined hardware compressor primitives as well as user-defined primitives, which allows researchers to evaluate efficient hardware compressor algorithms for future computing architecture and detector systems. Riken's compressor TEZip[127] (time evolutionary zip) uses a deep convolutional recurrent neural network to exploit the similarity between consecutive images in instrument data in order to achieve extremely high compression ratios at high speeds separating signal (differences between frames) from noise (similarities between frames). Argonne's compressor ROIBIN-SZ[150] instead relies upon more traditional computing approaches but leverages domain-specific knowledge from serial crystallography to separate the noisy background from features of interest while achieving high compression ratios at extremely high speeds while preserving scientific quality. Together these approaches serve distinct roles in the data life cycle at light sources—ROIBIN-SZ is better suited for extremely high-rate applications early in the life cycle when the data is collected, and TEZip is better suited for extremely high compression applications for archival purposes later in the life cycle.

*Streaming Hardware Compressor Generator:* Research and development of on-chip hardware compressors for detector application-specific integrated circuits (ASICs) is a mid to long-term investment, similar to any other custom chip development. Due to the strict timing and resource constraints in detectors, hardware implementation of general-purpose compression algorithms, such as LZ77, is not feasible. Hardware compressors need to process data without relying on a large buffer, ideally with all processing occurring in a data-flow manner, eliminating the need for any buffer. Additionally, the unique characteristics of instrument data, such as sparsity and unusual data precision, may not
fit well with general-purpose compressor algorithms. However, these specific attributes present an opportunity for developing highly effective and simple custom hardware compression algorithms. Previously (outside of the JLESC context), RIKEN, Argonne National Laboratory, and other institutes have proposed custom hardware compression algorithms for scientific instrument data and numerical computation and evaluated their characteristics [63, 134, 135, 147]. In the context of our JLESC collaborative project, we have recently initiated a new research and development activity on hardware compressor generator frameworks in 2023. This activity focuses on hardware compressor generator frameworks, aiming to support future scientific detector co-design efforts. We have introduced Stream-Pressor [156], a generator framework for designing, verifying, and estimating resources in streaming hardware compressor architectures. This framework is intended to assist compressor developers in exploring different compressor architectures with different compressor building blocks, evaluating their characteristics (latency, throughput, gate counts), and generating RTL code for integrating them into custom accelerator designs. Our motivation is to bridge the gap between software and hardware experts through this proposed framework as a co-design tool, leveraging open-source hardware design tools.

Lossy Compression for Instrument Data by ROIBIN-SZ.. In order to understand ROIBIN-SZ, it is helpful to understand the general serial crystallography analysis workflow. First, a peak finder is used to identify so-called Bragg spots/peaks that represent the positions of electrons during the imaging process of serial crystallography. Second, since the peak-finding process has some false negatives and false positives, indexing is used to estimate the unit cell parameters of the crystalline structure from the positions of the Bragg peaks and the background of the images. Third, the indexed unit cells are merged into a 3D diffraction volume. Fourth, phase retrieval is performed on the merged volume to reconstruct the electron density, which is then studied to understand the structure and proprieties of the material.

The background for serial crystallography data can be noisy and difficult to compress to acceptable levels without very large error bounds. However, without preserving the data around the Bragg peaks carefully, the large error bounds needed to produce acceptable compression levels can cause the indexing and merging processes to fail, leading to failures in the later stages of the process. Given the importance of the peaks to recovering the unit cell and ultimately the electron density, one might guess that only the peaks can be stored. However, without preserving at least some of the background, we may inadvertently destroy some of the false negative peaks, thus limiting our ability to index, merge, and perform phase retrieval. By using knowledge of peak location determined by the peak finder we can assign error tolerances to the regions of interest and to the background, however, we can enable higher levels of compression.

A typical error-bounded lossy compressor will face challenges in compressing the background data. The reason is that the background is very noisy and appears to be uncorrelated. Since error-bounded lossy compressors such as SZ and ZFP are

designed to exploit spatial correlations in the data, the lack of correlation results in poor performance. To increase the correlation, we apply a 2x2 binning process to the background. This smooths the data and improves its compressibility while reducing the volume of the data by a factor of 4 on its own. Because we store the regions of interest separately and losslessly prior to binning, we maintain high resolution signal around key features which are critical to the scientific analysis while using lower resolution around regions of non-interest containing noise. As we will demonstrate in section 3.5, this preserves the scientific integrity of the data with very high compression ratios.

Figure 9 presents an overview of ROIBIN-SZ, the meta-compressor we designed that combines lossless compression of key regions of interest with binning, followed by aggressive binning and then error-bounded lossy compression of the remaining background data [150]. The combination of these two approaches allows high levels of compression at the extremely high line rates of modern and future beamlines. Together the pipeline is implemented as a hierarchical parallel code using the LibPressio and SZ3 libraries, where each of the tasks can be parallelized, and potentially in the future accelerated on the GPU, while giving us maximal flexibility to experiment with different compression pipelines. As an added benefit of our design, we can use the OptZConfig framework to autotune the runtime performance of ROIBIN-SZ and set the number of execution threads to use for each level of parallelism.

Lossy Compression for Instrument Data by TEZip.. Time Evolutionary Zip (TEZip) is a specialized tool designed for handling the compression of instrument data. In this context, it is investigated in the domain of lossy compression. TEZip's lossy compression methodology encompasses several key steps. To initiate, TEZip employs the recurrent neural network PredNet for training. PredNet's primary function is to predict expected image frames based on accessible frames. Subsequently, TEZip originates the disparities between these expected (predicted) frames and the real frames, generating d-(delta) frame that exhibits a higher inherent compressibility.

During the compression phase, we use the trained model to predict forthcoming frames. In this context, "Bi" represents

<figure><img src="image_11.png" /><figcaption>Figure 9: ROBIN-SZ overview diagram [150]</figcaption></figure>
fit well with general-purpose compressor algorithms. However, these specific attributes present an opportunity for developing highly effective and simple custom hardware compression algorithms. Previously (outside of the JLESC context), RIKEN, Argonne National Laboratory, and other institutes have proposed custom hardware compression algorithms for scientific instrument data and numerical computation and evaluated their characteristics [63, 134, 135, 147]. In the context of our JLESC collaborative project, we have recently initiated a new research and development activity on hardware compressor generator frameworks in 2023. This activity focuses on hardware compressor generator frameworks, aiming to support future scientific detector co-design efforts. We have introduced Stream-Pressor [156], a generator framework for designing, verifying, and estimating resources in streaming hardware compressor architectures. This framework is intended to assist compressor developers in exploring different compressor architectures with different compressor building blocks, evaluating their characteristics (latency, throughput, gate counts), and generating RTL code for integrating them into custom accelerator designs. Our motivation is to bridge the gap between software and hardware experts through this proposed framework as a co-design tool, leveraging open-source hardware design tools.

Lossy Compression for Instrument Data by ROIBIN-SZ.. In order to understand ROIBIN-SZ, it is helpful to understand the general serial crystallography analysis workflow. First, a peak finder is used to identify so-called Bragg spots/peaks that represent the positions of electrons during the imaging process of serial crystallography. Second, since the peak-finding process has some false negatives and false positives, indexing is used to estimate the unit cell parameters of the crystalline structure from the positions of the Bragg peaks and the background of the images. Third, the indexed unit cells are merged into a 3D diffraction volume. Fourth, phase retrieval is performed on the merged volume to reconstruct the electron density, which is then studied to understand the structure and proprieties of the material.

The background for serial crystallography data can be noisy and difficult to compress to acceptable levels without very large error bounds. However, without preserving the data around the Bragg peaks carefully, the large error bounds needed to produce acceptable compression levels can cause the indexing and merging processes to fail, leading to failures in the later stages of the process. Given the importance of the peaks to recovering the unit cell and ultimately the electron density, one might guess that only the peaks can be stored. However, without preserving at least some of the background, we may inadvertently destroy some of the false negative peaks, thus limiting our ability to index, merge, and perform phase retrieval. By using knowledge of peak location determined by the peak finder we can assign error tolerances to the regions of interest and to the background, however, we can enable higher levels of compression.

A typical error-bounded lossy compressor will face challenges in compressing the background data. The reason is that the background is very noisy and appears to be uncorrelated. Since error-bounded lossy compressors such as SZ and ZFP are

designed to exploit spatial correlations in the data, the lack of correlation results in poor performance. To increase the correlation, we apply a 2x2 binning process to the background. This smooths the data and improves its compressibility while reducing the volume of the data by a factor of 4 on its own. Because we store the regions of interest separately and losslessly prior to binning, we maintain high resolution signal around key features which are critical to the scientific analysis while using lower resolution around regions of non-interest containing noise. As we will demonstrate in section 3.5, this preserves the scientific integrity of the data with very high compression ratios.

Figure 9 presents an overview of ROIBIN-SZ, the meta-compressor we designed that combines lossless compression of key regions of interest with binning, followed by aggressive binning and then error-bounded lossy compression of the remaining background data [150]. The combination of these two approaches allows high levels of compression at the extremely high line rates of modern and future beamlines. Together the pipeline is implemented as a hierarchical parallel code using the LibPressio and SZ3 libraries, where each of the tasks can be parallelized, and potentially in the future accelerated on the GPU, while giving us maximal flexibility to experiment with different compression pipelines. As an added benefit of our design, we can use the OptZConfig framework to autotune the runtime performance of ROIBIN-SZ and set the number of execution threads to use for each level of parallelism.

Lossy Compression for Instrument Data by TEZip.. Time Evolutionary Zip (TEZip) is a specialized tool designed for handling the compression of instrument data. In this context, it is investigated in the domain of lossy compression. TEZip's lossy compression methodology encompasses several key steps. To initiate, TEZip employs the recurrent neural network PredNet for training. PredNet's primary function is to predict expected image frames based on accessible frames. Subsequently, TEZip originates the disparities between these expected (predicted) frames and the real frames, generating d-(delta) frame that exhibits a higher inherent compressibility.

During the compression phase, we use the trained model to predict forthcoming frames. In this context, "Bi" represents

<figure><img src="image_11.png" /><figcaption>Figure 9: ROBIN-SZ overview diagram [150]</figcaption></figure>
<figure><img src="image_12.png" /><figcaption>Figure 10: Workflow of TEZip</figcaption></figure>

the ith original frame from the time evolutionary data, which serves as a reference before compression, while "Pi" stands for the predicted frame corresponding to "Bi" (2). Subsequently, we calculate the discrepancies between "Bi" and "Pi," resulting in the creation of "Di," which signifies the d-(delta) frame (3). We then apply a series of encoding techniques developed within our framework, culminating in "Ci," which denotes the final compressed frame of "Bi" (4). We note that the first frame, "B0," lacks a previous frame for prediction, so we retain it in its original form (5).

### 3.4. Performance Evaluations

**TEZip Performance Evaluation.** The TEZip performance evaluation includes several types of instrument data, such as KITTI [51], XPCS [73], XCT2K/4K [73], Victoria [60], and Malaga [22]. In figure 10 compares with compression ratio performance TEZip and other compressors. Additionally, the initial evaluation of TEZip is based on the training root measure mean squared error (RMSE), discussed in Section 3. Table 1 lists RMSE values of the tested datasets.

<table><caption>Table 1: RMSE as a % of the value range of prediction mechanism in TEZip with different datasets trained on the KITTI-1 dataset</caption><thead><tr><td>Dataset</td><td>RMSE</td></tr></thead><tbody><tr><td>kitti-1 [51]</td><td>0.265</td></tr><tr><td>kitti-2 [51]</td><td>0.436</td></tr><tr><td>xpcs [73]</td><td>0.141</td></tr><tr><td>xct2k [73]</td><td>0.055</td></tr><tr><td>malaga [22]</td><td>0.308</td></tr><tr><td>victoria [60]</td><td>0.301</td></tr><tr><td>Eumet-19 [3]</td><td>0.196</td></tr></tbody></table>

In our study, we evaluated TEZip predictive model quality by segmenting the dataset into distinct subsets for training and testing. This approach subsequently evaluates prediction models for each training set across all testing datasets. Utilizing the test datasets, we calculated the root mean square error
$$RMSE = \sqrt{\frac{1}{n} \sum_{i=0}^{n} (x_i - p_i)^2},$$
that measures the average error between the predicted values and validation set on a per-pixel basis. We can further interpret this by dividing the RMSE by the value range (vr) and presenting the outcome as a percentage, that is, $RMSE_{rel} = RMSE/vr$. The original dataset has a value in the range of [0,255]. Table-1 shows the $RMSE_{rel}$ percentage

for the model trained on the kitti-1 dataset. The results highlighted an error rate of less than 0.5 percent across all datasets which shows the prediction method reduces errors. For additional information and results, refer to our previous work [122]. We'll also compare the quality of TEZip with another top compressor, they're tested at the same compression ratio, in Section 3.5. First, we evaluate the quality of the underlying prediction model used by TEZip as follows. First, we divide each of the various datasets into training and test subsets. We then train prediction models for each training set and evaluate it on the test sets from all datasets. With the test set and the prediction models, we can compute the Root Mean Square Error on the test set as follows $RMSE = \sqrt{\frac{1}{n} \sum_{i=0}^{n} (x_i - p_i)^2}$. The RMSE measures the average error in the predictor vs the validation set per pixel. We can contextualize this further by dividing RMSE by the value range (vr) and expressing the result as a percent, i.e., $RMSE_{rel} = RMSE/vr$. The original dataset has a value in the range of [0,255]. Table 1 presents the $RMSE_{rel}$ percentages for a model trained on the kitti-1 dataset. These results demonstrate less than 0.5% error across all datasets which underscores the effectiveness of the prediction method in minimizing prediction errors. For more details and results please see our prior work [122]. We will consider the comparative quality between TEZip and another leading compressor at the same compression ratio between compressors later in Section 3.5.

**ROIBIN-SZ Performance Evaluation.** ROIBIN-SZ utilizes multiple levels of parallelism to scale to the line rate of data produced by serial crystallography. Generally, ROIBIN-SZ scales linearly with the number of nodes each processing different subsets of the detector panels used in an experiment. Since its development, ROIBIN-SZ has been tested at line rate speeds as part of the Exascale Computing Project, and is now used as part of the data reduction pipeline for serial crystallography at SLAC. For more details please see our prior work [150]

### 3.5. Lossy Compression Quality Performance

In the lossy compression approach, TEZip adapts to different error bounds ($\alpha$). During experiments, we systematically adjusted $\alpha$ based on a method elaborated later in this section to cater to different datasets. To assess the performance of our lossy TEZip scheme, we conducted comparisons with established lossy compression methods such as SZ and zfp. Based on our time evaluation, we found that the compression time for lossy TEZip is higher. Also, when it comes to decompression, SZ significantly outperforms TEZip in terms of speed.

**TEZip Image Quality Comparison.** We applied the multiscale structure similarity index measure (MS-SSIM) to compare the image quality of decompressed frames between TEZip and ZFP, considering the maximum error bounds. The MS-SSIM index evaluates structural similarity at various scales between the image and the reference image. TEZip and zfp in terms of the image quality of the decompressed frames for the same maximum point-wise relative error bounds. MS-SSIM index is calculated by combining the MS-SSIM between different versions
<figure><img src="image_12.png" /><figcaption>Figure 10: Workflow of TEZip</figcaption></figure>

the ith original frame from the time evolutionary data, which serves as a reference before compression, while "Pi" stands for the predicted frame corresponding to "Bi" (2). Subsequently, we calculate the discrepancies between "Bi" and "Pi," resulting in the creation of "Di," which signifies the d-(delta) frame (3). We then apply a series of encoding techniques developed within our framework, culminating in "Ci," which denotes the final compressed frame of "Bi" (4). We note that the first frame, "B0," lacks a previous frame for prediction, so we retain it in its original form (5).

### 3.4. Performance Evaluations

**TEZip Performance Evaluation.** The TEZip performance evaluation includes several types of instrument data, such as KITTI [51], XPCS [73], XCT2K/4K [73], Victoria [60], and Malaga [22]. In figure 10 compares with compression ratio performance TEZip and other compressors. Additionally, the initial evaluation of TEZip is based on the training root measure mean squared error (RMSE), discussed in Section 3. Table 1 lists RMSE values of the tested datasets.

<table><caption>Table 1: RMSE as a % of the value range of prediction mechanism in TEZip with different datasets trained on the KITTI-1 dataset</caption><thead><tr><td>Dataset</td><td>RMSE</td></tr></thead><tbody><tr><td>kitti-1 [51]</td><td>0.265</td></tr><tr><td>kitti-2 [51]</td><td>0.436</td></tr><tr><td>xpcs [73]</td><td>0.141</td></tr><tr><td>xct2k [73]</td><td>0.055</td></tr><tr><td>malaga [22]</td><td>0.308</td></tr><tr><td>victoria [60]</td><td>0.301</td></tr><tr><td>Eumet-19 [3]</td><td>0.196</td></tr></tbody></table>

In our study, we evaluated TEZip predictive model quality by segmenting the dataset into distinct subsets for training and testing. This approach subsequently evaluates prediction models for each training set across all testing datasets. Utilizing the test datasets, we calculated the root mean square error
$$RMSE = \sqrt{\frac{1}{n} \sum_{i=0}^{n} (x_i - p_i)^2},$$
that measures the average error between the predicted values and validation set on a per-pixel basis. We can further interpret this by dividing the RMSE by the value range (vr) and presenting the outcome as a percentage, that is, $RMSE_{rel} = RMSE/vr$. The original dataset has a value in the range of [0,255]. Table-1 shows the $RMSE_{rel}$ percentage

for the model trained on the kitti-1 dataset. The results highlighted an error rate of less than 0.5 percent across all datasets which shows the prediction method reduces errors. For additional information and results, refer to our previous work[122]. We'll also compare the quality of TEZip with another top compressor, they're tested at the same compression ratio, in Section 3.5. First, we evaluate the quality of the underlying prediction model used by TEZip as follows. First, we divide each of the various datasets into training and test subsets. We then train prediction models for each training set and evaluate it on the test sets from all datasets. With the test set and the prediction models, we can compute the Root Mean Square Error on the test set as follows $RMSE = \sqrt{\frac{1}{n} \sum_{i=0}^{n} (x_i - p_i)^2}$. The RMSE measures the average error in the predictor vs the validation set per pixel. We can contextualize this further by dividing RMSE by the value range (vr) and expressing the result as a percent, i.e., $RMSE_{rel} = RMSE/vr$. The original dataset has a value in the range of [0,255]. Table 1 presents the $RMSE_{rel}$ percentages for a model trained on the kitti-1 dataset. These results demonstrate less than 0.5% error across all datasets which underscores the effectiveness of the prediction method in minimizing prediction errors. For more details and results please see our prior work[122]. We will consider the comparative quality between TEZip and another leading compressor at the same compression ratio between compressors later in Section 3.5.

**ROIBIN-SZ Performance Evaluation.** ROIBIN-SZ utilizes multiple levels of parallelism to scale to the line rate of data produced by serial crystallography. Generally, ROIBIN-SZ scales linearly with the number of nodes each processing different subsets of the detector panels used in an experiment. Since its development, ROIBIN-SZ has been tested at line rate speeds as part of the Exascale Computing Project, and is now used as part of the data reduction pipeline for serial crystallography at SLAC. For more details please see our prior work[150]

### 3.5. Lossy Compression Quality Performance

In the lossy compression approach, TEZip adapts to different error bounds ($\alpha$). During experiments, we systematically adjusted $\alpha$ based on a method elaborated later in this section to cater to different datasets. To assess the performance of our lossy TEZip scheme, we conducted comparisons with established lossy compression methods such as SZ and zfp. Based on our time evaluation, we found that the compression time for lossy TEZip is higher. Also, when it comes to decompression, SZ significantly outperforms TEZip in terms of speed.

**TEZip Image Quality Comparison.** We applied the multiscale structure similarity index measure (MS-SSIM) to compare the image quality of decompressed frames between TEZip and ZFP, considering the maximum error bounds. The MS-SSIM index evaluates structural similarity at various scales between the image and the reference image. TEZip and zfp in terms of the image quality of the decompressed frames for the same maximum point-wise relative error bounds. MS-SSIM index is calculated by combining the MS-SSIM between different versions
<figure><img src="image_13.png" /><figcaption>Figure 11: Compression ratios compared with baseline approach [127]</figcaption></figure>

<table><caption>Table 2: Comparison of TEZip and zfp image quality using MS-SSIM at the same compression ratio</caption><thead><tr><td>Dataset</td><td>TEZip</td><td>zfp</td></tr></thead><tbody><tr><td>kitti-1 [51]</td><td>0.9726</td><td>0.9553</td></tr><tr><td>kitti-2 [51]</td><td>0.9669</td><td>0.9582</td></tr><tr><td>xpcs [73]</td><td>0.9988</td><td>0.9786</td></tr><tr><td>xct4k [73]</td><td>0.9692</td><td>0.9537</td></tr><tr><td>malaga [22]</td><td>0.9813</td><td>0.9639</td></tr><tr><td>victoria [60]</td><td>0.9948</td><td>0.9840</td></tr></tbody></table>

of the image and reference image. We measure the MS-SSIM score for each frame and then calculate the mean across all frames. Table 2 demonstrates that TEZip consistently brings quality images while enhancing compression. We have compared compression ratios with other compressors for different error bounds. TEZip compression ratio is the highest for all datasets at an absolute error bound of 1e-06 and 1e-05. For more information please see our recent work [137].

**ROIBIN-SZ Reconstruction Performance.** Careful attention must be paid to the tuning of compression pipelines to ensure that the electron density can be reconstructed accurately. We describe the process for tuning these parameters in our work [150]. In Figure 12 we show the results of a reconstruction on a sample of Streptavidin. In Figure 13 we show the results of a reconstruction after compression and decompression using ROIBIN-SZ. As can be seen by visual inspection<fnref n="2" /> of the two images, the reconstruction quality is high, presenting nearly identical reconstructions indicating that the scientific integrity of the data is preserved—this while reducing the volume of the data 46.44× and up to 154.18× when combined with non-hit rejection.

## 3.6. Future Work

Our research has been focused on developing compression techniques and frameworks for both upstream and downstream edge computing. Within the JLESC initiative, we have initiated three significant projects: StreamPressor, ROIBIN-SZ, and TEZip.

StreamPressor continues to focus on research for a framework that evaluates hardware-level compressor algorithms. Currently, we assess the behavior of digitally generated circuits for

<figure><img src="image_14.png" /><figcaption>Figure 12: Original Streptavidin electron density reconstruction [150]</figcaption></figure>

compressors using cycle-accurate software simulators. In future work, we plan to include FPGA-specific optimizations in the framework, aiming to create additional computational possibilities in the upstream edge stage. Additionally, we are exploring potential synergies with downstream compressors, such as ROIBIN-SZ and TEZip.

We're upgrading TEZip to predict future data more accurately. Our focus is on understanding what happens next in instrument data. It will make instrument data smaller and faster. By fine-tuning how TEZip predicts, we can manage changes in data more efficiently.

For ROIBIN-SZ we are investigating the performance trade-offs when porting to the GPU and the appropriateness of different types of masking types to different kinds of datasets from beamlines. Adopting GPUs may allow us to reduce the amount of hardware required to compress the data.

We are improving TEZip to be more user-friendly by integrating it with LibPressio. This is important because many compressors, such as SZ, ZFP, MGARD, and TTHRESH, are written in a computer language called C/C++. By linking TEZip with LibPressio, we're making them work smoothly together. This means TEZip can use compressors like SZ through LibPressio, and other compressors can use TEZip. This enhancement makes TEZip more helpful and adaptable for various compression needs, bringing different compression technologies to-

<footnote n="2">²An astute reader may notice that this is not particularly precise to use visual inspection and it would argue that it is better to use image quality metrics such as PSNR, SSIM, or $L_\infty$ norm, or to present a difference plot. However, there is some amount of natural variation in the reconstructed image derived from both the individual scientist's discretion regarding how to parameterize the reconstruction and the particular versions of software packages used which can result in small translations or distortions in the reconstruction. Therefore, what is acceptable in the reconstruction would be unduly penalized by these measures because they favor pixel-for-pixel reproducibility over semantic similarity. There are more quantifiable measures from earlier stages of the analysis which we present in [150], but we omit them here for space.</footnote>
<figure><img src="image_13.png" /><figcaption>Figure 11: Compression ratios compared with baseline approach [127]</figcaption></figure>

<table><caption>Table 2: Comparison of TEZip and zfp image quality using MS-SSIM at the same compression ratio</caption><thead><tr><td>Dataset</td><td>TEZip</td><td>zfp</td></tr></thead><tbody><tr><td>kitti-1 [51]</td><td>0.9726</td><td>0.9553</td></tr><tr><td>kitti-2 [51]</td><td>0.9669</td><td>0.9582</td></tr><tr><td>xpcs [73]</td><td>0.9988</td><td>0.9786</td></tr><tr><td>xct4k [73]</td><td>0.9692</td><td>0.9537</td></tr><tr><td>malaga [22]</td><td>0.9813</td><td>0.9639</td></tr><tr><td>victoria [60]</td><td>0.9948</td><td>0.9840</td></tr></tbody></table>

of the image and reference image. We measure the MS-SSIM score for each frame and then calculate the mean across all frames. Table 2 demonstrates that TEZip consistently brings quality images while enhancing compression. We have compared compression ratios with other compressors for different error bounds. TEZip compression ratio is the highest for all datasets at an absolute error bound of 1e-06 and 1e-05. For more information please see our recent work [137].

### ROIBIN-SZ Reconstruction Performance.
Careful attention must be paid to the tuning of compression pipelines to ensure that the electron density can be reconstructed accurately. We describe the process for tuning these parameters in our work [150]. In Figure 12 we show the results of a reconstruction on a sample of Streptavidin. In Figure 13 we show the results of a reconstruction after compression and decompression using ROIBIN-SZ. As can be seen by visual inspection<fnref n="2" /> of the two images, the reconstruction quality is high, presenting nearly identical reconstructions indicating that the scientific integrity of the data is preserved—this while reducing the volume of the data 46.44× and up to 154.18× when combined with non-hit rejection.

## 3.6. Future Work

Our research has been focused on developing compression techniques and frameworks for both upstream and downstream edge computing. Within the JLESC initiative, we have initiated three significant projects: StreamPressor, ROIBIN-SZ, and TEZip.

StreamPressor continues to focus on research for a framework that evaluates hardware-level compressor algorithms. Currently, we assess the behavior of digitally generated circuits for

<figure><img src="image_14.png" /><figcaption>Figure 12: Original Streptavidin electron density reconstruction [150]</figcaption></figure>

compressors using cycle-accurate software simulators. In future work, we plan to include FPGA-specific optimizations in the framework, aiming to create additional computational possibilities in the upstream edge stage. Additionally, we are exploring potential synergies with downstream compressors, such as ROIBIN-SZ and TEZip.

We're upgrading TEZip to predict future data more accurately. Our focus is on understanding what happens next in instrument data. It will make instrument data smaller and faster. By fine-tuning how TEZip predicts, we can manage changes in data more efficiently.

For ROIBIN-SZ we are investigating the performance trade-offs when porting to the GPU and the appropriateness of different types of masking types to different kinds of datasets from beamlines. Adopting GPUs may allow us to reduce the amount of hardware required to compress the data.

We are improving TEZip to be more user-friendly by integrating it with LibPressio. This is important because many compressors, such as SZ, ZFP, MGARD, and TTHRESH, are written in a computer language called C/C++. By linking TEZip with LibPressio, we're making them work smoothly together. This means TEZip can use compressors like SZ through LibPressio, and other compressors can use TEZip. This enhancement makes TEZip more helpful and adaptable for various compression needs, bringing different compression technologies to-

<footnote n="2">²An astute reader may notice that this is not particularly precise to use visual inspection and it would argue that it is better to use image quality metrics such as PSNR, SSIM, or $L_\infty$ norm, or to present a difference plot. However, there is some amount of natural variation in the reconstructed image derived from both the individual scientist's discretion regarding how to parameterize the reconstruction and the particular versions of software packages used which can result in small translations or distortions in the reconstruction. Therefore, what is acceptable in the reconstruction would be unduly penalized by these measures because they favor pixel-for-pixel reproducibility over semantic similarity. There are more quantifiable measures from earlier stages of the analysis which we present in [150], but we omit them here for space.</footnote>
<figure><img src="image_15.png" /><figcaption>Figure 13: ROIBIN-SZ Streptavidin electron density reconstruction. 46.44× smaller than the original, and 154.18× when combined with non-hit rejection [150]</figcaption></figure>

together to handle data more effectively.

More broadly, more work is needed to expand the consid-
erations of these kinds of techniques across different kinds of
beamlines and experimental methodologies and detector types
that produce different structures of data. We encourage the
community to consider the compression approaches developed
in our work.

4. Compression for Numerical Methods

The work described in this section was initiated as part of
a collaboration between ANL and Inria in the context of the
JLESC initiative [6] and then extended in [7]. It involves many
collaborators who are not co-authors of the present survey but
who deserve to be acknowledged; namely, Olivier Coulaud,
Martina Innacito, Xin Liang, Gilles Marait, and Nick Schenkels.

Compression is ubiquitous in scientific computing in gen-
eral and numerical linear algebra in particular. One of the most
well-known methods for compression in this latter field is trun-
cated singular value decomposition (TSVD). TSVD allows for
compressing matrices in some optimum sense. However, there
are fewer techniques for compressing vectors. An old but cur-
rently intensively studied method is to design numerical algo-
rithms able to use mixed-precision arithmetic. These methods
have been recently reviewed in [69, 5]. Still, data are stored in
a way that sticks to the hardware processing capacity, typically
in the form of 64-, 32-, and 16-bit words.

The idea of variable-accuracy storage is instead to rely on
a compressor such as SZ to compress vectors independently
from hardware constraints and apply it to the solution of large
sparse linear systems. In [6] we investigated how to exploit
that to compress the non-orthogonal basis of flexible GMRES
(FGMRES) [128] while maintaining a result in high accuracy.
In another study [7] we investigated how to compress the (or-
thogonal) basis of GMRES [129] to reach a prescribed accuracy
(typically lower than the machine precision).

In the first study [6], we examined the FGMRES algorithm,
which requires the storage of a basis for the Krylov subspace

and for the search space spanned by the solutions of the pre-
conditioning systems. We showed that the vectors spanning
this search space Z can be compressed by looking at the combi-
nation of FGMRES and compression in the context of inexact
Krylov subspace methods. This allowed us to derive a bound on
the normwise relative compression error in each iteration. We
used this bound to formulate several different practical com-
pression strategies and validate and compare them through nu-
merical experiments.

<figure><img src="image_16.png" /><figcaption>Figure 14: Total compression ratio of the search space Z for various compression strategies; each bar per strategy corresponds to a different matrix (see [6] for details). Horizontal reference lines are added at $\rho = 2$ and 4 corresponding to ideal (no extra iteration needed) 32- and 16-bit uniform storage, respectively.</figcaption></figure>

In the second study [7], we investigated the backward sta-
bility of two backward-stable implementations of the GMRES
method, namely, the so-called modified Gram–Schmidt (MGS)
and the Householder variants. When the data representation
of vectors introduces componentwise perturbations, we showed
that the existing backward stability analyses of MGS-GMRES [119]
and Householder-GMRES [151, 39] still apply. We illustrated
this backward stability property in a practical context where the
agnostic SZ lossy compressor is employed and enables the re-
duction of the memory requirement to store the orthonormal
Arnoldi basis or the Householder reflectors. We also consid-
ered the normwise relative perturbations of the vector storage
from an experimental perspective, showing numerically that the
backward stability is maintained; that is, the attainable norm-
wise backward error is of the same order as the normwise per-
turbations induced by the data storage. We illustrated it with
numerical experiments in two practical contexts. The first one
corresponds to the use of the SZ-agnostic compressor where
vector compression is controlled normwise. The second one
arises in the solution of tensor linear systems, where low-rank
tensor approximations based on the tensor-train approach [118]
are considered to tackle the curse of dimensionality.

We refer to the respective reports [6, 7] for further details.

5. Fast Data compression for shared memory mixed preci-
sion algorithms

On virtually all modern hardware architectures, the perfor-
mance of sparse linear algebra is limited by the memory band-
width. In consequence, to accelerate the algorithms, it is not
<figure><img src="image_15.png" /><figcaption>Figure 13: ROIBIN-SZ Streptavidin electron density reconstruction. 46.44× smaller than the original, and 154.18× when combined with non-hit rejection [150]</figcaption></figure>

together to handle data more effectively.

More broadly, more work is needed to expand the consid-
erations of these kinds of techniques across different kinds of
beamlines and experimental methodologies and detector types
that produce different structures of data. We encourage the
community to consider the compression approaches developed
in our work.

4. Compression for Numerical Methods

The work described in this section was initiated as part of
a collaboration between ANL and Inria in the context of the
JLESC initiative [6] and then extended in [7]. It involves many
collaborators who are not co-authors of the present survey but
who deserve to be acknowledged; namely, Olivier Coulaud,
Martina Innacito, Xin Liang, Gilles Marait, and Nick Schenkels.

Compression is ubiquitous in scientific computing in gen-
eral and numerical linear algebra in particular. One of the most
well-known methods for compression in this latter field is trun-
cated singular value decomposition (TSVD). TSVD allows for
compressing matrices in some optimum sense. However, there
are fewer techniques for compressing vectors. An old but cur-
rently intensively studied method is to design numerical algo-
rithms able to use mixed-precision arithmetic. These methods
have been recently reviewed in [69, 5]. Still, data are stored in
a way that sticks to the hardware processing capacity, typically
in the form of 64-, 32-, and 16-bit words.

The idea of variable-accuracy storage is instead to rely on
a compressor such as SZ to compress vectors independently
from hardware constraints and apply it to the solution of large
sparse linear systems. In [6] we investigated how to exploit
that to compress the non-orthogonal basis of flexible GMRES
(FGMRES) [128] while maintaining a result in high accuracy.
In another study [7] we investigated how to compress the (or-
thogonal) basis of GMRES [129] to reach a prescribed accuracy
(typically lower than the machine precision).

In the first study [6], we examined the FGMRES algorithm,
which requires the storage of a basis for the Krylov subspace

and for the search space spanned by the solutions of the pre-
conditioning systems. We showed that the vectors spanning
this search space Z can be compressed by looking at the combi-
nation of FGMRES and compression in the context of inexact
Krylov subspace methods. This allowed us to derive a bound on
the normwise relative compression error in each iteration. We
used this bound to formulate several different practical com-
pression strategies and validate and compare them through nu-
merical experiments.

<figure><img src="image_16.png" /><figcaption>Figure 14: Total compression ratio of the search space Z for various compression strategies; each bar per strategy corresponds to a different matrix (see [6] for details). Horizontal reference lines are added at $\rho = 2$ and 4 corresponding to ideal (no extra iteration needed) 32- and 16-bit uniform storage, respectively.</figcaption></figure>

In the second study [7], we investigated the backward sta-
bility of two backward-stable implementations of the GMRES
method, namely, the so-called modified Gram–Schmidt (MGS)
and the Householder variants. When the data representation
of vectors introduces componentwise perturbations, we showed
that the existing backward stability analyses of MGS-GMRES [119]
and Householder-GMRES [151, 39] still apply. We illustrated
this backward stability property in a practical context where the
agnostic SZ lossy compressor is employed and enables the re-
duction of the memory requirement to store the orthonormal
Arnoldi basis or the Householder reflectors. We also consid-
ered the normwise relative perturbations of the vector storage
from an experimental perspective, showing numerically that the
backward stability is maintained; that is, the attainable norm-
wise backward error is of the same order as the normwise per-
turbations induced by the data storage. We illustrated it with
numerical experiments in two practical contexts. The first one
corresponds to the use of the SZ-agnostic compressor where
vector compression is controlled normwise. The second one
arises in the solution of tensor linear systems, where low-rank
tensor approximations based on the tensor-train approach [118]
are considered to tackle the curse of dimensionality.

We refer to the respective reports [6, 7] for further details.

5. Fast Data compression for shared memory mixed preci-
sion algorithms

On virtually all modern hardware architectures, the perfor-
mance of sparse linear algebra is limited by the memory band-
width. In consequence, to accelerate the algorithms, it is not
important to accelerate the arithmetic operations (e.g., by us-
ing low precision), but to accelerate the memory access, e.g.,
by reducing the data volume. Reducing the data volume can
even come at the cost of additional in-register operations as pro-
cessors are increasingly over-provisioned for FLOP/s. The ap-
proach we investigate for shared memory architectures is there-
fore based on the idea of decoupling the arithmetic precision
from the memory precision and using in-register data compres-
sion. The algorithms we derive from this concept employ IEEE
double precision for the arithmetic operations and then com-
press the double-precision data in registers to reduce pressure
on the main memory bandwidth. Different concepts can be used
for compression. Either IEEE standard low-precision formats,
such as single or half-precision, or arbitrary storage formats,
including compression schemes.

An example of an algorithm using the idea of data compres-
sion is the Compressed Basis GMRES (CB-GMRES) solver
presented in [9]. GMRES is a Krylov solver that creates and ex-
pands an orthogonal basis of Krylov vectors that span a Krylov
subspace. The CB-GMRES method, which we review in detail
in the next section, compresses the basis vectors to reduce the
memory access volume.

5.1. CB-GMRES

CB-GMRES is based on classical Gram-Schmidt with re-
orthogonalization. Similar to [7], it compresses the basis vec-
tors to decrease the data access volume in the orthogonalization
step, but CB-GMRES focuses on accelerating the algorithm's
execution time. The compression is handled with the memory
accessor introduced in [59], which is a lightweight software in-
terface that allows us to quickly select a suitable memory format
without touching the algorithm. In [9] we presented numerical
experiments using two separate approaches: (1) Using IEEE
formats half-, single- and double-precision as the storage for-
mat; and (2) Using a fixed-point format based on 32- and 16-bit
integers that store the significand for each element with an ad-
ditional scalar for each vector. Both of them are lossy compres-
sion techniques that compress each vector element individually.

Using IEEE formats as compression formats is easy to im-
plement. We can simply leverage the existing cast functions
to transform between the arithmetic double-precision and the
chosen storage format. Since these operations are supported
by hardware, the in-register conversion routine does not in-
crease the runtime. This is similar to other mixed-precision al-
gorithms with the difference that all arithmetic operations hap-
pen in double-precision.

Also, the fixed-point format is a lossy compression format.
Conversion is significantly more complicated. Initially, the max-
imum value for each vector has to be determined. The fixed-
point format is a lossy compression format as well. However,
the conversion is significantly more complicated. Initially, the
maximum value for each vector has to be determined. Then,
all vector values are stored as integers by uniformly scaling be-
tween zero and the identified maximum value. This requires
one additional double-precision value per vector that needs to
be read for every value.

The experimental evaluation in [9] demonstrates that using
single-precision as the storage format gives the best trade-off
between convergence and runtime performance. The analysis
is based on 49 test matrices and time-to-convergence analysis
for both unpreconditioned and preconditioned GMRES. The
experimental results show that convergence is very similar to
double precision while providing an average speedup of 1.4.
Using half-precision as the basis storage format allows for even
faster GMRES iterations but impacts convergence, requiring ei-
ther more iterations or not converging at all.

The 32-bit fixed-point format has similar convergence char-
acteristics to single- and double-precision, but the conversion
introduces a small overhead, making it inferior to single-precision
storage overall. The 16-bit fixed-point basis storage is not com-
petitive with the other strategies.

5.2. Evaluating CB-GMRES with compression using LibPressio

As the next step, we want to test the feasibility of using
more sophisticated compression algorithms for the basis com-
pression. We use LibPressio [149] to get access to many dif-
ferent compressors and *initially* only simulate the compression.
For that, we compress the Krylov vectors in each iteration, fol-
lowed by an immediate decompression. This simulates the in-
formation loss induced by the compression scheme without writ-
ing a time-consuming high-performance implementation first.
At the same time, this experimental setup is only a first step, as
*it does not allow for evaluation of the runtime benefits*. The ex-
periment aims to identify compression strategies that success-
fully preserve the information in the Krylov basis vectors. Once
compression strategies have been identified to be suitable suit-
able compression strategies have been identified, they can be
integrated into the accessor for production runs. We use the
following *lossy* compressors with various settings: Bit groom-
ing [159], digit-rounding [34], SZ3 [101] with absolute error
bounds, and zfp [103] with absolute error bounds.

<figure><img src="image_17.png" /><figcaption>Figure 15: Residual norm plot for different precisions and the zfp and SZ3 compressions for the Transport matrix. The iteration count was fixed to 5000 for this experiment; the average number of bits per value is shown as the suffix of the legend the absolute error bound magnitude of zfp and SZ3 is shown as the suffix of the legend.</figcaption></figure>

We run all mentioned compression strategies with all ma-
trices in the original paper [9]. Both bit grooming and digit-
rounding fail for some of the test cases, so we exclude them
from further analysis.

Figure 15 shows the residual norm plots exemplary for the
Transport matrix with the zfp and SZ3 compressors and the
important to accelerate the arithmetic operations (e.g., by us-
ing low precision), but to accelerate the memory access, e.g.,
by reducing the data volume. Reducing the data volume can
even come at the cost of additional in-register operations as pro-
cessors are increasingly over-provisioned for FLOP/s. The ap-
proach we investigate for shared memory architectures is there-
fore based on the idea of decoupling the arithmetic precision
from the memory precision and using in-register data compres-
sion. The algorithms we derive from this concept employ IEEE
double precision for the arithmetic operations and then com-
press the double-precision data in registers to reduce pressure
on the main memory bandwidth. Different concepts can be used
for compression. Either IEEE standard low-precision formats,
such as single or half-precision, or arbitrary storage formats,
including compression schemes.

An example of an algorithm using the idea of data compres-
sion is the Compressed Basis GMRES (CB-GMRES) solver
presented in [9]. GMRES is a Krylov solver that creates and ex-
pands an orthogonal basis of Krylov vectors that span a Krylov
subspace. The CB-GMRES method, which we review in detail
in the next section, compresses the basis vectors to reduce the
memory access volume.

5.1. CB-GMRES

CB-GMRES is based on classical Gram-Schmidt with re-
orthogonalization. Similar to [7], it compresses the basis vec-
tors to decrease the data access volume in the orthogonalization
step, but CB-GMRES focuses on accelerating the algorithm's
execution time. The compression is handled with the memory
accessor introduced in [59], which is a lightweight software in-
terface that allows us to quickly select a suitable memory format
without touching the algorithm. In [9] we presented numerical
experiments using two separate approaches: (1) Using IEEE
formats half-, single- and double-precision as the storage for-
mat; and (2) Using a fixed-point format based on 32- and 16-bit
integers that store the significand for each element with an ad-
ditional scalar for each vector. Both of them are lossy compres-
sion techniques that compress each vector element individually.

Using IEEE formats as compression formats is easy to im-
plement. We can simply leverage the existing cast functions
to transform between the arithmetic double-precision and the
chosen storage format. Since these operations are supported
by hardware, the in-register conversion routine does not in-
crease the runtime. This is similar to other mixed-precision al-
gorithms with the difference that all arithmetic operations hap-
pen in double-precision.

Also, the fixed-point format is a lossy compression format.
Conversion is significantly more complicated. Initially, the max-
imum value for each vector has to be determined. The fixed-
point format is a lossy compression format as well. However,
the conversion is significantly more complicated. Initially, the
maximum value for each vector has to be determined. Then,
all vector values are stored as integers by uniformly scaling be-
tween zero and the identified maximum value. This requires
one additional double-precision value per vector that needs to
be read for every value.

The experimental evaluation in [9] demonstrates that using
single-precision as the storage format gives the best trade-off
between convergence and runtime performance. The analysis
is based on 49 test matrices and time-to-convergence analysis
for both unpreconditioned and preconditioned GMRES. The
experimental results show that convergence is very similar to
double precision while providing an average speedup of 1.4.
Using half-precision as the basis storage format allows for even
faster GMRES iterations but impacts convergence, requiring ei-
ther more iterations or not converging at all.

The 32-bit fixed-point format has similar convergence char-
acteristics to single- and double-precision, but the conversion
introduces a small overhead, making it inferior to single-precision
storage overall. The 16-bit fixed-point basis storage is not com-
petitive with the other strategies.

5.2. Evaluating CB-GMRES with compression using LibPressio

As the next step, we want to test the feasibility of using
more sophisticated compression algorithms for the basis com-
pression. We use LibPressio [149] to get access to many dif-
ferent compressors and *initially* only simulate the compression.
For that, we compress the Krylov vectors in each iteration, fol-
lowed by an immediate decompression. This simulates the in-
formation loss induced by the compression scheme without writ-
ing a time-consuming high-performance implementation first.
At the same time, this experimental setup is only a first step, as
*it does not allow for evaluation of the runtime benefits*. The ex-
periment aims to identify compression strategies that success-
fully preserve the information in the Krylov basis vectors. Once
compression strategies have been identified to be suitable suit-
able compression strategies have been identified, they can be
integrated into the accessor for production runs. We use the
following *lossy* compressors with various settings: Bit groom-
ing [159], digit-rounding [34], SZ3 [101] with absolute error
bounds, and zfp [103] with absolute error bounds.

<figure><img src="image_17.png" /><figcaption>Figure 15: Residual norm plot for different precisions and the zfp and SZ3 compressions for the Transport matrix. The iteration count was fixed to 5000 for this experiment; the average number of bits per value is shown as the suffix of the legend the absolute error bound magnitude of zfp and SZ3 is shown as the suffix of the legend.</figcaption></figure>

We run all mentioned compression strategies with all ma-
trices in the original paper [9]. Both bit grooming and digit-
rounding fail for some of the test cases, so we exclude them
from further analysis.

Figure 15 shows the residual norm plots exemplary for the
Transport matrix with the zfp and SZ3 compressors and the
<table><caption>Table 3: Compression details for the benchmark run shown in Figure 15</caption><thead><tr><th>Name</th><th>absolute error bound used</th><th>average bits per value</th><th>Final relative residual norm</th></tr></thead><tbody><tr><td>float16</td><td>-</td><td>16</td><td>8.2 · 10<sup>-12</sup></td></tr><tr><td>float32</td><td>-</td><td>32</td><td>2.0 · 10<sup>-13</sup></td></tr><tr><td>float64</td><td>-</td><td>64</td><td>4.8 · 10<sup>-13</sup></td></tr><tr><td>sz3_06</td><td>10<sup>-6</sup></td><td>12</td><td>7.8 · 10<sup>-11</sup></td></tr><tr><td>sz3_07</td><td>10<sup>-7</sup></td><td>17</td><td>3.9 · 10<sup>-11</sup></td></tr><tr><td>sz3_08</td><td>10<sup>-8</sup></td><td>41</td><td>1.3 · 10<sup>-12</sup></td></tr><tr><td>zfp_06</td><td>10<sup>-6</sup></td><td>16</td><td>4.2 · 10<sup>-8</sup></td></tr><tr><td>zfp_10</td><td>10<sup>-10</sup></td><td>28</td><td>4.2 · 10<sup>-12</sup></td></tr></tbody></table>

common IEEE floating-point precisions from the original paper [9]. Table 3 details the most important metrics of this benchmark run. For this experiment, we always run 5,000 iterations of CB-GMRES. Single- and double-precision display almost the same convergence behavior, while half-precision is two orders of magnitude convergence delay, which is a common occurrence. From the shown SZ3 and zfp convergence, only sz3_41 sz3_08 (which needs 41 bits per value on average) and zfp_28 zfp_10 (which needs 28 bits per value on average) show convergence behavior similar to double-precision, but both of them use roughly the same memory footprint as single-precision storage. When looking at all the results, none of our settings consistently beat the single-precision storage format. For some matrices, the absolute error bound of $10^{-8}$ with SZ3 results in an average of 24 bits per value and is enough to achieve the same convergence behavior as double-precision, but the same settings yield 60 bits per value for other scenarios requiring tuning for each matrix for the variable bitrate versions of SZ3 and zfp used here. While these particular algorithms perform poorly with respect to concerning the residual norm performance per bitrate, the fact that very low bitrates can perform comparability to double precision suggests that a more customized scheme may be possible for higher residual norm performance with lower bitrates. Additionally, for fast performance on hardware, decompression needs to be possible in registers or shared memory on the GPU and need fine-grained access to individual blocks of values. Neither the variable bitrate modes of zfp nor SZ3 allow for this kind of access as the decoding step requires space to decompress on account of the encoding stages<fnref n="3" />. An algorithm that addresses these concerns would need to be fixed-rate and use a very limited number of both instructions and memory to meet the performance demands compared to floating point truncation methods.

As part of our investigation, we also analyzed the correlation of the values of the Krylov vectors. After the first iteration, no correlation could be identified in the mantissa's of the floating point values, which explains the low compression rates for SZ3 and zfp<fnref n="4" />. Compressors such as SZ3 and zfp decorrelate

data using techniques such as prediction or a near orthogonal transform, respectively. On spatially correlated data, the decorrelation stage of these compressors increases the sharpness and decreases the spread of the distribution of input values by recognizing the predictable nature of the data as a function of nearby points allowing the values to be encoded more efficiently. However, when these techniques are applied to random data, they at best have no effect, and at worst introduce overhead to store the metadata used to undo the decorrelation steps (such as prediction type indexes in SZ). This likely is the cause of the poor residual norm performance per bit of storage. Algorithms that respond to this concern would need to be designed not to exploit correlations in the mantissa bits.

### 5.3. Future work

Future work will focus on analyzing the Krylov basis vectors in more detail to derive suitable compression schemes that can operate at very high bandwidths. As the compression happens in processor registers, the chunk size used in the compression needs to be small. At the same time, information loss can be reduced when compressing larger chunks. Additionally, the decompression must be lightweight in order to reach the memory bandwidth. SZ3 and zfp are too complex to achieve that, so we need a more simplistic strategy that might operate on small data blocks. We will investigate how to draw a balance to preserve information while achieving high performance.

## 6. Compression for AI

### 6.1. Introduction

Deep neural networks (DNNs) have rapidly evolved into the state-of-the-art technique for various artificial intelligence tasks across multiple science and technology domains, including image and vision recognition [132], recommendation systems [152], and natural language processing [29]. DNNs contain millions of parameters in an unparalleled representation, efficiently modeling complex nonlinearities. Numerous works [88, 136, 68] have suggested that using either deeper or wider DNNs is an effective way to improve analysis quality. Many recent DNNs have become significantly deeper and/or wider [153, 76].

The continually expanding size of DNN models generates a significant volume of intermediate data throughout the training process. This expansion results in a substantial increase in model size, which poses challenges for synchronization and inference. Similar to the challenges faced in scientific applications, the widening performance gap between computation and memory/storage systems presents data-related obstacles in deep learning. These challenges can be viewed primarily from three perspectives: (1) frequent large model/gradient synchronization against limited bandwidth; (2) extremely large intermediate data against limited GPU memory during training; and (3) effective handling of sizable input data for the model within the constraints of limited memory and bandwidth.

<footnote n="3">³While zfp does offer a fixed rate mode, it presents other challenges for this kind of data we will address in the next paragraph.</footnote>

<footnote n="4">⁴We do observe correlated patterns in the initial values of the Krylov vectors, but this is attributable to the initialization performed by the solver. After one iteration, the patterns we observe in the initial values disappear.</footnote>
<table><caption>Table 3: Compression details for the benchmark run shown in Figure 15</caption><thead><tr><th>Name</th><th>absolute error bound used</th><th>average bits per value</th><th>Final relative residual norm</th></tr></thead><tbody><tr><td>float16</td><td>-</td><td>16</td><td>8.2 · 10<sup>-12</sup></td></tr><tr><td>float32</td><td>-</td><td>32</td><td>2.0 · 10<sup>-13</sup></td></tr><tr><td>float64</td><td>-</td><td>64</td><td>4.8 · 10<sup>-13</sup></td></tr><tr><td>sz3_06</td><td>10<sup>-6</sup></td><td>12</td><td>7.8 · 10<sup>-11</sup></td></tr><tr><td>sz3_07</td><td>10<sup>-7</sup></td><td>17</td><td>3.9 · 10<sup>-11</sup></td></tr><tr><td>sz3_08</td><td>10<sup>-8</sup></td><td>41</td><td>1.3 · 10<sup>-12</sup></td></tr><tr><td>zfp_06</td><td>10<sup>-6</sup></td><td>16</td><td>4.2 · 10<sup>-8</sup></td></tr><tr><td>zfp_10</td><td>10<sup>-10</sup></td><td>28</td><td>4.2 · 10<sup>-12</sup></td></tr></tbody></table>

common IEEE floating-point precisions from the original paper [9]. Table 3 details the most important metrics of this benchmark run. For this experiment, we always run 5,000 iterations of CB-GMRES. Single- and double-precision display almost the same convergence behavior, while half-precision is two orders of magnitude convergence delay, which is a common occurrence. From the shown SZ3 and zfp convergence, only sz3_41 sz3_08 (which needs 41 bits per value on average) and zfp_28 zfp_10 (which needs 28 bits per value on average) show convergence behavior similar to double-precision, but both of them use roughly the same memory footprint as single-precision storage. When looking at all the results, none of our settings consistently beat the single-precision storage format. For some matrices, the absolute error bound of $10^{-8}$ with SZ3 results in an average of 24 bits per value and is enough to achieve the same convergence behavior as double-precision, but the same settings yield 60 bits per value for other scenarios requiring tuning for each matrix for the variable bitrate versions of SZ3 and zfp used here. While these particular algorithms perform poorly with respect to concerning the residual norm performance per bitrate, the fact that very low bitrates can perform comparability to double precision suggests that a more customized scheme may be possible for higher residual norm performance with lower bitrates. Additionally, for fast performance on hardware, decompression needs to be possible in registers or shared memory on the GPU and need fine-grained access to individual blocks of values. Neither the variable bitrate modes of zfp nor SZ3 allow for this kind of access as the decoding step requires space to decompress on account of the encoding stages<fnref n="3" />. An algorithm that addresses these concerns would need to be fixed-rate and use a very limited number of both instructions and memory to meet the performance demands compared to floating point truncation methods.

As part of our investigation, we also analyzed the correlation of the values of the Krylov vectors. After the first iteration, no correlation could be identified in the mantissa's of the floating point values, which explains the low compression rates for SZ3 and zfp<fnref n="4" />. Compressors such as SZ3 and zfp decorrelate

data using techniques such as prediction or a near orthogonal transform, respectively. On spatially correlated data, the decorrelation stage of these compressors increases the sharpness and decreases the spread of the distribution of input values by recognizing the predictable nature of the data as a function of nearby points allowing the values to be encoded more efficiently. However, when these techniques are applied to random data, they at best have no effect, and at worst introduce overhead to store the metadata used to undo the decorrelation steps (such as prediction type indexes in SZ). This likely is the cause of the poor residual norm performance per bit of storage. Algorithms that respond to this concern would need to be designed not to exploit correlations in the mantissa bits.

### 5.3. Future work

Future work will focus on analyzing the Krylov basis vectors in more detail to derive suitable compression schemes that can operate at very high bandwidths. As the compression happens in processor registers, the chunk size used in the compression needs to be small. At the same time, information loss can be reduced when compressing larger chunks. Additionally, the decompression must be lightweight in order to reach the memory bandwidth. SZ3 and zfp are too complex to achieve that, so we need a more simplistic strategy that might operate on small data blocks. We will investigate how to draw a balance to preserve information while achieving high performance.

## 6. Compression for AI

### 6.1. Introduction

Deep neural networks (DNNs) have rapidly evolved into the state-of-the-art technique for various artificial intelligence tasks across multiple science and technology domains, including image and vision recognition [132], recommendation systems [152], and natural language processing [29]. DNNs contain millions of parameters in an unparalleled representation, efficiently modeling complex nonlinearities. Numerous works [88, 136, 68] have suggested that using either deeper or wider DNNs is an effective way to improve analysis quality. Many recent DNNs have become significantly deeper and/or wider [153, 76].

The continually expanding size of DNN models generates a significant volume of intermediate data throughout the training process. This expansion results in a substantial increase in model size, which poses challenges for synchronization and inference. Similar to the challenges faced in scientific applications, the widening performance gap between computation and memory/storage systems presents data-related obstacles in deep learning. These challenges can be viewed primarily from three perspectives: (1) frequent large model/gradient synchronization against limited bandwidth; (2) extremely large intermediate data against limited GPU memory during training; and (3) effective handling of sizable input data for the model within the constraints of limited memory and bandwidth.

<footnote n="3">³While zfp does offer a fixed rate mode, it presents other challenges for this kind of data we will address in the next paragraph.</footnote>

<footnote n="4">⁴We do observe correlated patterns in the initial values of the Krylov vectors, but this is attributable to the initialization performed by the solver. After one iteration, the patterns we observe in the initial values disappear.</footnote>
## 6.2. DNN Model Compression

As pointed out by Wang et al. [153], the deep learning community acknowledges that increasing the scales of DNNs can improve the inference accuracy of image recognition tasks. For example, a 9-layer AlexNet, proposed by Krizhevsky et al. [88], won the 2012 ImageNet Large-Scale Visual Recognition Challenge (ILSVRC) [91] with a top-5 accuracy of 83%. In the 2014 ILSVRC, a 22-layer GoogLeNet [136] proposed by Szegedy et al. further improved the top-5 accuracy record to 93.3%. In the 2015 ILSVRC, He et al. [68] proposed a 152-layer ResNet, which set a new record of 96.43%. More recently, models such as GPT-3 [49] have exceeded 100 billion parameters, offering groundbreaking capabilities beyond those of conventional neural networks. This trend suggests that networks will continue to grow larger. Moreover, we note that during training, a model's gradient typically shares a size equivalence with the model itself. Consequently, the substantial model size presents communication challenges when exchanging gradients during training and difficulties in deploying these models on platforms with limited resources. Researchers at JLESC have proposed advanced lossy compression techniques for models aimed primarily at facilitating deployment on resource-limited platforms.

Specifically, Jin et al. [76] proposed DeepSZ, a lossy compression framework for DNNs. DeepSZ adapts the SZ lossy compression framework [36, 139, 98] to the context of DNN compression. This work enhances the efficiency of deploying DNN models on end devices with constrained bandwidth. Additionally, it also enables the storage of a larger number of models on each device, thereby significantly broadening their utility and application scope. In this adaptation, SZ achieves a much higher compression ratio for the compression of nonzero weights compared with other state-of-the-art compressors such as ZFP [103], especially due to its efficient linear-scaling quantization, in contrast to the simple vector quantization applied to the original weights in other related work [57, 65]. The general workflow of the DeepSZ framework is presented in Figure 16. As illustrated, DeepSZ consists of four key steps: network pruning, error-bound assessment, optimization of error-bound configuration, and generation of the compressed model. The first step involves adopting network pruning to reduce network complexity and mitigate the overfitting problem caused by a large number of parameters. The second step applies error-bounded lossy compression to the pruned layers and assesses the impacts of different error bounds on the inference accuracy for each layer. Based on the degradation of inference accuracy, DeepSZ identifies a feasible range of error bounds for each layer, collecting results on inference accuracy degradation and compressed layer size based on these bounds. This step effectively narrows the range of optimal error bounds for each layer. The third step determines the best-fit error bound for each layer based on the narrowed feasible range from the second step, compressing the network as much as possible while meeting the user-set inference accuracy requirement. The fourth step generates the compressed network based on the optimized error bounds and the best-fit lossless compressor.

## 6.3. Activation Data Compression

Training deep and wide neural networks has become increasingly challenging. While many state-of-the-art deep learning frameworks, such as TensorFlow [4] and PyTorch [120], can provide high training throughput by leveraging massive parallelism on general-purpose accelerators such as GPUs, one of the most common bottlenecks remains the high memory consumption during the training process. This issue is particularly acute considering the limited on-chip memory available on modern DNN accelerators. The primary cause is the ever-increasing size of the activation data computed during the training process.

Training a neural network involves numerous training epochs to update and learn the model weights. Each iteration includes forward and backward propagation. The intermediate activation data (the output from each neuron) generated by every layer is commonly kept in memory until backpropagation reaches that layer again. Several works [27, 53, 126, 30, 43] have highlighted the large gap between the time when activation data is generated in forward propagation and the time when it is reused in backpropagation, especially when training very deep models.

To address these challenges, researchers at JLESC propose a memory-efficient deep neural network training framework [82] named COMET (for Compression Optimized Memory-Efficient Training), which compresses activation data using adaptive error-bounded lossy compression. The key insights explored in this work include (i) the impact of compression errors in the activation data on the gradients and the entire CNN training process under strict error-controlling lossy compression can be theoretically and experimentally analyzed, and (ii) validation accuracy can be well maintained based on adaptive, fine-grained control over error-bounded lossy compression (i.e., compression error). This is the first work to investigate the impact of lossy compression error during CNN training and to leverage this analysis to significantly reduce memory consumption for training large CNNs while maintaining high validation accuracy.

An overview of COMET is shown in Figure 17. COMET iteratively repeats the process shown in the figure for each convolutional layer in every iteration. COMET mainly includes four phases, as shown in the figure from left to right: (1) parameter collection of current training status for adaptive compression, (2) gradient assessment to determine the maximum acceptable gradient error, (3) estimation of compression configuration (e.g., absolute error bound), and (4) compression/decompression of activation data with a modified cuSZ, the CUDA implementation of SZ (detailed in §7.3). Additionally, this research includes an analysis of the error-bound control scheme for lossy compression of activation data.

## 6.4. Input Data Compression for Inference

Another significant bottleneck in deploying DNNs for real-world applications is managing the extensive data collected from the source. In scenarios such as pedestrian detection, the DNN is typically hosted on cloud-based services, with end devices responsible only for collecting and sending data to the host for inference. A key challenge in such setups is the frequently restricted bandwidth of these end devices. Researchers at JLESC
## 6.2. DNN Model Compression

As pointed out by Wang et al. [153], the deep learning community acknowledges that increasing the scales of DNNs can improve the inference accuracy of image recognition tasks. For example, a 9-layer AlexNet, proposed by Krizhevsky et al. [88], won the 2012 ImageNet Large-Scale Visual Recognition Challenge (ILSVRC) [91] with a top-5 accuracy of 83%. In the 2014 ILSVRC, a 22-layer GoogLeNet [136] proposed by Szegedy et al. further improved the top-5 accuracy record to 93.3%. In the 2015 ILSVRC, He et al. [68] proposed a 152-layer ResNet, which set a new record of 96.43%. More recently, models such as GPT-3 [49] have exceeded 100 billion parameters, offering groundbreaking capabilities beyond those of conventional neural networks. This trend suggests that networks will continue to grow larger. Moreover, we note that during training, a model's gradient typically shares a size equivalence with the model itself. Consequently, the substantial model size presents communication challenges when exchanging gradients during training and difficulties in deploying these models on platforms with limited resources. Researchers at JLESC have proposed advanced lossy compression techniques for models aimed primarily at facilitating deployment on resource-limited platforms.

Specifically, Jin et al. [76] proposed DeepSZ, a lossy compression framework for DNNs. DeepSZ adapts the SZ lossy compression framework [36, 139, 98] to the context of DNN compression. This work enhances the efficiency of deploying DNN models on end devices with constrained bandwidth. Additionally, it also enables the storage of a larger number of models on each device, thereby significantly broadening their utility and application scope. In this adaptation, SZ achieves a much higher compression ratio for the compression of nonzero weights compared with other state-of-the-art compressors such as ZFP [103], especially due to its efficient linear-scaling quantization, in contrast to the simple vector quantization applied to the original weights in other related work [57, 65]. The general workflow of the DeepSZ framework is presented in Figure 16. As illustrated, DeepSZ consists of four key steps: network pruning, error-bound assessment, optimization of error-bound configuration, and generation of the compressed model. The first step involves adopting network pruning to reduce network complexity and mitigate the overfitting problem caused by a large number of parameters. The second step applies error-bounded lossy compression to the pruned layers and assesses the impacts of different error bounds on the inference accuracy for each layer. Based on the degradation of inference accuracy, DeepSZ identifies a feasible range of error bounds for each layer, collecting results on inference accuracy degradation and compressed layer size based on these bounds. This step effectively narrows the range of optimal error bounds for each layer. The third step determines the best-fit error bound for each layer based on the narrowed feasible range from the second step, compressing the network as much as possible while meeting the user-set inference accuracy requirement. The fourth step generates the compressed network based on the optimized error bounds and the best-fit lossless compressor.

## 6.3. Activation Data Compression

Training deep and wide neural networks has become increasingly challenging. While many state-of-the-art deep learning frameworks, such as TensorFlow [4] and PyTorch [120], can provide high training throughput by leveraging massive parallelism on general-purpose accelerators such as GPUs, one of the most common bottlenecks remains the high memory consumption during the training process. This issue is particularly acute considering the limited on-chip memory available on modern DNN accelerators. The primary cause is the ever-increasing size of the activation data computed during the training process.

Training a neural network involves numerous training epochs to update and learn the model weights. Each iteration includes forward and backward propagation. The intermediate activation data (the output from each neuron) generated by every layer is commonly kept in memory until backpropagation reaches that layer again. Several works [27, 53, 126, 30, 43] have highlighted the large gap between the time when activation data is generated in forward propagation and the time when it is reused in backpropagation, especially when training very deep models.

To address these challenges, researchers at JLESC propose a memory-efficient deep neural network training framework [82] named COMET (for Compression Optimized Memory-Efficient Training), which compresses activation data using adaptive error-bounded lossy compression. The key insights explored in this work include (i) the impact of compression errors in the activation data on the gradients and the entire CNN training process under strict error-controlling lossy compression can be theoretically and experimentally analyzed, and (ii) validation accuracy can be well maintained based on adaptive, fine-grained control over error-bounded lossy compression (i.e., compression error). This is the first work to investigate the impact of lossy compression error during CNN training and to leverage this analysis to significantly reduce memory consumption for training large CNNs while maintaining high validation accuracy.

An overview of COMET is shown in Figure 17. COMET iteratively repeats the process shown in the figure for each convolutional layer in every iteration. COMET mainly includes four phases, as shown in the figure from left to right: (1) parameter collection of current training status for adaptive compression, (2) gradient assessment to determine the maximum acceptable gradient error, (3) estimation of compression configuration (e.g., absolute error bound), and (4) compression/decompression of activation data with a modified cuSZ, the CUDA implementation of SZ (detailed in §7.3). Additionally, this research includes an analysis of the error-bound control scheme for lossy compression of activation data.

## 6.4. Input Data Compression for Inference

Another significant bottleneck in deploying DNNs for real-world applications is managing the extensive data collected from the source. In scenarios such as pedestrian detection, the DNN is typically hosted on cloud-based services, with end devices responsible only for collecting and sending data to the host for inference. A key challenge in such setups is the frequently restricted bandwidth of these end devices. Researchers at JLESC
<figure><img src="image_18.png" /><figcaption>Figure 16: Overview of the DeepSZ framework for neural network compression.</figcaption></figure>

<figure><img src="image_19.png" /><figcaption>Figure 17: Overview of our proposed memory-efficient CNN training framework COMET.</figcaption></figure>

have introduced frameworks designed to dynamically adjust the lossy compression configuration for the data [124]. This adaptive approach aims to maintain high detection accuracy across a diverse range of environmental conditions.

Rahman et al. [124] proposed a dynamic error-bounded lossy compression strategy to mitigate the bandwidth requirements in real-time vision-based pedestrian safety applications. This study introduced a dynamic feedback control system that adjusts the compression level dynamically to maintain the same detection accuracy as a system communicating raw lossless video data. The approach involves using a video compression unit installed on a roadside video monitoring camera. This unit compresses the raw video stream using a predetermined tolerance level, based on the peak signal-to-noise ratio that is dynamically adjusted in response to varying conditions, such as rain and darkness.

Following compression, the wirelessly transmitted compressed video streams are processed by the roadside edge computing infrastructure, which includes three main components: (1) a collection of calibrated pedestrian detection models tailored for diverse environmental conditions, (2) an active pedestrian detection model designed to process video images, and (3) an environmental condition detection model tasked with identifying the prevailing environmental conditions depicted in the video.

## 6.5. Performance

In memory compression experiments, results based on the tested neural networks show that DeepSZ [76] can achieve compression ratios of up to 116×, outperforming the second-best approach by up to 1.43×. Our experiments with four Nvidia

<figure><img src="image_20.png" /><figcaption>Figure 18: Normalized encoding time with different solutions on GPUs.</figcaption></figure>

<figure><img src="image_21.png" /><figcaption>Figure 19: Breakdown of decoding time with different solutions on CPU. Each sub-figure shows the comparison with a given model, including LeNet-5, AlexNet, and VGG-16.</figcaption></figure>
<figure><img src="image_18.png" /><figcaption>Figure 16: Overview of the DeepSZ framework for neural network compression.</figcaption></figure>

<figure><img src="image_19.png" /><figcaption>Figure 17: Overview of our proposed memory-efficient CNN training framework COMET.</figcaption></figure>

have introduced frameworks designed to dynamically adjust the lossy compression configuration for the data [124]. This adaptive approach aims to maintain high detection accuracy across a diverse range of environmental conditions.

Rahman et al. [124] proposed a dynamic error-bounded lossy compression strategy to mitigate the bandwidth requirements in real-time vision-based pedestrian safety applications. This study introduced a dynamic feedback control system that adjusts the compression level dynamically to maintain the same detection accuracy as a system communicating raw lossless video data. The approach involves using a video compression unit installed on a roadside video monitoring camera. This unit compresses the raw video stream using a predetermined tolerance level, based on the peak signal-to-noise ratio that is dynamically adjusted in response to varying conditions, such as rain and darkness.

Following compression, the wirelessly transmitted compressed video streams are processed by the roadside edge computing infrastructure, which includes three main components: (1) a collection of calibrated pedestrian detection models tailored for diverse environmental conditions, (2) an active pedestrian detection model designed to process video images, and (3) an environmental condition detection model tasked with identifying the prevailing environmental conditions depicted in the video.

## 6.5. Performance

In memory compression experiments, results based on the tested neural networks show that DeepSZ [76] can achieve compression ratios of up to 116×, outperforming the second-best approach by up to 1.43×. Our experiments with four Nvidia

<figure><img src="image_20.png" /><figcaption>Figure 18: Normalized encoding time with different solutions on GPUs.</figcaption></figure>

<figure><img src="image_21.png" /><figcaption>Figure 19: Breakdown of decoding time with different solutions on CPU. Each sub-figure shows the comparison with a given model, including LeNet-5, AlexNet, and VGG-16.</figcaption></figure>
<table><caption>Table 4: Inference accuracy of DeepSZ compressed LeNet-5, AlexNet, and VGG-16</caption><thead><tr><th>Neural Network</th><th>Top-1 Accuracy</th><th>Top-5 Accuracy</th><th>layer* Size</th><th>Compress Ratio</th></tr></thead><tbody><tr><td>LeNet-300-100 original</td><td>98.35%</td><td>-</td><td>1056 KB</td><td rowspan="2">55.8x</td></tr><tr><td>LeNet-300-100 DeepSZ</td><td>98.31%</td><td>-</td><td><b>19.1 KB</b></td></tr><tr><td>LeNet-5 original</td><td>99.13%</td><td>-</td><td>1620 KB</td><td rowspan="2">57.3x</td></tr><tr><td>LeNet-5 DeepSZ</td><td>99.16%</td><td>-</td><td><b>28.3 KB</b></td></tr><tr><td>AlexNet original</td><td>57.41%</td><td>80.40%</td><td>234.5 MB</td><td rowspan="2">45.5x</td></tr><tr><td>AlexNet DeepSZ</td><td>57.28%</td><td>80.58%</td><td><b>5.15 MB</b></td></tr><tr><td>VGG-16 original</td><td>68.05%</td><td>88.34%</td><td>494.5 MB</td><td rowspan="2">115.6x</td></tr><tr><td>VGG-16 DeepSZ</td><td>67.80%</td><td>88.20%</td><td><b>4.277 MB</b></td></tr></tbody></table>

<figure><img src="image_22.png" /><figcaption>Figure 20: Comparison of validation accuracy between baseline training and COMET (batch size = 256). Lines represent the validation accuracy during training; dots represent the compression ratio of COMET on the secondary y-axis.</figcaption></figure>

Tesla V100 GPUs demonstrate that DeepSZ can obtain a performance improvement in the encoding of 1.8x to 4.0x compared with the previous state of the art. Moreover, DeepSZ can improve the decoding performance by 4.5x to 6.2x compared with the second-best solution. DeepSZ also provides high flexibility to balance the compression ratio and inference accuracy.

Table 4 illustrates the compression ratio achieved by DeepSZ on multiple tested neural networks. Additionally, Figures 18 and 19 show that DeepSZ has lower encoding and decoding time overheads compared with Deep Compression [65] and Weight- less [125], the two works prior to DeepSZ.

In activation data compression, COMET [82] significantly reduces memory usage by up to 13.5x with little or no loss in

accuracy. Compared with the previous state-of-the-art compression-based approach, COMET offers an improvement in memory reduction of up to 1.8x. By leveraging the saved memory, COMET can enhance end-to-end training performance (e.g., approximately 2x improvement on AlexNet).

Figure 20 illustrates the accuracy curve and the compression ratio for AlexNet and ResNet-50. We observe that these two curves are very close to each other, indicating that COMET does not noticeably affect the validation accuracy. In the early stages of training, the compression ratio can be slightly unstable due to the relatively large changes in the model itself. We note that the compression ratio will change slightly when the learning rate changes since the learning rate is significant only when updating the gradient to the weights.

Table 5 shows the average compression ratio of COMET compared with a JPEG-based solution [43], which employs a hardware-implemented image-based compressor to achieve up to 7x compression ratios. COMET outperforms this solution by 1.5x and 1.8x on ResNet-18 and ResNet-50, respectively.

In the context of inference data compression, Table 6 illustrates the bandwidth reduction rates achieved by the framework introduced by Rahman et al. [124]. The data in the table demonstrates that, across various conditions, this framework consistently delivers high compression ratios for videos, effectively mitigating communication bottlenecks.

## 6.6. Future Work

Our upcoming research of compression for AI focuses mainly on three directions: (1) reduce the model/gradient size to enhance communication performance during training, (2) further reduce memory consumption during training through the compression of activation data, and (3) reduce the cost of loading training data and/or checkpoints to boost overall training performance. Specifically, in (1), our plans involve introducing multiple frameworks to enhance communication efficiency in various scenarios. First, we target the compression of communication data for deep learning recommendation model training, a process that typically employs a hybrid training infrastructure encompassing data parallelism, model parallelism, and pipeline- ing. We aim to propose an adaptive framework to compress both gradient and tensor data, thereby mitigating communication overhead. Second, we plan to establish a comprehensive compression workflow to enhance the performance of all-to-all communication, a widely employed operation in machine learning. Third, we intend to develop a framework tailored for reducing gradients in K-FAC training to significantly improve training performance. We plan on a compression scheme to control the distribution of compression errors in this task, thereby effectively limiting accuracy degradation for K-FAC training.

In terms of activation data compression, in (2) we noticed distinct advantages from two techniques: recomputing and data migration, apart from lossy compression for reducing memory consumption. Our goal is to create a hybrid framework that combines all three techniques to improve training performance. Additionally, we plan to explore lossy compression for activation data of a wide range of layers to identify which types of

<table><caption>Table 5: Comparison of validation accuracy between baseline training and COMET; comparison of compression ratio between JPEG-ACT and COMET.</caption><thead><tr><th>Neural Nets</th><th>Top-1 Accuracy</th><th>Peak Mem.</th><th>Max Batch</th><th>Conv. Act. Size</th><th>COMET</th><th>JPEG-ACT</th></tr></thead><tbody><tr><td>b.</td><td>57.41%</td><td>2.17 GB</td><td>512</td><td>407 MB</td><td rowspan="2">13.5x</td><td rowspan="2">-</td></tr><tr><td>AlexNet c.</td><td>57.42%</td><td>0.85 GB</td><td>2048</td><td><b>30 MB</b></td></tr><tr><td>b.</td><td>68.05%</td><td>17.29 GB</td><td>64</td><td>6.91 GB</td><td rowspan="2">11.1x</td><td rowspan="2">-</td></tr><tr><td>VGG-16 c.</td><td>68.02%</td><td>5.04 GB</td><td>256</td><td><b>0.62 GB</b></td></tr><tr><td>b.</td><td>67.57%</td><td>5.16 GB</td><td>256</td><td>1.71 GB</td><td rowspan="2">10.7x</td><td rowspan="2">7.3x</td></tr><tr><td>ResNet-18 c.</td><td>67.43%</td><td>1.37 GB</td><td>1024</td><td><b>0.16 GB</b></td></tr><tr><td>b.</td><td>75.55%</td><td>15.57 GB</td><td>128</td><td>5.14 GB</td><td rowspan="2">11.0x</td><td rowspan="2">6.0x</td></tr><tr><td>ResNet-50 c.</td><td>75.51%</td><td>4.40 GB</td><td>512</td><td><b>0.46 GB</b></td></tr></tbody></table>

b.= baseline, c.= compressed
<table><caption>Table 4: Inference accuracy of DeepSZ compressed LeNet-5, AlexNet, and VGG-16</caption><thead><tr><th>Neural Network</th><th>Top-1 Accuracy</th><th>Top-5 Accuracy</th><th>layer' Size</th><th>Compress Ratio</th></tr></thead><tbody><tr><td>LeNet-300-100 original</td><td>98.35%</td><td>-</td><td>1056 KB</td><td rowspan="2">55.8x</td></tr><tr><td>LeNet-300-100 DeepSZ</td><td>98.31%</td><td>-</td><td><strong>19.1 KB</strong></td></tr><tr><td>LeNet-5 original</td><td>99.13%</td><td>-</td><td>1620 KB</td><td rowspan="2">57.3x</td></tr><tr><td>LeNet-5 DeepSZ</td><td>99.16%</td><td>-</td><td><strong>28.3 KB</strong></td></tr><tr><td>AlexNet original</td><td>57.41%</td><td>80.40%</td><td>234.5 MB</td><td rowspan="2">45.5x</td></tr><tr><td>AlexNet DeepSZ</td><td>57.28%</td><td>80.58%</td><td><strong>5.15 MB</strong></td></tr><tr><td>VGG-16 original</td><td>68.05%</td><td>88.34%</td><td>494.5 MB</td><td rowspan="2">115.6x</td></tr><tr><td>VGG-16 DeepSZ</td><td>67.80%</td><td>88.20%</td><td><strong>4.277 MB</strong></td></tr></tbody></table>

<figure><img src="image_22.png" /><figcaption>Figure 20: Comparison of validation accuracy between baseline training and COMET (batch size = 256). Lines represent the validation accuracy during training; dots represent the compression ratio of COMET on the secondary y-axis.</figcaption></figure>

Tesla V100 GPUs demonstrate that DeepSZ can obtain a performance improvement in the encoding of 1.8x to 4.0x compared with the previous state of the art. Moreover, DeepSZ can improve the decoding performance by 4.5x to 6.2x compared with the second-best solution. DeepSZ also provides high flexibility to balance the compression ratio and inference accuracy.

Table 4 illustrates the compression ratio achieved by DeepSZ on multiple tested neural networks. Additionally, Figures 18 and 19 show that DeepSZ has lower encoding and decoding time overheads compared with Deep Compression [65] and Weight- less [125], the two works prior to DeepSZ.

In activation data compression, COMET [82] significantly reduces memory usage by up to 13.5x with little or no loss in

<table><caption>Table 5: Comparison of validation accuracy between baseline training and COMET; comparison of compression ratio between JPEG-ACT and COMET.</caption><thead><tr><th>Neural Nets</th><th>Top-1 Accuracy</th><th>Peak Mem.</th><th>Max Batch</th><th>Conv. Act. Size</th><th>COMET</th><th>JPEG-ACT</th></tr></thead><tbody><tr><td>b.</td><td>57.41%</td><td>2.17 GB</td><td>512</td><td>407 MB</td><td rowspan="2">13.5x</td><td rowspan="2">-</td></tr><tr><td>AlexNet c.</td><td>57.42%</td><td>0.85 GB</td><td>2048</td><td><strong>30 MB</strong></td></tr><tr><td>b.</td><td>68.05%</td><td>17.29 GB</td><td>64</td><td>6.91 GB</td><td rowspan="2">11.1x</td><td rowspan="2">-</td></tr><tr><td>VGG-16 c.</td><td>68.02%</td><td>5.04 GB</td><td>256</td><td><strong>0.62 GB</strong></td></tr><tr><td>b.</td><td>67.57%</td><td>5.16 GB</td><td>256</td><td>1.71 GB</td><td rowspan="2">10.7x</td><td rowspan="2">7.3x</td></tr><tr><td>ResNet-18 c.</td><td>67.43%</td><td>1.37 GB</td><td>1024</td><td><strong>0.16 GB</strong></td></tr><tr><td>b.</td><td>75.55%</td><td>15.57 GB</td><td>128</td><td>5.14 GB</td><td rowspan="2">11.0x</td><td rowspan="2">6.0x</td></tr><tr><td>ResNet-50 c.</td><td>75.51%</td><td>4.40 GB</td><td>512</td><td><strong>0.46 GB</strong></td></tr></tbody></table>

b.= baseline, c.= compressed

accuracy. Compared with the previous state-of-the-art compression-based approach, COMET offers an improvement in memory reduction of up to 1.8x. By leveraging the saved memory, COMET can enhance end-to-end training performance (e.g., approximately 2x improvement on AlexNet).

Figure 20 illustrates the accuracy curve and the compression ratio for AlexNet and ResNet-50. We observe that these two curves are very close to each other, indicating that COMET does not noticeably affect the validation accuracy. In the early stages of training, the compression ratio can be slightly unstable due to the relatively large changes in the model itself. We note that the compression ratio will change slightly when the learning rate changes since the learning rate is significant only when updating the gradient to the weights.

Table 5 shows the average compression ratio of COMET compared with a JPEG-based solution [43], which employs a hardware-implemented image-based compressor to achieve up to 7x compression ratios. COMET outperforms this solution by 1.5x and 1.8x on ResNet-18 and ResNet-50, respectively.

In the context of inference data compression, Table 6 illustrates the bandwidth reduction rates achieved by the framework introduced by Rahman et al. [124]. The data in the table demonstrates that, across various conditions, this framework consistently delivers high compression ratios for videos, effectively mitigating communication bottlenecks.

## 6.6. Future Work

Our upcoming research of compression for AI focuses mainly on three directions: (1) reduce the model/gradient size to enhance communication performance during training, (2) further reduce memory consumption during training through the compression of activation data, and (3) reduce the cost of loading training data and/or checkpoints to boost overall training performance. Specifically, in (1), our plans involve introducing multiple frameworks to enhance communication efficiency in various scenarios. First, we target the compression of communication data for deep learning recommendation model training, a process that typically employs a hybrid training infrastructure encompassing data parallelism, model parallelism, and pipelineing. We aim to propose an adaptive framework to compress both gradient and tensor data, thereby mitigating communication overhead. Second, we plan to establish a comprehensive compression workflow to enhance the performance of all-to-all communication, a widely employed operation in machine learning. Third, we intend to develop a framework tailored for reducing gradients in K-FAC training to significantly improve training performance. We plan on a compression scheme to control the distribution of compression errors in this task, thereby effectively limiting accuracy degradation for K-FAC training.

In terms of activation data compression, in (2) we noticed distinct advantages from two techniques: recomputing and data migration, apart from lossy compression for reducing memory consumption. Our goal is to create a hybrid framework that combines all three techniques to improve training performance. Additionally, we plan to explore lossy compression for activation data of a wide range of layers to identify which types of
<table><caption>Table 6: Maximum compression ratio achieved for different weather conditions.</caption><thead><tr><th>Weather condition</th><th>Baseline<br>Pedestrian Detection Accuracy<br>(no compression)</th><th>Dynamic EBLC Framework<br>Pedestrian Detection Accuracy</th><th>Improvement in Pedestrian Detection using Dynamic EBLC Framework</th><th>Maximum Constant Rate Factor (CRF) (or Minimum PSNR)</th><th>Required Bandwidth (MBits/sec)</th><th>Bandwidth Reduction</th></tr></thead><tbody><tr><td>Normal</td><td>98%</td><td>97%</td><td>-1%</td><td>30 (41dB)</td><td>0.53</td><td>18x</td></tr><tr><td>Light Dark</td><td>93%</td><td>97%</td><td>4%</td><td>30 (41dB)</td><td>0.68</td><td>14x</td></tr><tr><td>Medium Dark</td><td>92%</td><td>97%</td><td>5%</td><td>20 (43dB)</td><td>1.01</td><td>9.5x</td></tr><tr><td>High Dark</td><td>90%</td><td>97%</td><td>7%</td><td>10 (56dB)</td><td>4.05</td><td>2.5x</td></tr><tr><td>Light Rain</td><td>89%</td><td>97%</td><td>8%</td><td>10 (56dB)</td><td>5.15</td><td>1.5x</td></tr><tr><td>Medium Rain</td><td>88%</td><td>97%</td><td>9%</td><td>0</td><td>9.82</td><td>0x</td></tr><tr><td>Heavy Rain</td><td>83%</td><td>97%</td><td>14%</td><td>0</td><td>9.82</td><td>0x</td></tr></tbody></table>

layers can be efficiently compressed without significant accuracy degradation.

In (3), we aim to implement compression techniques for both training data and checkpoint data. We observed that in numerous AI applications for scientific purposes, the input data often surpasses the scale of traditional AI image processing. As a result, a substantial portion of the model's training time is allocated to loading the extensive input data. Offering frameworks that efficiently reduce the size of input data without compromising convergence speed and accuracy can effectively reduce the expenses associated with transferring large input data.

# 7. Optimizing Compression on the GPU

## 7.1. Introduction

Recent advancements in scientific computing, notably marked by the transition to exascale (over 1e18 FLOPS) computing [1], impose data challenges in two ways: the sheer volume of data and the rapid rate of data generation. Data compression has increasingly been researched to meet these challenges. Notably, a lossy compressor designed for scientific data reduction should fulfill three primary design goals: high-fidelity preservation in accordance with specific domain science constraints, a high compression ratio, and robust processing capability. On the other hand, GPUs have demonstrated significantly higher data processing capabilities than CPUs have in modern heterogeneous computing systems, making them well suited to address data generation issues by conducting in situ reduction.

SZ is a state-of-the-art error-bounded lossy compression framework for scientific data, meeting the requirements of high-fidelity preservation and a high compression ratio. However, as shown in prior work [36, 139], SZ suffers from (1) low single-core compression and decompression speeds at hundreds of megabytes per second and (2) limited multicore performance. Therefore, adopting a data-parallel programming model and leveraging GPUs can be an attractive solution to keep pace with the high data production rate. Today, GPUs contribute significantly to compute capabilities thanks to a mature ecosystem; their high programmability, as opposed to other heterogeneous hardware platforms such as FPGAs [144], enables easier parallelization.

Parallelizing SZ requires addressing two main issues. First, a loop-carried data dependency exists among the loop iterations

of SZ's prediction and error quantization step—not until iteration *i* is finished can iteration (*i* + 1) start. This tight data dependency incurs extraordinarily costly synchronization to ensure correctness.<fnref n="5" /> Second, during Huffman coding, a variable-length encoding step in the SZ algorithm, the uneven data access pattern for grouped threads hurts performance by breaking the rigid SIMD parallel pattern (known as warp divergence).

The challenge centers on unleashing GPUs' potential by identifying parallelisms in the existing SZ algorithm and balancing the three primary design goals—achieving a high compression ratio, high throughput, and high data quality. Therefore, over the past five years, JLESC researchers have proposed a series of works to address the low-throughput issue and expand the usability of SZ compression. A summary of the proposed GPU-based compressors is given in Table 6.

<table><caption>Table 7: Comparison of cuSZ, FZ-GPU, cuSZp, and cuSZ-I.</caption><thead><tr><th>Work</th><th>Generality</th><th>Speed</th><th>Quality</th><th>Comp. Ratio</th></tr></thead><tbody><tr><td>cuSZ(+)</td><td>general-purpose</td><td>baseline</td><td>baseline</td><td>high</td></tr><tr><td>FZ-GPU</td><td>general-purpose</td><td>high</td><td>baseline</td><td>balanced medium</td></tr><tr><td>cuSZp</td><td>special (1D only)</td><td>high (end-to-end)</td><td>baseline</td><td>medium</td></tr><tr><td>cuSZ-I</td><td>special (3D only)</td><td>medium</td><td>highest</td><td>highest</td></tr></tbody></table>

## 7.2. The State of the Art

Given the surge in GPU-based HPC systems and their applications, there has been a proliferation of GPU-based compression methods for scientific data. Notable examples include cuZFP [32], MGARD [55], and nvCOMP's Bitcomp [115]. These GPU-based compressors offer significantly higher compression throughputs compared with CPU-based compressors.

cuZFP, employing a transformation-based approach, allows users to specify their desired bitrate, representing the average number of bits per value after compression. In contrast, Bitcomp (a quantization-based compressor) and MGARD (a multigrid-based compressor) enable users to set the maximum permissible error. Notably, cuZFP in fixed-rate mode consistently delivers

<footnote n="5">⁵Preliminary results show 30 to 40 GB/s for the major kernel on V100, which is approximately 1/20 of memory copy speed.</footnote>