Title: MeLLoC: Lossless Compression with High-order Mechanism Learning
Venue: NeurIPS 2024
Accepted: yes
Keywords: lossless compression

mechanism learning

inverse problem
Abstract: Lossless compression of large-scale scientific floating-point data is critical yet challenging due to the presence of high-order information and noise that arises from model truncation and discretization errors. Existing entropy coding techniques fail to effectively leverage the mechanisms underlying the data generation process. This paper introduces MeLLoC(Mechanism Learning for Lossless Compression), a novel approach that combines high-order mechanism learning with classical encoding to enhance lossless compression for scientific data. The key idea is to treat the data as discrete samples from an underlying physical field described by differential equations and solve an inverse problem to identify the governing equation coefficients exhibiting more compressible numeric representations. Periodic extension techniques are employed to accelerate the decompression. Through extensive experiments on various scientific datasets, MeLLoC consistently outperforms state-of-the-art lossless compressors while offering compelling trade-offs between compression ratios and computational costs. This work opens up new avenues for exploiting domain knowledge and high-order information to improve data compression in scientific computing.

MeLLoC: Lossless Compression with High-order Mechanism Learning

Abstract

Lossless compression of large-scale scientific floating-point data is critical yet challenging due to the presence of noise and high-order information that arises from model truncation and discretization errors. Existing entropy coding techniques fail to effectively leverage the mechanisms underlying the data generation process. This paper introduces MeLLoC(Mechanism Learning for Lossless Compression), a novel approach that combines high-order mechanism learning with classical encoding to enhance lossless compression for scientific data. The key idea is to treat the data as discrete samples from an underlying physical field described by differential equations, and solve an inverse problem to identify the governing equation coefficients exhibiting more compressible numeric representations. Periodic extension techniques are employed to accelerate the decompression. Through extensive experiments on various scientific datasets, MeLLoC consistently outperforms state-of-the-art lossless compressors while offering compelling trade-offs between compression ratios and computational costs. This work opens up new avenues for exploiting domain knowledge and high-order information to improve data compression in scientific computing.

1 Introduction

In the modern era, vast amounts of floating-point data are generated from large-scale simulations and experiments, often reaching petabyte levels. Efficient compression of data is essential for reducing storage costs and facilitating data transfer and analysis. Consequently, the compression of scientific data has gained increasing attention.

Lossless compression aims to reduce the size of data while allowing perfect reconstruction of the original data from its compressed representation. Classical lossless compression algorithms like Gzip and Bzip2 employ entropy encoding techniques like Huffman coding and arithmetic coding to exploit statistical redundancies in the data[1, 2]. While effective for general-purpose data, these methods typically underperform when applied to scientific floating-point datasets, as they fail to capture the distinct characteristics of such data. In contrast, specialized compression algorithms like FPZIP[3], ZFP[4], zstd[5] and ALP[6] are designed to handle floating-point data more effectively by employing predictive coding or customized entropy coders that account for the spatial correlations and value distributions present in scientific datasets.

A common challenge in scientific data compression is the presence of high-order information and randomness, or noise, which arises from roundoff errors during simulation, as well as model truncation and discretization errors. This noise, termed as "false precision" by Zender[7], can significantly lower the compression ratio. The reason is that lossless compression algorithms do not distinguish the informative mechanism from the non-informative noise in data, thus leaving the level of meaningful precision unassessed and failing to exploit the high-order information in the data. By leaving the

noisy components, the data is more likely to exhibit detectable patterns, indicating that their size can be effectively reduced by a compressor[8].

To address this issue, Klöwer et al. propose a method of truncation based on evaluating the bitwise dependence between adjacent grid points [9]. However, denoising by truncation on bit digits may not be an optimal approach in practice due to the non-uniform distribution and varying levels of noise in scientific data across different spatial and temporal contexts, which risks confounding the scientific insights gleaned from the data. A recent work by Luo et al. [10] considers locally characterizing the data's spatial relationship with linear differential equations, then separating the noise instead of using truncation. While this method offers a more nuanced approach to noise handling, it's important to note that both these techniques still constitute forms of lossy compression.

Scientific data typically conforms to well-defined model mechanisms, often characterized using differential equations[11]. According to the theory of well-posedness for differential equations, the solutions to the differential equations are determined by the initial and boundary conditions, as well as the source terms[12]. This suggests that the entire physical field could be represented by a smaller, compressed set of values that describe these mechanisms. However, most existing lossless compression techniques do not exploit this inherent structure and high-order information in scientific floating-point data.

In this paper, we aim to further enhance the effectiveness of lossless compression by learning the inherent mechanisms underlying scientific data, building on previous near-lossless work[10]. The key idea is to treat the data as samples from a discretized physical field and solve an inverse problem for the governing differential equations to obtain the source terms, which exhibit a more compressible numeric distribution.

Our main results can be summarized as follows:

We propose MeLLoC (Mechanism Learning for Lossless Compression), a novel approach that combines high-order mechanism learning with classical encoding algorithms to achieve efficient lossless compression of scientific floating-point data.

We implement periodic extension techniques to accelerate the solution of the large linear systems arising in the parameter identification problem.

Through comprehensive numerical experiments, we demonstrate that MeLLoC consistently outperforms the existing methods while offering compelling trade-offs between compression ratios and computational requirements.

2 Background

Partial Differential Equations behind Scientific Data. Partial Differential Equations (PDEs) are fundamental in modeling complex physical phenomena across various scientific disciplines. They provide a mathematical framework for describing spatiotemporal changes in physical quantities. In oceanography and climate science, PDEs are widely used to model phenomena such as heat transfer, fluid dynamics, and wave propagation. Specifically in oceanography, these equations model ocean circulation, temperature distribution, and salinity patterns, often involving multiple variables and intricate interactions. While the specific equations can vary depending on the phenomenon being studied, they generally capture the fundamental physical principles governing the system's behavior.

The complexity of these PDEs and their numerical solutions often results in large-scale floating-point datasets, which are the focus of our compression efforts. Understanding the underlying mathematical structure of these datasets is crucial for developing effective compression techniques that can preserve important physical features while reducing data volume.

Mechanism Learning of Scientific data Mechanism learning, or data-driven discovery of governing equations, has gained significant attention in scientific computing. This approach aims to uncover the underlying physical laws from observational or simulation data. Key techniques in this area include Sparse Identification of Nonlinear Dynamics(SINDy)[13] ,Physics-Informed Neural Networks(PINNs)[14] . These methods provide a foundation for identifying compact representations of complex systems, which can potentially be applied for data compression. However, SINDy requires temporal evolution data to learn the underlying dynamics, while PINNs typically need large datasets for training, making them less suitable for case-by-case data compression scenarios.

Entropy Encoding. Entropy encoding is a fundamental lossless data compression technique that assigns shorter codes to frequently occurring symbols and longer codes to rare ones, based on the statistical properties of the data. The encoding scheme is designed to minimize the overall bit length of the encoded message. Shannon's source coding theorem[15] provides the theoretical basis, suggesting that the optimal code length for a symbol x is -log D ( x ) , where the expected code length is bounded by the entropy of the source distribution D .

Finite State Entropy (FSE)[16] is a notable entropy encoding method that offers competitive compression ratios and fast encoding and decoding speeds. FSE constructs a finite state machine to model the probability of symbols, adapting to the changing statistics of the input data. This adaptability makes FSE particularly effective for compressing data with non-uniform or unknown distributions.

3 Methodology

3.1 Overview of the MeLLoC Framework

Building upon PDE theory and mechanism learning concepts discussed earlier, we develop our compression methodology specifically for scientific data. We demonstrate our approach using twodimensional (2D) scenarios, though the methodology can be extended to higher dimensions. Our method takes advantage of the inherent mathematical structures present in scientific data to achieve more effective compression.

Source Term

𝒇

Boundary

𝒖𝒃𝒅

Precision

Control

ത 𝑢

Encoding and Decoding

Solve

ℒ𝜃 ∗ 𝑢 = 𝑓

with

𝑢|𝜕𝐷

= 𝑢𝑏𝑑

Original Data

𝒖

Local mechanism

𝐾ℒ𝑢 𝑖𝑛

Coefficient

𝜽 ∗

𝑣

ത 𝑢

Reconstructed Data

ෝ 𝒖

We first exemplify the idea through the Laplace equation, a canonical PDE representing steady-state conditions across various fields such as temperature distribution and electric potential

When the data represents the numerical solution of the aforementioned equation, derived from a second-order central difference scheme applied to a grid with uniform spacing, it complies with the following equation

This implies that the value at any central point is calculable based on the values of its immediate neighboring points. Furthermore, in this scenario, the data is entirely determined by its boundary values, suggesting that only the boundary data need be stored to fully reconstruct the original dataset.

In practice, data may exhibit non-homogeneous conditions, described by ∆ u = f , where f denotes the source term. In such cases, both the boundary values and the source f need to be stored. Compression remains viable, provided the information entropy of f is subordinate to that of u , a condition often satisfied when data exhibits elliptic properties or when the source term is of low magnitude and sparse.

To address data governed by diverse mechanisms, our model is parameterized and calibrated through learning from the data. We assume that a set of two-dimensional data satisfies a difference equation L u i,j = f i,j , where L represents the difference operator. Globally, a linear system for the interior unknowns u in can be constructed as follows

Here, K L is the stiffness matrix corresponding to the difference operator, while b u bd ,f comprises boundary data and source term data and represents the 'data support' which refers to the data needed for the well-posedness of the equation.

Assume that the scientific data can be locally approximated by a homogeneous second-order linear PDE with constant coefficients. In this context, our goal is to learn the difference operator L by minimizing the source term | f | , thereby achieving maximal sparsity and reduced information entropy. Consequently, only the boundary data u bd and the sparse source term f are stored. During decompression, the data can be reconstructed using the matrix K L .The proposed method is not limited to specific difference formats and can effectively handle variable coefficient equations.

To facilitate the compression, the difference operator L is formulated in a compact 9-point form and parameterized by θ = { C i } 9 i =1 . The coefficient template for the combination of neighboring points is depicted in Figure 2(a), illustrating how local representation translates to global connectivity among data points, as shown in Figure 2(b). Figure 2(c) provides some typical templates for differential equations.

𝑪

𝟒

𝒖

𝒊#𝟏,𝒋'𝟏

𝑪

𝟖

𝒖

𝒊,𝒋'𝟏

𝑪

𝟗

𝒖

𝒊'𝟏,𝒋'𝟏

𝑪

𝟑

𝒖

𝒊#𝟏,𝒋

𝑪

𝟓

𝒖

𝒊,𝒋

𝑪

𝟕

𝒖

𝒊'𝟏,𝒋

𝑪

𝟏

𝒖

𝒊#𝟏,𝒋#𝟏

𝑪

𝟐

𝒖

𝒊,𝒋#𝟏

𝑪

𝟔

𝒖

𝒊'𝟏,𝒋#𝟏

-1/4

-1/4

1

-1/4

-1/4

1

-1

-1

1

1/2

-1/2

1

-1/2

-1/2

Elliptic

Hyperbolic

Parabolic

(a) Coefficient template.

(b) Intra predictions.

(c) Templates for some differential equations.

In the compression process, we need to determine the parameters θ = { C i } 9 i =1 such that

and minimize the magnitude of source term f i,j . Therefore, learning the operator (equivalently, the corresponding stiffness matrix) transforms into solving an optimization problem formulated as

where, by normalizing θ with C 5 = -1 , the loss function is defined by

In applications, to reduce computational cost or highlight local features, a strategic approach involves selecting only a localized subset of the data to learn the difference operator L θ . During decompression, we obtain u in by solving equation K L u in = b u bd ,f .

Our approach is particularly suitable for compressing numerical solutions of differential equations. In addition, it performs well for data with explicit physical mechanisms, such as those encountered in atmospheric and oceanic observational datasets or within the industrial sector. The learned ninepoint coefficient template L θ (the difference operator) corresponds to the mechanism described by a second-order differential equation and is generally applicable. Furthermore, the stability of both the compression and decompression processes is assured by the well-posedness of the difference equation.

3.2 Model identification and its well-posedness

Consider the underlying model of the data. For u ∈ C 4 (Ω) , Ω ⊂ R 2 template is related to the second order linear differential operator as

, the nine-point difference

Here C k,l are the coefficients in Figure 2(a) and the subscripts k, l represent the relative position to the data point ( x, y ) . The relationship between C k,l and c n can be expressed as

where C is the matrix of C k,l , A n are basis matrices, and c n are corresponding coefficients. The matrices { A n } 9 n =1 are defined as

Based on this representation, encoding u becomes encoding the sparser high-order term o ( h 2 ) , i.e., the source term f . Therefore, the optimization objective is to obtain a minimized high-order term, which can be mathematically expressed as

Once the template θ := C k,l is learned, one can calculate the coefficients of the differential operator, allowing to classify the mechanism as elliptic, parabolic, or hyperbolic due to the reversibility of C k,l and c n , which provides interpretability of the model.

Next, we discuss the well-posedness of the model identification problem (compression process), while that of the decompression process will be discussed in Section 3.5. The above minimization problem is equivalent to solving the least square problem

where c = [ C 1 , · · · , C 9 ] T , A ∈ R N × 9 , N is the number of data points in domain D , P : D → R , and k = P ( i, j ) , ( i, j ) ∈ D is the index after rearranging the data into 1-dimensional vector, with ( i, j ) = P -1 ( k ) , k = 1 , · · · , N , as its inverse mapping.

To find non-trivial solutions is to obtain the null space (kernel) of B = ( A T A ) . If B is full-rank, there is only trivial solution, while otherwise, there is no uniqueness. To address this issue, we set the coefficient of u i,j to -1 (i.e. set C 5 to -1 in Figure 2(a)), and fix the template size to 8. The problem then becomes

where ˜ c ∈ R 8 , ˜ A ∈ R N × 8 , and b = [ u P -1 (1) , · · · , u P -1 ( N ) ] T ∈ R N . The problem has a unique least squares solution ˜ c = ˜ A † b , provided the data are not all degenerated. We assemble ˜ A and directly solve the pseudo inverse, which serves as a fast solver for the compression process.

3.3 Data Composition

To illustrate the method's capacity for lossless compression, consider the decomposition of data u predicated upon linear superposition

This equation stratifies the dataset into three fundamental components:

L -1 f , representing G ∗ f , with G being the Green's function for the domain. This part corresponds to solution to the non-homogeneous equation with homogeneous boundary conditions, determined by the source f (2D).

u 0 (= ∂G ∂ν ∗ u bd ) denotes the solution to the homogeneous equation is determined only by the boundary data (1D).

u err denotes the residual part (2D).

The precision control discussed in the following enables us to optimize the source term and retaining sufficient significant figures based on data precision to nullify u err , thereby yielding u = L -1 f + u 0 . This decomposition allows us to store only the boundary data u bd and the sparse source term f , achieving compression while preserving the essential information content of the original dataset.

3.4 Precision Control

Suppose the data u possesses a precision of m bits, expressed in decimal form, with a binary equivalent exhibiting similar properties. This precision implies an accuracy to the order of 10 -m , as depicted in Figure 3(a). By analogy, the precision of the model coefficients C i is quantified as n bits, corresponding to 10 -n . Consequently, when the source term f is computed via f = L ( u ) = ∑ 9 i =1 C i u i , the precision of f is up to 10 -m × 10 -n = 10 -( m + n ) .

The data can be losslessly recovered or decompressed once the error from solving the linear system K L u in = b u bd ,f is less than 10 -( m + n ) , which is feasible with proper n . As n increases, the admissible set for C i enlarges, contributing to a lower absolute value for f but higher precision. The optimal compression ratio is reached when significant digits of f are minimized. We optimize n for coefficients C i by starting with high precision and gradually reducing it while monitoring reconstruction error and compression ratio. After several calibrations, n can be fixed for the remaining dataset if the compression ratio for subsequent batches shows no significant fluctuation. 0 . 0 … 0 0 0 𝒃 𝟏 𝒃 𝟐 … … 𝒃 𝒌 𝟏 Case I: High precision digits. 0 . 0 … 0 0 𝒃 𝟏 … … 𝒃 𝒌 𝟑 Case II: Large absolute value. Case III: Optimal scenario. 0 . 0 … 0 𝒃 𝟏 𝒃 𝟏 … 𝒃 𝒌 𝟐 0 . 0 … 0 0 0 𝒃𝟏 𝒃𝟐 … … … 𝒃𝒌𝟏

! 𝒂

𝒑

…

! 𝒂

𝟏

.

𝒂

𝟏

𝒂

𝟐

…

…

𝒂

𝒎%𝟏

𝒂

𝒎

𝒍

digits with true precision

𝒎

decimal places

Source

𝒇

Data

𝒖

0

.

0

…

0

𝒃

𝟏

𝒃

𝟐

…

𝒃

𝒌%𝟏

𝒃

𝒌

𝒎

+

𝒏

decimal places

𝒌

digits with true precision

0

.

0

…

0

0

0

𝒃𝟏

𝒃𝟐

…

…

…

𝒃𝒌𝟏

Case I: High precision digits.

0

.

0

…

0

0

𝒃𝟏

…

…

𝒃𝒌𝟑

Case II: Large absolute value.

Case III: Optimal scenario.

0

.

0

…

𝒃𝟏

𝒃𝟏

…

…

𝒃𝒌𝟐

(a) Precision digits and decimal places.

𝒖

(b) Trade-off between values and digits.

Data

෥ 𝒂𝒑 … ෥ 𝒂𝟏 . 𝒂𝟏 𝒂𝟐 … … 0 . 0 … 0 𝒃𝟏 𝒃𝟐 … This approach considers different scenarios to balance significant digits and value magnitudes, as illustrated in Figure 3(b). The trade-off between fewer effective digits of f compared to u contributes to compression, while increasing precision in coefficient truncation expands the solution space, potentially decreasing the absolute value of the optimized source term but increasing storage requirements. Our method, MeLLoC, optimizes compression efficiency, balancing perfect reconstruction with computational feasibility. The precision control is adaptive and can be tailored to different scientific datasets, ensuring that the compression process is optimized for various types of atmospheric and climate model data.

𝒇

Source

0

.

0

…

𝒃𝟏

𝒃𝟐

…

𝒂𝒎-𝟏

𝒃𝒌-𝟏

…

𝒃𝒌𝟐

𝒂𝒎

𝒃𝒌

𝒍

𝒎

𝒌

𝒎

3.5 Fast Fourier-based Solver

The traditional approach to solving systems of equations (1) can be computationally intensive, particularly when dealing with large datasets. To address this issue, we propose a Fourier-based solver that accelerates computation while preserving data integrity.

We first extend the discrete field data periodically and expand it using a 2D Fourier series

where i = √ -1 represents the imaginary unit, and ˆ u j,k represents the Fourier coefficients. It is essential that the double-layer boundary condition enables periodic extension for the source field f . By substituting the Fourier series into the difference equation, we obtain a system relating these coefficients

̸

𝑖 = 0

1

𝑁

𝑁 + 1

Here, ˆ f j,k represents the Fourier coefficients of f . B j,k is a coefficient involving the difference operator and the Fourier basis functions. The specific form of B j,k depends on the discretization scheme used in the original differential equation[17]. In general cases, the double-layer boundary values are necessary to determine the base frequency Fourier coefficients, ensuring the well-posedness. For example, when n = 1 ,

By applying DFT to u ( m, 1) , we obtain ˜ u j satisfying u ( m, 1) = 1 N 1 N 1 ∑ j =1 ˜ u j e 2 π N 1 ( j -1)( m -1) . Utilizing

This approach allows for the efficient computation of ˆ u 1 k through similar reasoning. By employing FFT and its inverse (iFFT), the algorithm accelerates the matrix approximation process during both compression and decompression phases, with a computational complexity of O ( N 2 log N ) . This Fourier-based approach significantly enhances computational efficiency compared to traditional matrix computation methods, thereby ensuring the throughput of compression and decompression operations while maintaining lossless reconstruction of the original data.

3.6 High-Order Mechanisms

While the proposed 9-point difference operator L is primarily designed to capture second-order PDE dynamics, many real-world datasets may exhibit higher-order effects arising from more complex governing equations or fine-scale features. These higher-order mechanisms can manifest in the source term f , potentially limiting the compression performance if not properly addressed.

To better characterize these higher-order properties, a preprocessing step involving the introduction of diffusive operators can be employed. Let D denote a diffusive operator, such as D = α ∆ 2 + β ∆ , where ∆ is the Laplacian and α, β are constants. We can apply this operator to the original data u to obtain a smoothed version ˜ u = D u , which effectively filters out the fine-scale, high-order features.

The compression is then performed on ˜ u instead of u , yielding a sparse source term ˜ f that encapsulates the higher-order effects L ˜ u = ˜ f . During decompression, the inverse diffusive operator D -1 is applied to recover the original data

This preprocessing strategy allows capturing high-order mechanisms by absorbing them into the sparse source term ˜ f , while the 9-point operator L focuses on representing the underlying secondorder dynamics in the smoothed data ˜ u . The diffusive operator approach maintains computational efficiency, lossless compression/decompression, and enhances handling of complex, high-order data. The diffusive operator D can be tailored to the dataset, potentially with learnable parameters for optimized filtering. Iterative applications of D enable progressive extraction of high-order features at multiple levels.

4 Algorithm

The algorithm uses an optimized 9-point template to achieve lossless compression and decompression of data, which is mainly divided into two parts: compression and decompression.

The compression part includes selecting the initial template, setting the coefficient truncation precision, and optimizing the template coefficients to minimize the residual error and information entropy of the compressed data. The compression process also includes truncating the source term to the required precision and encoding it using an appropriate encoding scheme. The compression part terminates when the compression rate is satisfactory or when the maximum number of iterations is reached.

Require: Template coefficients C (preprocessing template C 0 , if any), source term f , boundary values u bd (preprocessing boundary u bd , if any), and data precision m .

Algorithm 2 Decompression based on optimized source term 0 Ensure: Data array u . 1: if high-order mode is required then 2: Assemble the stiffness matrix K 0 ( C 0 ) . 3: Assemble the right-hand side d ( u bd 0 , f ) . 4: Solve the equation K 0 f 1 = d . 5: else 6: f 1 ← f . 7: end if 8: Assemble the stiffness matrix K ( C ) . 9: Assemble the right-hand side d ( u bd , f 1 ) . 10: Solve the equation Ku = d . 11: Reconstruct u with the precision of original data.

The decompression part includes assembling the stiffness matrix and right-hand vector using the optimized template coefficients, solving the linear system to obtain the compressed data, and truncating it to the original precision. The decompression process also includes decoding the compressed source term using the same encoding scheme and using it to reconstruct the original data.

5 Experimental Results

This section presents the experimental results of applying our proposed lossless compression algorithm to the CESM-ATM and Hurricane datasets from the SDRBench[18]. The performance is evaluated based on the original data, transformed data (source terms post-transformation), reconstruction error, frequency distribution plots, compression ratio, throughput, and comparison with existing algorithms such as Zstandard (zstd)[5] and fpzip[3]. All tests were conducted on a Mac with M1 Silicon, macOS 14.1.2, 16GB RAM.

5.1 Compression and Reconstruction

The original datasets were first transformed using our proposed method to generate source terms suitable for encoding. The transformed data were then compressed, and the reconstruction error was calculated by comparing the original data to the data reconstructed from the compressed representation.

(a)CESM-ATM.

(b)Hurricane.

400

OriginalData

SourceTerm

500

OriginalData

SourceTerm

300

300

300

300

0.05

400

250

400

0.5

280

300

200

300

200

260

200

0

200

150

200

0

100

240

100

-0.05

100

50

100

100

-0.5

-0.1

0

100

200

300

400

100

200

300

100200300400500

100

2003004

400

Figure 5 illustrates the results obtained by applying the proposed scheme to the CESM-ATM and Hurricane datasets. For the CESM-ATM data, while the original data values span a range of 220 to 300, the source term exhibits a more compressed range of -0.1 to 0.1, indicating effective compression. Notably, the reconstruction error was found to be around 10 -11 , which is smaller than the least significant digit of single-precision floating-point representation. For the Hurricane dataset, the reconstruction error is even smaller, on the order of 10 -12 . These results demonstrate that the proposed method preserves the numerical precision essential for scientific computations. The extremely low reconstruction errors ensure that the compressed data can be used reliably in high-precision applications, maintaining the integrity of the original datasets for subsequent analyses and simulations.

5.2 Frequency Distribution Analysis

To further illustrate the effectiveness of our compression method, we compared the frequency distribution plots of the data before and after compression. The left represents the original data, while the right represents the stored source terms. The histograms employ logarithmic binning for clarity, demonstrating the reduced mean and standard deviation of the source term, which contributes to decreased entropy and, consequently, a better compression rate.

240

260

280

300

Values

0

5

10

15

log

2

(Count)

0

Values

10

15

log

2

(Count)

5

0

-0.1

0.1

0

50

100

150

200

250

300

Values

0

5

10

15

log

2

(Count)

Values

10

15

log

2

(Count)

5

0

0

0.25

-0.25

-0.5

0.5

(a)CESM-ATM.

(b)Hurricane.

5.3 Performance Evaluation

We evaluate MeLLoC against several state-of-the-art compression algorithms using two key metrics: compression ratio and throughput. The compression ratio indicates the efficiency of data reduction, with higher values representing more compact representations. Throughput measures the speed of compression and decompression processes, which is crucial for handling large scientific datasets. Table 1 presents a comprehensive comparison across CESM-ATM and Hurricane datasets.

ALP and FPZIP are lossless floating-point compression algorithms, while ZFP is set to lossless mode with zero error tolerance. Blosc, Gzip, and Zstandard are general-purpose lossless compression algorithms. MeLLoC, the proposed method, consistently achieves the highest compression ratios (3.36× for CESM-ATM and 3.29× for Hurricane) while maintaining competitive throughput.

5.4 Scalability Performance Analysis

3.5

CompressionRatioandSpeedvsFileSize

250

3.4

200

3.3

CompressionRatio

3.2

150

Speed (MB/s)

3.1

3.0

100

2.9

50

2.8

2.7

0

20

40

60

80

100

120°

0

File Size (MB)

CompressionRatio

CompressionSpeed

DecompressionSpeed

6 Conclusion

In this paper, we present a novel mechanism-learning method for compressing scientific data. Our proposed scheme effectively preserves the true precision of the data while enabling subsequent lossless compression. Additionally, the approach offers flexibility by allowing the use of various lossless encoding algorithms, which can be selected based on factors such as availability, speed, and resulting file sizes, tailored to specific application requirements. Experiments demonstrate that the proposed method has competitive reconstruction performance with the general-purpose methods, while providing a novel perspective on scientific data compression. MeLLoC has several limitations. As discussed in Section 3.1, its performance is contingent upon the accuracy of the differential equation models representing the data. The method is particularly suitable for scientific data whose order of magnitude aligns well with the assumed physical models. For datasets that do not fit these criteria, the method may not be as effective. The precision control, crucial for optimizing compression, also necessitates careful calibration, which might be challenging with diverse data characteristics. Despite these constraints, MeLLoC presents a valuable approach for scientific data compression. In future work, we intend to apply the 'mechanism-learning' concept to several pertinent real-world domains, such as medical imaging, oceanography, and other related fields.

In Figure 7,the compression ratio, represented by the orange line, shows a slight decline as file size increases, dropping from around 3.4 to just above 3.1 for the largest files. Compression and decompression speeds, depicted by the blue and green dashed lines respectively, remain relatively stable across varying file sizes, with minimal fluctuations. Compression speeds consistently outperform decompression speeds, both hovering within a range of 150 to 200 MB/s across all file sizes. This indicates a stable and efficient scalability pattern in the data compression process.

Acknowledgments and Disclosure of Funding

This work is supported by National Key Research and Development Programs of China (No. 2023YFA1009103), National Natural Science Foundation of China (Nos. 12201386, 12241103), Science and Technology Commission of Shanghai Municipality (23JC1400501) and the Sino-German Mobility Programme (M-0187) by Sino-German Center for Research Promotion.

References

Khalid Sayood. Introduction to data compression . Morgan Kaufmann, 2017.

David JC MacKay. Information theory, inference and learning algorithms . Cambridge university press, 2003.

Peter Lindstrom and Martin Isenburg. Fast and efficient compression of floating-point data. IEEE transactions on visualization and computer graphics , 12(5):1245-1250, 2006.

Peter Lindstrom. Fixed-rate compressed floating-point arrays. IEEE transactions on visualization and computer graphics , 20(12):2674-2683, 2014.

Yann Collet and Murray Kucherawy. Zstandard compression and the application/zstd media type. Technical report, 2018.

Azim Afroozeh, Leonardo X Kuffo, and Peter Boncz. Alp: Adaptive lossless floating-point compression. Proceedings of the ACM on Management of Data , 1(4):1-26, 2023.

Charles S Zender. Bit grooming: statistically accurate precision-preserving quantization with compression, evaluated in the netcdf operators (nco, v4. 4.8+). Geoscientific Model Development , 9(9):3199-3211, 2016.

Peter Lindstrom. Error distributions of lossy floating-point compressors. Technical report, Lawrence Livermore National Lab.(LLNL), Livermore, CA (United States), 2017.

Milan Klöwer, Miha Razinger, Juan J Dominguez, Peter D Düben, and Tim N Palmer. Compressing atmospheric data into its real information content. Nature Computational Science , 1 (11):713-724, 2021.

Xinyue Luo, Jin Cheng, Zhaoyi Sun, and Yu Chen. Precision-preserving compression of scientific data: Learn mechanism from data. In 2024 Data Compression Conference (DCC) , page 572. IEEE, 2024.

Jianhong Shen and Tony F Chan. Mathematical models for local nontexture inpaintings. SIAM Journal on Applied Mathematics , 62(3):1019-1043, 2002.

Lawrence C Evans. Partial differential equations , volume 19. American Mathematical Society, 2022.

Steven L Brunton, Joshua L Proctor, and J Nathan Kutz. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proceedings of the national academy of sciences , 113(15):3932-3937, 2016.

Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Physics-informed neural networks: Adeep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics , 378:686-707, 2019.

Claude Elwood Shannon. A mathematical theory of communication. The Bell system technical journal , 27(3):379-423, 1948.

Yann Collet. Finite state entropy. https://github.com/Cyan4973/FiniteStateEntropy , 2013.

Xinyue Luo, Yu Chen, Jin Cheng, and Zhaoyi Sun. A mechanism-based method for image inpainting. Applied Mathematics Letters , page 109113, 2024.

Kai Zhao, Sheng Di, Xin Lian, Sihuan Li, Dingwen Tao, Julie Bessac, Zizhong Chen, and Franck Cappello. Sdrbench: Scientific data reduction benchmark for lossy compressors. In 2020 IEEE international conference on big data (Big Data) , pages 2716-2724. IEEE, 2020.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The abstract and introduction clearly outline the novel approach of MeLLoC, its application in lossless compression of scientific data, and the key idea of using high-order mechanism learning. They set the stage for the paper's contributions, such as the proposed method's effectiveness and the experiments conducted to validate its performance, which are in line with the detailed content of the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: The paper recognizes that MeLLoC's performance is tied to accurate model representation and highlights the computational challenges for large datasets. It also notes the necessity for precision tuning, showing an awareness of the method's limitations in Section 6.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [NA]

Justification: The paper focuses on presenting a novel method for lossless compression of scientific data and does not explicitly present formal theoretical results or proofs. The emphasis is on the practical application and empirical evaluation of the MeLLoC framework, with performance demonstrated through experimental results. As for the theoretical guarantees, we refer to the previous work in [17].

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: The paper thoroughly details the MeLLoC framework, including the algorithmic process and the datasets used for evaluation. It provides performance metrics and comparative analysis with existing methods, which are crucial for replicating the study.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [No]

Justification: The paper utilizes open-sourced data, the origins of which are clearly cited, enabling the replication of experiments. However, the code is not publicly available due to restrictions of an ongoing commercial project. Despite this, the authors are open to academic collaboration, offering an avenue for interested researchers to engage in further communications.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Although the proposed method does not involve neural networks or a training process, the paper thoroughly specifies the experimental framework. It includes comprehensive details about the datasets utilized, the mechanism learning optimization process, and the precision control strategy, which are pivotal for understanding and replicating the results.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: The paper presents a clear reconstruction error measure and includes frequency distribution plots, which support the statistical significance of the results.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: The paper directs readers to section 5 for detailed experiment settings, and execution times are specifically listed in Table 1.

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research conducted in the paper is designed to comply with the NeurIPS Code of Ethics, adhering to the principles of fairness, accountability, and transparency in scientific research.

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: The paper does not explicitly discuss societal impacts as it focuses on a novel lossless compression method for scientific data. The primary focus is on the technical advancement and its application in scientific computing rather than on societal implications.

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper does not discuss safeguards for data or model release as the focus is on a lossless compression technique rather than on models or datasets that inherently carry a high risk for misuse. The research does not involve pretrained language models, image generators, or scraped datasets that would typically necessitate such precautions.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: The paper ensures that all creators and original owners of the assets used are properly credited, and it includes explicit mentions of the licenses and terms of use, which are respected accordingly.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: The paper does not introduce new assets such as datasets, code, or models that would require documentation to be provided alongside.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects, as it focuses on a technical methodology for lossless data compression.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve research with human subjects, as it is centered on a lossless compression technique for scientific data.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a novel approach combining high-order mechanism learning with classical encoding techniques to enhance lossless compression for large-scale scientific floating-point data. The core innovation lies in treating data as discrete samples from a physical field governed by differential equations and solving inverse problems to identify compressible coefficients. Experiments demonstrate MeLLoC's superior performance over existing methods, achieving better compression ratios and computational efficiency.\n\nSTRENGTHS:\n1) Innovative Approach: Combines mechanism learning with classical encoding, leveraging differential equations to compress scientific data effectively.\n2) Superior Performance: Outperforms state-of-the-art lossless compression techniques in terms of compression ratios and computational efficiency.\n3) Comprehensive Experiments: Extensive testing on various datasets highlights the robustness and efficacy of the proposed method.\n4) Flexibility: Capable of handling different types of scientific data, including those with high-order information and noise.\n\nWEAKNESSES:\n1) Model Dependency: Performance relies heavily on the accuracy of the differential equation models representing the data.\n2) Complex Calibration: Precision control requires careful calibration, which can be challenging with diverse data characteristics.\n3) Computational Intensity: Despite improved efficiency, the method still demands significant computational resources for large datasets.\n4) Generalization: May not be as effective for datasets that do not conform well to the assumed physical models.\n\nQUESTIONS:\n1) Clarification on Differential Equation Models:\nCan you provide more details on how you determine the appropriate differential equation models for different datasets?\nHow sensitive is the performance of MeLLoC to the choice of these models?\n2) Precision Control:\nCould you elaborate on the process for calibrating the precision control?\nAre there any guidelines or heuristics you follow to balance the compression efficiency and computational cost?\n\nLIMITATIONS:\nThe authors have recognized and addressed several limitations of their work:\n1) Model Dependency: The performance of MeLLoC depends on the accuracy of the differential equation models representing the data.\n2) Calibration Complexity: Precision control requires careful calibration, which can be challenging with diverse data characteristics.\n3) Computational Intensity: The method demands significant computational resources for large datasets.\n4) Generalization Issues: MeLLoC may not be as effective for datasets that do not align well with the assumed physical models.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces MeLLoC (Mechanism Learning for Lossless Compression), an approach that combines high-order mechanism learning with classical encoding to enhance lossless compression for scientific data. The core concept is to interpret the data as discrete samples derived from an underlying physical field described by differential equations. It addresses an inverse problem to identify coefficients of the governing equations, aiming to achieve a more compressible numerical representation.\n\nSTRENGTHS:\nThe proposed MeLLoC is innovative in its approach of treating data as samples from a discretized physical field. It solves an inverse problem to determine the source terms of the governing differential equations, resulting in a more compressible numerical distribution. MeLLoC demonstrates a higher compression ratio and faster compression speed compared with existing methods.\n\nWEAKNESSES:\nThe readability is weak. More detailed explanations and experimental analyses are necessary. For instance, the concept of Mechanism Learning is not novel and should be introduced in more detail. Additionally, the comparison with other methods and datasets is insufficient. For example, the authors should compare the proposed approach with [1], [2], and other relevant methods, if feasible.\n\nRef: \n\n[1] Knorr, Fabian, Peter Thoman, and Thomas Fahringer. \"ndzip: A high-throughput parallel lossless compressor for scientific data.\" 2021 Data Compression Conference (DCC). IEEE, 2021.\n\n[2] Afroozeh, Azim, Leonardo X. Kuffo, and Peter Boncz. \"ALP: Adaptive Lossless floating-Point Compression.\" Proceedings of the ACM on Management of Data 1.4 (2023): 1-26.\n\nQUESTIONS:\n1.\tIs [1] a lossless compression method? It is recommended to clarify the differences between [1] and this paper for a better understanding.\n2.\tHurricane appears to have a smaller reconstruction error as in Figure 5. Why then does Hurricane exhibit a lower compression ratio compared to CESM-ATM?\n3.\tWhat are the challenges in applying this method to other domains (e.g., medical imaging, oceanography as mentioned in the Conclusion)?\n\nRef: Luo, Xinyue, et al. \"Precision-preserving Compression of Scientific Data: Learn Mechanism from Data.\" 2024 Data Compression Conference (DCC). IEEE, 2024.\n\nLIMITATIONS:\nNone", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-08"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper propose a near lossless compression method named MeLLoC to compress the scientific data by learning the inherent mechanisms. By solving the inverse problem of Partial Differential Equations (PDEs), MeLLoC transforms the scientific data from original data domain into discretized source domain, which is much easier to compress. Besides, several techniques including precision control, fast Fourier-based solver and preprocessing for high-order mechanisms are proposed to facilitate the compression efficiency. Experimental results show that MeLLoC outperfoms two previous methods (FPZIP and Zstandard).\n\nSTRENGTHS:\nThe paper studies an interesting and promising direction that leverages mechanism learning to compress scientific data.\n\nWEAKNESSES:\n1. The proposed MeLLoC is built on the previous work [Luo et al. (2024a)]. However, it seems that the whole architecture of MeLLoC is the same as that in [Luo et al. (2024a)] and there is no new contribution.\n\n2. The proposed method is not truly lossless compression as claimed, but is near-lossless compression. Section 5.1 shows that the largest reconstruction error between the original data and reconstructed data is approximately in the order of 10^{-11}. However, A single-precision floating-point number can be accurate to the order of 10^{-38}.\n\n3. Presentation in Section 3 is somewhat confusing. For example, it is not clear why the difference operator $\\mathcal{L}$ is formulated in exactly a 9-point form and how the loss function $F$ is minimized.\n\n4. Evaluations in Section 5.3 are not sufficient. Some recent methods such as [Klöwer et al.(2021)] and [Luo et al.(2024a)] are not compared.\n\n5. Sections 2.1 and 2.2 are not related to the topic of this paper. It would be better to provide some background about PDEs and mechanisms of scientific data to inform more preliminary knowledge to Section 3.\n\nQUESTIONS:\n1. What is the differences between MeLLoC and the method proposed in [Luo et al. (2024a)]?\n\n2. Does MeLLoC achieve lossless compression or near-lossless compression? If it is near-lossless compression, what is the actual bit-rate to achieve lossless compression?\n\n3. Please explain the formulation of difference operator $\\mathcal{L}$ and loss function $F$.\n\n4. What is the performance of MeLLoC compared to [Klöwer et al.(2021)] and [Luo et al.(2024a)]?\n\nLIMITATIONS:\nThe authors have discussed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-13"}