MeLLoC: Lossless Compression with High-order Mechanism Learning
Authors: Xinyue Luo
Jin Cheng
Yu Chen
Venue: NeurIPS 2024 poster (2024)
Keywords: lossless compression
mechanism learning
inverse problem
Primary area: interpretability_and_explainability

Abstract: Lossless compression of large-scale scientific floating-point data is critical yet challenging due to the presence of high-order information and noise that arises from model truncation and discretization errors. Existing entropy coding techniques fail to effectively leverage the mechanisms underlying the data generation process. This paper introduces MeLLoC(Mechanism Learning for Lossless Compression), a novel approach that combines high-order mechanism learning with classical encoding to enhance lossless compression for scientific data. The key idea is to treat the data as discrete samples from an underlying physical field described by differential equations and solve an inverse problem to identify the governing equation coefficients exhibiting more compressible numeric representations. Periodic extension techniques are employed to accelerate the decompression. Through extensive experiments on various scientific datasets, MeLLoC consistently outperforms state-of-the-art lossless compressors while offering compelling trade-offs between compression ratios and computational costs. This work opens up new avenues for exploiting domain knowledge and high-order information to improve data compression in scientific computing.

MeLLoC: Lossless Compression with High-order Mechanism Learning
Xinyue Luo' Jin Cheng Yu Chen
School of Mathematical Sciences, Fudan University
"Scoool of Mathematics, Shanghai University of Finance and Economics
xinyueluo21 @m. fudan. edu. jchengåfudan. edu. yuchenãsufe edu.
Abstract
Lossless compression of large-scale scientific floating-poing data is critical yet chal- lenging due to the presence of noise and high-order information that arises from model truncation and discretization error.. Existing entropy coding techniques fail to effectively leverage the mechanisms underlying the data generation process. This paper introduces MeLLoC(Mechanism Learning for Lossless Compression), a novel approach that combines high-order order mechanism learning with classical encoding to enhance lossless compression for scientific data. The key idea is treat the data as discrete samples from an underlying physical field described by differential equations, and solve inverse problem identify the governing equa- tion coefficients exhibiting more compressible numeric representations. Periodic extension techniques employed accelerate the decompression. Through extensive experiments on various scientific datasets, MeLLoC consistently outper- forms state-of-the-art lossless compressors while offering compelling trade-off's between compression ratios and computational costs. This work opens up new avenues for exploiting domain knowledge and high-order information to improve data compression in scientific computing.
Introduction
In the modern era, vast amounts of floating-point data are generated from large-scale simulations and experiments, often reaching petabyte levels. Efficient compression of data essential for reducing storage costs and facilitating data transfer and analysis. Consequently, the compression of scientific data has gained increasing attention.
Lossless compression aims to reduce the size of data while allowing perfect reconstruction of the original data from its compressed representation. Classical lossless compression algorithms like Gzip and Bzip2 employ entropy encoding techniques like Huffman coding and arithmetic coding to exploit statistical redundancies in the data 2]. While effective for general-pus purposee data, these methods typically underperform when applied to scientific flotting-poin datasets, as they fail to capture the distinct characteristics of such data. In contrast, specialized compression algorithms like FPZIP(3), 4FP 25td[5] and ALP(6) are designed to handle floating-pon data more effectively by employing predictive coding customized entropy coders that account for the spatial correlations and value distributions present in scientific datasets.
A common challenge in scientific data compression is the presence of high-order information and randomness, or noise, which arises from roundoff errors during simulation, as well as model truncation and discretization enrors. This noise, termed as "false precision" by Zender(7), can significantly lower the compression ratio. The reason is that lossless compression algorithms do not distinguish the informative mechanism from the non- non-informative noise in data, thus leaving the level of meaningful precision unassessed and failing to exploit the high-order information in the data. By leaving the
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

noisy components, the data is more likely to exhibit detectable patterns, indicating that their size can be effectively reduced by compressor
To address this issue, Klöwer et al. propose a method of truncation based on evaluating the bitwise dependence between adjacent grid points (9). However, denoising by truncation bit digits may not be an optimal approach in practice due to the non-uniform distribution and varying levels of noise in scientific data across different spatial and temporal contexts, which risks confounding the scientific insights gleaned from the data. A recent work by Luo et al. (TO] considers locally characterizing the data's 's spatial relationship with linear differential equations, then separating the noise instead of using truncation. While this method offers more nuanced approach to noise handling, it's important to note that both these techniques still constitute forms of lossy compression.
Scientific data typically conforms to well-defined model mechanisms, often characterized using differential equations [1]] According to the theory of well-posedness for differential equations, the solutions to the differential equations are determined by the initial and boundary conditions, as well as the source terms[12]. This suggests that the entire physical field could be represented by smaller, compressed set of values that describe these mechanisms. However, most existing lossless compression techniques do not exploit this inherent structure and high-order information in scientific floating-point data.
In this paper, aim further enhance the effectiveness of lossless compression by learning the inherent mechanisms underlying scientific data, building on previous ncar-lossless work[IO]. The key idea is to treat the data samples from discretized physical field and solve an inverse problem for the governing differential equations to obtain the source terms, which exhibit a more compressible numeric distribution.
Our main results can be summarized as follows:
1. We propose McLLoC (Mechanism Learning for Lossless Compression), a novel approach that combines high-order mechanism learning with classical encoding algorithms to achieve efficient lossless compression of scientific srating-poim data.
2. We implement periodic extension techniques to accelerate the solution of the large linear systems arising in the parameter identification problem.
3. Through comprehensive numerical experiments, we demonstrate that MeLLoC consistently outperforms the existing methods while offering compelling trade-offss between compression ratios and computational requirements.
2 Background
Partial Differential Equations behind Scientific Data. Partial Differential Equations (PDEs) are fundamental in modeling complex physical phenomena across various scientific disciplines. They provide a mathematical framework for describing spatiotemporal changes in physical quantities. In occanography and climate science, PDEs are widely used to model phenomena such as heat transfer, fluid dynamics, and wave propagation. Specifically in oceanography, these equations model ocean circulation, temperature distribution, and salinity patterns, often involving multiple variables and intricate interactions. While the specific equations can vary depending on the phenomenon being studied, they generally capture the fundamental physical principles governing the system's behavior.
The complexity of these PDEs and their numerical solutions often results in large-scale doating-poin datasets, which are the focus of our compression efforts. Understanding the underlying mathematical structure of these datasets crucial for developing effective compression techniques that can preserve important physical features while reducing data volume.
Mechanism Learning of Scientific data Mechanism learning, or data-driven discovery of gov- erning equations, has gained significant attention in scientific computing. This approach aims to uncover the underlying physical laws from observational or simulation data. Key techniques in this area include Sparse Identification of Nonlinear Dynamics(SINDy [3]] Phhssics-Informed Neural Networks(PINNs) 14 These methods provide foundation for identifying compact representations of complex systems, which can potentially be applied for data compression. However, SINDy requires temporal evolution data to learn the underlying dynamics, while PINNs typically need large datasets for training, making them less suitable for case-by-case data compression scenarios.

---

Entropy Encoding, Entropy encoding is fundamental lossless data compression technique that assigns shorter codes frequently occurring symbols and longer codes rare ones, based on the statistical properties of the data. The encoding scheme is designed to minimize the overall bit length of the encoded message. Shannon's ' source coding theorem [5] provides the theoretical basis, suggesting that the optimal code length for symbol x is log (x), where the expected code length bounded by the entropy of the source distribution D.
Finite State Entropy [6 is notable entropy encoding method that offers competitive compres- sion ratios and fast encoding and decoding speeds. FSE constructs finite state machine to model the probability of symbols, adapting to the changing statistics of the imput data. This adaptability makes FSE particularly effective for compressing data with non-uniform or unknown distributions.
3 Methodology
3.1 Overview of the MeLLoC Framework
Building upon PDE theory and mechanism learning concepts discussed earlier, we develop our compression methodology specifically for scientific data. We demonstrate our approach using two- dimensional (2D) scenarios, though the methodology can extended to higher dimensions. Our method takes advantage of the inherent mathematical structures present in scientific data to achieve more effective compression.
Original Data
Boundary Upd Source Term f Coefficient 0
Local mechanism Kru buna I
Precision Control
Solve Lo- = with ulao Upd
Reconstructed Data ù
Encoding and Decoding
Figure 1: Overview of the proposed compression architecture.
We first exemplify the idea through the Laplace equation, canonical PDE representing steady-state conditions across various fields such as temperature distribution and electric potential
Au 0.
When the data represents the numerical solution of the aforementioned equation, derived from a second-order central difference scheme applied to grid with uniform spacing, it complies with the following equation
Ua. (us.j-1 + Mij+1 0:-1.j + ((+1g))
This implies that the value at any central point is calculable based on the values of its immediate neighboring points. Furthermore, in this scenario, the data is entirely determined by its boundary values, suggesting that only the boundary data need be stored to fully reconstruct the original dataset.
In practice, data may exhibit non non-hooogens conditions, described by Au = f. where f denotes source term. In such cases, both the boundary values and the source need to be stored. Compression remains viable, provided the information entropy of f is subordinate to that of condition often satisfied when data exhibits elliptie properties or when the source term is of low magnitude and sparse.

---

To address data governed by diverse mechanisms, our model is parameterized and calibrated through learning from the data. We assume that of two-dimensional data satisfies a difference equation Lui,j = Jūjj. where L represents the difference operator. Globally, linear system for the interior unknowns n can be constructed as follows
Keu babd.j-
Here, Kg. the stiffness matrix corresponding to the difference operator, while by of comprises boundary data and source term data and represents the 'data support' which refers to the data needed for the well-posedness the equation.
Assume that the scientific data can be locally approximated by a homogeneous second-order linear this context, our goal to learn the difference operator
PDE with constant coefficients. by minimizing the source term Ifl, thereby achieving maximal sparsity and reduced information entropy. Consequently, only the boundary data ad and the sparse source term / are stored. During decompression, the data can be reconstructed using the matrix Ke.The proposed method is not limited to specific difference formats and can effectively handle variable coefficient equations.
To facilitate the compression, the difference operator C is formulated in compact 9-point form and parameterized by [C:]a-1- The coefficient template for the combination of neighboring points is depicted in Figure 2 a), illustrating how local representation translates global connectivity among data points, as shown in Figure bb... Figure c)) provides some typical templates for differential equations.
Callygn
-1/4
-1/4
-1/2
Gall-ng
Cally
-1/4
Elliptic
Hyperbolic
Parabolic
(a) Coefficient template.
(b) Intra predictions.
(c) Templates some differential equations.
Figure 2: Local representation notations.
In the compression process, we need to determine the parameters 0 (c:)%11 such that
C145-15-1 + Cgus41g+1 = fi.j:
and minimize the magnitude of source term ft.j Therefore, learning the operator (equivalently, the corresponding stiffness matrix) transforms into solving an optimization problem formulated
bd argminF (0; udd ud
0
where, by normalizing 0 with C3 = 11, the loss function is defined by
= Loud 1l2 =
Σ f
(Cuu-15-1+ +... C9u441541 Wi.j)
In applications, to reduce computational cost or highlight local features, a strategic approach involves selecting only localized subset of the data to learn the difference operator Lg. During decompression, we obtain n by solving equation Kgu* b.
Our approach is particularly suitable for compressing numerical solutions of differential equations. In addition, it performs well for data with explicit physical mechanisms, such as those encountered in atmospheric and oceanic observational datasets within the industrial sector. The learned nine- point coefficient template L. (the difference operator) corresponds second-order differential equation and is generally applicable. Furthermore, the stability of both the compression and decompression processes assured by the well-posedness of the difference equation.
mechanism described by

---

3.2 Model identification and its well-posedness
Consider the underlying model of the data. For u (9)), Ω R2, the nine-point difference template is related to the second order linear differential operator as
CR kh, y + 1h) =((c; + + cz) + (cs + 07 og)
2(c7 cz) 2(c1 + ca)hO, + + + cs + c,) (hh²)
ca)h
Here CR.t are the coefficients in Figure 2 a) and the subscripts k, represent the relative position the data point (x. v). The relationship between CK./ and Ca can be expressed as
C= C2 + c3A3 + 44 + A5 + ca AB + c7A, + As cg
where is the matrix of CR.-- A, are basis matrices, and c, are corresponding coefficients. The matrices {A, 10 are defined as
0
0
0
1
0
0
0
0
A
-1
0
1
A2
0
0
0
A3
1
2
1
0
0
0
0
0
-1
0
0 0
0
T
1 -2 1
07 0
0 0
0
17
0
A4
0
As
0
A6
0
0
[O
0
-
0
0
0
-1
0
0
17
0
07
0
07
A7
0
-2
AA
0
-2
A9
0
1 0
0
0
0
0
[
0
0
Based on this representation, encoding becomes encoding the sparser high-order term ohh?), i.e., the source term f. Therefore, the optimization objective is to obtain a minimized high-order term, which can be mathematically expressed
2
[C argmin F(Ck,1, argmin Ct,1
Σ A1==11
Cn, + kh, 1h)
CLa
Once the template 0 = Ck is learned, one can calculate the coefficients of the differential operator, allowing classify the mechanism as elliptic, parabolic, or hyperbolic due to the reversibility of Ck, and ca. which provides interpretability of the model.
Next, discuss the well-posedness of the model identification problem (compression process), while that the decompression process will be discussed in Section 3.5 The above minimization problem is equivalent to solving the least square problem
where [C Co]T. A Nx9 N is the number of data points in domain D, R. and & = P(i.j), (i.j) E D is the index after rearranging the data into 1-dimensional vector, with (i.j) P [k), k 1.... .N, as its inverse mapping.
A |.--,,j- [++1++1] - -P
To find non non-trivial solutions is to obtain the null space (kernel) of B = AA If B is full-rank, there only trivial solution, while otherwise, there no uniqueness. To address this issue, we set the coefficient of úiyj to (i.e. set Cs to -1 in Figure a)). and fix the template size to 8. The problem then becomes
-ē= b.
where ē _8 À I Nx8 and b = [up 1(1). ... up INN) N The problem has unique least squares solution ē == Ätb, provided the data are not all degenerated. We assemble À and directly solve the pseudo inverse, which serves a fast solver for the compression process.
5

---

3.3 Data Composition
To illustrate the method's capacity for lossless compression, consider the decomposition of data predicated upon linear superposition
= L un + Merr-
This equation stratifies the dataset into three fundamental components:
L- representing G f. with G being the Green's function for the domain. This part corresponds to solution to the non-homogeneous equation with homogeneous boundary conditions, determined by the source f (2D).
BC ) denotes the solution to the homogeneous equation is determined only by uo(= By the boundary data (1D).
Verr denotes the residual part (2D).
The precision control discussed in the following enables us optimize the source term and retaining sufficient significant figures based on data precision to nullify Yerr, thereby yielding f + uo. This decomposition allows to store only the boundary data 1dd and the sparse source term f. achieving compression while preserving the essential information content of the original dataset.
3.4 Precision Control
Suppose the data u possesses a precision of m bits, expressed in decimal form, with a binary equivalent exhibiting similar properties. This precision implies accuracy to the order of 10-m. as depicted in Figure a). By analogy, the precision of the model coefficients C₁ quantified as bits, corresponding to 10 ". Consequently, when the source term is computed via f L(u) Σ Cru,, the precision of f is up 10- m 10-a 10-(m+))
The data can be losslessly recovered or decompressed once the error from solving the linear system K Keuin by,dd_r .f less than 10-(m+n), which feasible with proper n. As n increases, the admissible set for C; enlarges, contributing to lower absolute value for but higher precision. The optimal compression ratio is reached when significant digits of f are minimized. We optimize for coefficients C₁ by starting with high precision and gradually reducing it while monitoring reconstruction error and compression ratio. After several calibrations, n can fixed for the remaining dataset if the compression ratio for subsequent batches shows no significant fluctuation.
Data u
digits with precision
Case I: High prec Is lon digits.
ap
21
d1
d2
am
b
by
JTt decimal places
Large absolute value.
digits with true precision
b by Dka
0
0
b b2
bx-1
bx
Case III: Optimal scenario.
Source f
m + n decimal places
bgs
(a) Precision digits and decimal places.
(b) Trade-off between values and digits.
Figure 3: Illustration of precision control.
This approach considers different scenarios balance significant digits and value magnitudes, as illustrated in Figure ().. The trade-off between fewer effective digits of compared to contributes compression, while increasing precision in coefficient truncation expands the solution space, poten- tially decreasing the absolute value of the optimized source term but increasing storage requirements. Our method, MeL.oC, optimizes compression efficiency, balancing perfect reconstruction with computational feasibility. The precision control is adaptive and can be tailored to different scientific datasets, ensuring that the compression process optimized for various types of atmospheric and climate model data.

---

3.5 Fast Fourier-based Solver
The traditional approach to solving systems of equations T can be computationally intensive, particularly when dealing with large datasets. To address this issue, we propose Fourier-bast -ased solver that accelerates computation while preserving data integrity.
We first extend the discrete field data periodically and expand it using a 2D Fourier series
N2
Ni
uske x1 (J 1))-11) 2ei (k- 1)n-))
u(m, n)
N1
where i v-1 represents the imaginary unit, and ú repre- sents the Fourier coefficients. is essential that the double-layer boundary condition enables periodic extension for the source field By substituting the Fourier series into the difference equation, we obtain a system relating these coefficients
Figure 4: Schematic representation of periodic continuation.
BI. +
for j, 1, ,N1 and N2.
ÜjAk O,
By
Here, f represents the Fourier coefficients of f. By. is coefficient involving the difference operator and the Fourier hasis functions. The specific form of By. depends on the discretization scheme used in the original differential equation[17]. In general cases, the double-layer boundary values are necessary to determine the base frequency Fourier coefficients, ensuring the well-posedness: For example, when n
N₂
N1
u(m, 1)
úyke Ni (m-))(m-1)
1 N
ügk
(i-1)(m-1)
N1
N2
By applying DFT to u(m, 1), obtain ūy satisfying u(m,
Uge
L1
(j (1-1)(m-1) Utilizing
No
the orthogonality, we can derive ūj
ujk and therefore úÿ1 N2ñ;
Uik--
Nº R=1
R=2
This approach allows the efficient computation of u1R through similar reasoning. By employing FFT and its inverse (iFFT), the algorithm accelerates the matrix approximation process during both compression and decompression phases, with a computational complexity of ( log N). This Fourier-based approach significantly enhances computational efficiency compared traditional matrix computation methods, thereby ensuring the throughput of compression and decompression operations while maintaining lossless reconstruction of the original data.
3.6 High-Orde Mechanisms
While the proposed 9-point difference operator L primarily designed to capture second-order PDE dynamics, many real-world datasets may exhibit higher-order effects arising from more complex governing equations or fine-scale features. These higher-order mechanisms can manifest the source term f. potentially limiting the compression performance if not properly addressed.
To better characterize these higher-order properties, preprocessing step involving the introduction of diffusive operators can be employed. Let denote a diffusive operator, such as = a 2 + 34, where A is the Laplacian and a, 3 are constants. We can apply this operator to the original data to obtain a smoothed version ă = Du, which effectively filters out the fine-scale, high-order features.
The compression then performed on ă instead of u, yielding sparse source term that encapsulates the higher-order effects Lä = During decompression, the inverse diffusive operator D-1 applied to recover the original data
D ( D- D (c-1f ūo).
(3)
7

---

This preprocessing strategy allows capturing high-order mechanisms by absorbing them into the sparse source term f, while the 9-point operator L focuses representing the underlying second- order dynamics in the smoothed data û. The diffusive operator approach maintains computational efficiency, lossless compressio/decompressi The diffusive operator D can be tailored to dataset, potentially with learnable parameters for optimized filtering. Iterative applications of D enable progressive extraction of high-order features at multiple levels.
and enhances handling of complex, high-order data.
4 Algorithm
The algorithm uses an optimized 9-point template to achieve lossless compression and decompression of data, which is mainly divided into two parts: compression and decompression.
Algorithm 1 Compression based optimizing source term
Require: Data and precision
Ensure: Coefficients C, source term f. and boundary values rnd
1: Select initial template Lou (hyperbolic, elliptic, other) and coefficient truncation precision
2: Set S1 + 1, the adjustment count S2 <- 1, and the upper limit for the adjustment count S2".
3: while $1 > 0 and 52 < $2" do
4: if high-order mode is required then
5:
Compute the source term ú1 Lou using the initial template.
6: else
7:
U1 u. end if
8:
9:
Optimize the template coefficients C. arg mine (L
10:
Obtain the source term f +- L.U1. Truncate the source term (m-(m))) (or binary truncation): +- truncated source term. Encode f. increment S2, check the residual error, and verify the compression rate. if the compression rate is unsatisfactory then
12:
14:
Adjust n, set $1 <- else
16:
S1 + O.
17:
end
18:
end while
The compression part includes selecting the initial template, setting the coefficient truncation precision, and optimizing the template coefficients to minimize the residual error and information entropy of the compressed data. The compression process also includes truncating the source term to the required precision and encoding it using appropriate encoding scheme. The compression part terminates when the compression rate satisfactory or when the maximum number of iterations is reached.
Algorithm 2 Decompression based on optimized source term
Require: Template coefficients (preprocessing template Co. if any), source term f, boundary values ad (preprocessing boundary un bd if any), and data precision m.
Ensure: Data array u.
1: if high-order mode is required then
2: Assemble the stiffness matrix Ko(Co).
3: Assemble the right-hand side d(ufrt hd f).
4: Solve the equation Kofi = d.
5: else
6: f1
7: end if
8: Assemble the stiffness matrix KCC).
9: Assemble the right-hand side dd fi).
Solve the equation K d.
11: Reconstruct u with the precision of original data.

---

The decompression part includes assembling the stiffness matrix and right-hand vector using the opti- mized template coefficients, solving the linear system obtain the compressed data, and truncating it to the original precision. The decompression process also includes decoding the compressed source term using the same encoding scheme and using to reconstruct the original data.
5 Experimental Results
This section presents the experimental results of applying proposed lossless compression algorithm to the CESM-ATM and Hurricane datasets from the SDRBench[[8, 18]. The performance is evaluated based on the original data, transformed data (source terms post- transformmatin)) reconstruction error, frequency distribution plots, compression ratio, throughput, and comparison with existing algorithms such Zstandard (zstd) and fpzip 3). All tests were conducted Mac with M1 Silicon, macOS 1,,1.2, 16GB RAM.
5.1
Compression and Reconstruction
The original datasets were first transformed using our proposed method to generate source terms suitable for encoding. The transformed data were then compressed, and the reconstruction error was calculated by comparing the original data to the data reconstructed from the compressed representation.
400
Original Data
Source Term
500
Origial Data
Source Term
300
400
300
500
300
200
200
100
100
100
100
100
200
300
400
120
200
200
230
300
(a)CESM-ATM,
(b)Hurricane.
Figure 5: Demo of the proposed scheme on CESM-ATM and Hurricane datasets.
Figure illustrates the results obtained by applying the proposed scheme to the CESM-ATM and Hurricane datasets. For the CESM-ATM data, while the original data values span a range of 220 to 300, the source term exhibits a more compressed range of -0.1 to 0.1, indicating effective compression. Notably, the reconstruction error was found to be around 10-11, which is smaller than the least significant digit of wegge--preiisonn floating- point representation. For the Hurricane dataset, the reconstruction error is even smaller, on the order of 10 12, These results demonstrate that the proposed method preserves the numerical precision essential for scientific computations. The extremely low reconstruction errors ensure that the compressed data can be used reliably in high-precision applications, maintaining the integrity of the original datasets for subsequent analyses and simulations.
5.2 Frequency Distribution Analysis
To further illustrate the effectiveness of our compression method, we compared the frequency distribution plots of the data before and after compression. The left represents the original data, while the right represents the stored source terms. The histograms employ logarithmic binning for clarity, demonstrating the reduced mean and standard deviation of the source term, which contributes to decreased entropy and, consequently, a better compression rate.
Moy Cours
4.25
Valus
7400
CTM.
Hurrriccaen..
Figure 6: Frequency distribution plots of data before and after compression.

---

5.3 Performance Evaluation
We evaluate MeLLoC against several state-of-the-art compression algorithms using two key metrics: compression ratio and throughput. The compression ratio indicates the efficiency of data reduction, with higher values representing compression and decompression processes, which is crucial for handling large scientific datasets. Table presents comprehensive comparison across CESM-ATM and Hurricane datasets.
compact representations. Throughput measures the speed of
Table 1: Performance comparison.
CESM-ATM
Hurricane
Method
Ratio
Compression
Decompression
Method
Ratio
Compression
Decompression
ALP
1.16 x
46.93 Mb/s
1054.95 95 Mb/s
ALP
1.11
45.74 Mb/s
973.63 Mb/s
FPZIP
1.63 x
59.68 Mb/s
70.94Mb/s
FPZIP
41.22Mb/s
53.95Mb/s
ZFP
1.02x
96.17 Mb/s
81.97 Mb/s
ZFP
1.01
102.95 Mb/s
68.06 Mb/s
Blose
1.30x
293.71 Mb/s
632.76 Mb/s
Blosc
1.12
Mbís
6516.29 Mb/s.
Gzip
1.89 x
1.40 Mb/s
266.94 Mb/s
Gzip
33.25 Mb/s
212.35 Mb/s
Zstandard
2.69 x
10S.51Mb/s
152.81Mb/s
Zstandard
2.78 x
69.51Mb/s
271.32Mb/s
MeLLoC
3.36 x
188.77 Mb/s
179.76Mb/s
McLLoC
3.29x
206.80Mb/s
190.35Mb/s
ALP and FPZIP lossless floating-point compression algorithms, while ZFP set to lossless mode with zero error tolerance. Blosc, Gzip, and Zstandard are general-purpose lossless compression algorithms. MeLLoC, the proposed method, consistently achieves the highest compression ratios (3.36x for CESM-ATM and 3.29x for Hurricane) while maintaining competitive throughput.
5.4 Scalability Performance Analysis
Compression Ratio and Speed vs File Size
29
In Figure compression ratio, represented by the orange line, shows slight decline file size increases, dropping from around 3.4 to just above 3.1 for the largest files. Compression and decom- pression speeds, depicted by the blue and green dashed lines respectively, remain relatively stable across varying file sizes, with minimal fluctua- tions. Compression speeds consistently outper- form decompression speeds, both hovering within a range of 150 to 200 MB/s across all file sizes. This indicates stable and efficient scalability pattern in the data compression process.
35-
150
23
100
2.0
27
20
80
100
120
Skee
Compression Ruto
Compress en Spood
Ducompresoor Spood
Figure 7: Performance Metries Across File Sizes.
6 Conclusion
In this paper, we present a novel mechanism-learmine method for compressing scientific data. Our proposed scheme effectively preserves the true precision of the data while enabling subsequent lossless compression. Additionally, the approach offers flexibility by allowing the use of various lossless encoding algorithms, which can be selected based on factors such as availability, speed, and resulting file sizes, tailored to specific application requirements. Experiments demonstrate that the proposed method has competitive reconstruction performance with the general- pprrpose methods, while providing a novel perspective on scientific data compression. McLLoC has several limitations. As discussed in Section 3.1 its performance is contingent upon the accuracy of the differential equation models representing the data. The method is particularly suitable for scientific data whose order of magnitude aligns well with the assumed physical models. For datasets that do not fit these criteria, the method may not be as effective. The precision control, crucial for optimizing compression, also necessitates careful calibration, which might challenging with diverse data characteristics. Despite these constraints, MeILoC presents a valuable approach for scientific data compression. In future work, we intend to apply the meehanismm-lerningg concept to several pertinent real-world domains, such as medical imaging, occanography, and other related fields.
10

---

Acknowledgments and Disclosure of Funding
This work is supported by National Key Research and Development Programs of China (No. (009 103), National Natural Science Foundation of China (Nos. 12201386, 12241103), Science and Technology Commission of Shanghai Municipality (23IC1400501) and the Sino-German Mobility Programme (M-0187) by Sino-German Center for Research Promotion.
References
[1] Khalid Sayood. Introduction to data compression. Morgan Kaufmann, 2017.
[2] David JC MacKay. Information theory, inference and and learning algorithms. Cambridge university press, 2003.
[3] Peter Lindstrom and Martin Isenburg. Fast and efficient compression of floating- point data. IEEE transactions visualization and computer graphics, 12(5):1245-1250 2006.
[4] Peter Lindstrom. Fixed-rate compressed floating-point arrays. IEEE transactions on visualiza- tion and computer graphics, 20( 20(12):274-2688 2014.
[5] Yann Collet and Murray Kucherawy. Zstandard compression and the application/zstd media type. Technical report, 2018.
[6] Azim Afroozeh, Leonardo X Kuffo, and Peter Bonez. Alp: Adaptive lossless floating ppint compression. Proceedings of the ACM Management of Data, 144):1-26, 2023.
[7] Charles $ Zender. Bit grooming: statistically accurate wrecision-preserving quantization with compression, evaluated in the netedf operators (nco, v4. 4.8+). Geoscientific Model Development, (99:3199-3211 2016.
[8] Peter Lindstrom. Error distributions of lossy floating ppint compressors. Technical report, Lawrence Livermore National LLb.(LL,)), Livermore, CA (United States), 2017.
[9] Milan Klöwer, Miha Razinger, Juan J Dominguez, Peter D Düben, and Tim N Palmer. Com- pressing atmospheric data into real information content. Nature Computational Science, (11):713-724, 2021.
[10] Xinyue Luo, Jin Cheng, Zhaoyi Sun, and Yu Chen. Precision-presevvv compression of scientific data: Learn mechanism from data. In 2024 Duta Compression Conference (DCC), page 572. IEEE, 2024.
[11) Jianhong Shen and Tony F Chan. Mathematical models for local nontexture inpaintings. SIAM Journal on Applied Mathematics, 1019)110,, 2002.
[12] Lawrence C Evans. Partial differential equations, volume 19. American Mathematical Society, 2022.
[13] Steven L Brunton, Joshua L Proctor, and Nathan kutz. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proceedings of the national academy of sciences, 113(15):3322-393) 2016.
[14] Maziar Raissi, Paris Perdikaris, and George E Karniadakis. infooree neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686-707, 2019.
[15) Claude Elwood Shannon. A mathematical theory of communication. The Bell system technical journal, 27(3):379-422 1948.
[16) Yann Collet. Finite state entropy. https:///ithh/: htt.:hhww.w...ocaaa//////tttttta 2013.
[17] Xinyue Luo, Yu Chen, Jin Cheng, and Zhaoyi Sun. A mechanism-based method for image inpainting. Applied Mathematics Letters, page 109113, 2024.
[18] Kai Zhao, Sheng Di, Xin Lian, Sihuan Li, Dingwen Tao, Julie Bessac, Zizhong Chen, and Franck Cappello. Sdrbench: Scientific data reduction benchmark for lossy compressors. In 2020 IEEE international conference big data (Big Data), pages 2716-2724. IEEE, 2020.
11

---

NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly outline the novel approach of MeLLoC, its application in lossless compression of scientific data, and the key idea using high-order mechanism learning. They set the stage for the paper's contributions, such the proposed method's effectiveness and the experiments conducted to validate its performance, which are in line with the detailed content of the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: The paper recognizes that MeLLoC's performance tied accurate model representation and highlights the computational challenges for large datasets. It also notes the necessity for precision tuning, showing an awareness of the method's 's limitations in Section
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
The paper focuses on presenting a novel method for lossless compression scientific data and does not explicitly present formal theoretical results or proofs. The emphasis is on the practical application and empirical evaluation of the MeLLoC frame- work, with performance demonstrated through experimental results. As for the theoretical guarantees, we refer to the previous work in
Justification:
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: The paper thoroughly details the MeLLoC framework, including the algo- rithmic process and the datasets used for evaluation. It provides performance metrics and comparative analysis with existing methods, which are crucial for replicating the study.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, described in supplemental material?
Answer: [No]
Justification: The paper utilizes open-sourced data, the origins of which are clearly cited, enabling the replication of experiments. However, the code is publicly available due to restrictions of an ongoing commercial project. Despite this, the authors are open to academic collaboration, offering an avenue for interested researchers to engage in further communications.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
12

---

Justification: Although the proposed method does not involve neural networks or a training process, the paper thoroughly specifies the experimental framework. includes comprehen- sive details about the datasets utilized, the mechanism learning optimization process, and the precision control strategy, which are pivotal for understanding and replicating the results.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: The paper presents clear reconstruction error measure and includes frequency distribution plots, which support the statistical significance of the results.
8. Experiments Compute Resources
Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer: [Yes]
Justification: The paper directs readers to section for detailed experiment settings, and execution times are specifically listed in Table I
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: The research conducted in the paper is designed to comply with the NeurIPS Code of Ethies, adhering to the principles of fairness, accountability, and transparency in scientific research.
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification:
The paper does not explicitly discuss societal impacts as it focuses novel lossless compression method for scientific data. The primary focus is on the technical advancement and its application in scientific computing rather than on societal implications.
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper does discuss safeguards for data or model release as the focus is on lossless compression technique rather than models or datasets that inherently carry high risk for misuse. The research does not involve pretrained language models, image generators, or scraped datasets that would typically necessitate such precautions.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
The paper ensures that all creators and original owners of the assets used are properly credited, and it includes explicit mentions of the licenses and terms of use, which are respected accordingly.
13. New Assets
13

---

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not introduce new assets such as datasets, code, models that would require documentation to provided alongside.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects, focuses on technical methodology for lossless data compression.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
The paper does not involve research with human subjects, as it is centered on lossless compression technique for scientific data.
Justification:
14


Reviews:
[{"review_id": "zcdbfg1CaJ", "reviewer": "Reviewer_XGH7", "summary": "This paper propose a near lossless compression method named MeLLoC to compress the scientific data by learning the inherent mechanisms. By solving the inverse problem of Partial Differential Equations (PDEs), MeLLoC transforms the scientific data from original data domain into discretized source domain, which is much easier to compress. Besides, several techniques including precision control, fast Fourier-based solver and preprocessing for high-order mechanisms are proposed to facilitate the compression efficiency. Experimental results show that MeLLoC outperfoms two previous methods (FPZIP and Zstandard).", "questions": "1. What is the differences between MeLLoC and the method proposed in [Luo et al. (2024a)]?\n\n2. Does MeLLoC achieve lossless compression or near-lossless compression? If it is near-lossless compression, what is the actual bit-rate to achieve lossless compression?\n\n3. Please explain the formulation of difference operator $\\mathcal{L}$ and loss function $F$.\n\n4. What is the performance of MeLLoC compared to [Klöwer et al.(2021)] and [Luo et al.(2024a)]?", "limitations": "The authors have discussed the limitations.", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 1, "strengths": "The paper studies an interesting and promising direction that leverages mechanism learning to compress scientific data.", "weaknesses": "1. The proposed MeLLoC is built on the previous work [Luo et al. (2024a)]. However, it seems that the whole architecture of MeLLoC is the same as that in [Luo et al. (2024a)] and there is no new contribution.\n\n2. The proposed method is not truly lossless compression as claimed, but is near-lossless compression. Section 5.1 shows that the largest reconstruction error between the original data and reconstructed data is approximately in the order of 10^{-11}. However, A single-precision floating-point number can be accurate to the order of 10^{-38}.\n\n3. Presentation in Section 3 is somewhat confusing. For example, it is not clear why the difference operator $\\mathcal{L}$ is formulated in exactly a 9-point form and how the loss function $F$ is minimized.\n\n4. Evaluations in Section 5.3 are not sufficient. Some recent methods such as [Klöwer et al.(2021)] and [Luo et al.(2024a)] are not compared.\n\n5. Sections 2.1 and 2.2 are not related to the topic of this paper. It would be better to provide some background about PDEs and mechanisms of scientific data to inform more preliminary knowledge to Section 3.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We greatly appreciate the careful review of the manuscript. We sincerely hope that the following answers will better illustrate our work. We also recommend that the reviewer read the “Author Rebuttal.” We hope the reviewer finds the efforts and improvements we made in both the theoretical and experimental aspects.\n\n### 1. Clarification on Lossless Properties (To answer Q1 and Q2)\n\nWe appreciate your observation regarding the precision of single-precision floating-point numbers. However, it appears there is a misunderstanding. A single-precision float indeed has about 7 decimal digits of precision, which corresponds to approximately $10^{-7}$. MeLLoC achieves true lossless compression, allowing for perfect reconstruction of the original data from its compressed form. By controlling the precision of coefficients and source terms based on the inherent properties of the scientific data, this approach allows us to achieve better compression ratios while maintaining lossless compression.\nIn contrast, the work by [Luo et al. (2024a)] which incorporates a noise separation process, is indeed near-lossless. We emphasize that the proposed method does not introduce any loss of original information, and we have clarified this point in the revised manuscript.\n\n### 2. Novelty: Lossless Compressor & Fast Solver (To answer Q1)\n\nThanks for your valuable comments. We would like to highlight the novelty of MeLLoC: its lossless properties and a fast solver for the compression and decompression processes. As demonstrated above, the proposed method achieves lossless compression. Another novel aspect is the introduction of a fast solver based on PDE theory.  Our fast solver significantly accelerates both the compression and decompression processes. Specifically, our method has substantially enhanced performance, improving the compression throughput by 839.47\\% and the decompression throughput by 352.27\\% compared to [Luo et al. (2024a)].\n\nUnlike traditional compression methods, MeLLoC transforms the data into a sparser representation, i.e., high-order mechanisms derived from PDE models. The compression is accelerated due to the linearity and sparsity of this local representation, as shown in Figure 2. This allows for direct optimization to find the extremum. The existence and uniqueness of the minimizer during optimization are guaranteed by PDE theory. For more details, please refer to the \"Model identification and its well-posedness\" section in the \"Author Rebuttal.\" This is the first application of such rapid computational methods in the field of data compression. We believe this contribution is substantial and distinct from previous works, including [Klöwer et al.(2021)] and [Luo et al. (2024a)].\n\n### 3. Explanation of Technical Details (To answer Q3)\nThanks for your comments. We would like to expand on Section 3.1 to further explain how our method interprets data and how we formulate $\\mathcal{L}$ and $F$. The revised paper includes more detailed illustrations and analysis of model identification and the well-posedness of identification in the Appendix. For details, please kindly refer to the \"Model identification and its well-posedness\" section in the \"Author Rebuttal\".\n\n### 4. Evaluation Against Recent Methods (To answer Q4)\nWe appreciate your suggestion to include comparisons with recent methods. \nIn response, we have expanded our evaluation in Section 5.3 to include comparisons with other state-of-the-art lossless algorithms, ALP[1], ZFP[2], Blosc, and Gzip, as suggested by Reviewer VS2J. Please find the result in the attached PDF file in the \"Author Rebuttal\" section.\n\n> References:\n>\n> [1] Afroozeh, A., Kuffo, L. X., \\& Boncz, P. (2023). ALP: Adaptive Lossless Floating-Point Compression. Proceedings of the ACM on Management of Data, 1(4), 1-26.\n>\n> [2] Lindstrom, P. (2014). Fixed-rate compressed floating-point arrays. IEEE Transactions on Visualization and Computer Graphics, 20(12), 2674-2683.\n\n### 5. Relevance of Sections 2.1 and 2.2 (To respond to Weakness 5)\n\nWe apologize for the confusion caused by Sections 2.1 and 2.2. We intended to convey that the use of partial differential equations (PDEs) to describe human vision and observable phenomena throughout scientific history represents a process of compression and decompression within human intelligence. The process of abstracting observable phenomena, specifically translating human vision into mathematical equations, is a form of information extraction. This mechanism-learning process inspired us to propose a new data compression method, where compression and decompression reflect the extraction of data features and the intelligent associations behind human vision related to data mechanisms. We revised these sections to provide more relevant background on PDEs and the mechanisms of scientific data, ensuring that they better inform the preliminary knowledge necessary for understanding Section 3.\n\nIn summary, we appreciate your insights and have made revisions carefully according to your comments, to clarify the contributions and technical details of MeLLoC. Thank you again for your valuable feedback.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "A2ZmVrHyoF", "reviewer": "Reviewer_VS2J", "summary": "This paper introduces MeLLoC (Mechanism Learning for Lossless Compression), an approach that combines high-order mechanism learning with classical encoding to enhance lossless compression for scientific data. The core concept is to interpret the data as discrete samples derived from an underlying physical field described by differential equations. It addresses an inverse problem to identify coefficients of the governing equations, aiming to achieve a more compressible numerical representation.", "questions": "1.\tIs [1] a lossless compression method? It is recommended to clarify the differences between [1] and this paper for a better understanding.\n2.\tHurricane appears to have a smaller reconstruction error as in Figure 5. Why then does Hurricane exhibit a lower compression ratio compared to CESM-ATM?\n3.\tWhat are the challenges in applying this method to other domains (e.g., medical imaging, oceanography as mentioned in the Conclusion)?\n\nRef: Luo, Xinyue, et al. \"Precision-preserving Compression of Scientific Data: Learn Mechanism from Data.\" 2024 Data Compression Conference (DCC). IEEE, 2024.", "limitations": "None", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "The proposed MeLLoC is innovative in its approach of treating data as samples from a discretized physical field. It solves an inverse problem to determine the source terms of the governing differential equations, resulting in a more compressible numerical distribution. MeLLoC demonstrates a higher compression ratio and faster compression speed compared with existing methods.", "weaknesses": "The readability is weak. More detailed explanations and experimental analyses are necessary. For instance, the concept of Mechanism Learning is not novel and should be introduced in more detail. Additionally, the comparison with other methods and datasets is insufficient. For example, the authors should compare the proposed approach with [1], [2], and other relevant methods, if feasible.\n\nRef: \n\n[1] Knorr, Fabian, Peter Thoman, and Thomas Fahringer. \"ndzip: A high-throughput parallel lossless compressor for scientific data.\" 2021 Data Compression Conference (DCC). IEEE, 2021.\n\n[2] Afroozeh, Azim, Leonardo X. Kuffo, and Peter Boncz. \"ALP: Adaptive Lossless floating-Point Compression.\" Proceedings of the ACM on Management of Data 1.4 (2023): 1-26.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for the thoughtful comments and valuable suggestions to make the paper clearer. We have revised our manuscript taking all comments and suggestions into consideration.\n> Q1: Is [1] a lossless compression method? It is recommended to clarify the differences between [1] and this paper for a better understanding.\n\nThank you for your careful reading and instructive comments. While [1] does not achieve true lossless compression, our approach is designed specifically to ensure perfect reconstruction of the original data. In the revised version, we have emphasized the differences from [1] in terms of model principles and specific implementation, and we have added a more detailed analysis in the Appendix on the lossless implementation and the principles of fast solver. A discussion can also be found in the \"Differences between [Luo et al.(2024a)] and our work\" section of the \"Author Rebuttal\".\n\n> Q2: Hurricane appears to have a smaller reconstruction error as in Figure 5. Why then does Hurricane exhibit a lower compression ratio compared to CESM-ATM?\n\nThank you for this insightful question. The proposed algorithm achieves lossless compression on both CESM-ATM and Hurricane datasets since the reconstruction error is less than $10^{-7}$. The compression ratio is not directly related to reconstruction error. The ratio depends on two main factors: \n(a) The statistical properties of the source term $f$ after transformation. \n(b)The noise level of the original data. A sparser $f$ leads to better compression. Additionally, data with lower noise levels also leads to better compression.\nFrom this perspective, we can infer that CESM-ATM likely yields a more compressible $f$ than Hurricane data in terms of PDE representation, or that the CESM-ATM data contains lower levels of noise. \n\n> Q3: What are the challenges in applying this method to other domains (e.g., medical imaging, oceanography as mentioned in the Conclusion)?\n\nWe appreciate your valuable question. Our method has extended to medical imaging and oceanography, presenting both opportunities and challenges. In medical imaging analysis, the identification of high-order information (source term $f$) enables us to extract pathological features from images, such as detecting anomalies in OCTA scans. This capability arises from distinguishing between normal tissue mechanisms and abnormal external influences. However, interpreting these results requires expert guidance to correlate specific high-order information patterns with particular diseases. In oceanography, mechanism identification also plays a crucial role. By learning PDEs(perhaps higher-order ones), we can identify governing equations, which help us determine the impacts of diffusion and convection terms. Based on the identified models, we could predict the future evolution of oceanic systems. The main challenge here lies in the need for continuous observation to achieve accurate model characterization. For instance, robust ocean mechanism identification often requires data assimilation techniques.\n\n**Comparative Analysis with Other Methods**\n\nThank you for your suggestion to include comparisons with additional relevant methods. We carefully read the papers you provided and made the following adjustments to our study: We did not include ndzip in our comparative experiments because it is also based on the Lorenzo predictor, which is similar to FPZIP and it requires a parallel computing environment.\nWe have supplemented our study with comparative experiments involving: \n1) Floating-point compression algorithms: ALP and ZFP; \n2) General-purpose lossless compression algorithms: Blosc and Gzip.\n\nThe detailed results of these additional experiments can be found in the \"Additional Experiment Results\" section of the \"Author Rebuttal\".\n\n**More explanation on Mechanism Learning**\n\nThanks for your invaluable comment. We have provided a more comprehensive introduction to mechanism learning in the revised manuscript.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1oSNtPFqtw", "reviewer": "Reviewer_5CqK", "summary": "This paper introduces a novel approach combining high-order mechanism learning with classical encoding techniques to enhance lossless compression for large-scale scientific floating-point data. The core innovation lies in treating data as discrete samples from a physical field governed by differential equations and solving inverse problems to identify compressible coefficients. Experiments demonstrate MeLLoC's superior performance over existing methods, achieving better compression ratios and computational efficiency.", "questions": "1) Clarification on Differential Equation Models:\nCan you provide more details on how you determine the appropriate differential equation models for different datasets?\nHow sensitive is the performance of MeLLoC to the choice of these models?\n2) Precision Control:\nCould you elaborate on the process for calibrating the precision control?\nAre there any guidelines or heuristics you follow to balance the compression efficiency and computational cost?", "limitations": "The authors have recognized and addressed several limitations of their work:\n1) Model Dependency: The performance of MeLLoC depends on the accuracy of the differential equation models representing the data.\n2) Calibration Complexity: Precision control requires careful calibration, which can be challenging with diverse data characteristics.\n3) Computational Intensity: The method demands significant computational resources for large datasets.\n4) Generalization Issues: MeLLoC may not be as effective for datasets that do not align well with the assumed physical models.", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1) Innovative Approach: Combines mechanism learning with classical encoding, leveraging differential equations to compress scientific data effectively.\n2) Superior Performance: Outperforms state-of-the-art lossless compression techniques in terms of compression ratios and computational efficiency.\n3) Comprehensive Experiments: Extensive testing on various datasets highlights the robustness and efficacy of the proposed method.\n4) Flexibility: Capable of handling different types of scientific data, including those with high-order information and noise.", "weaknesses": "1) Model Dependency: Performance relies heavily on the accuracy of the differential equation models representing the data.\n2) Complex Calibration: Precision control requires careful calibration, which can be challenging with diverse data characteristics.\n3) Computational Intensity: Despite improved efficiency, the method still demands significant computational resources for large datasets.\n4) Generalization: May not be as effective for datasets that do not conform well to the assumed physical models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the overall positive feedback and the valuable comments. We have revised our manuscript taking your concerns and suggestions into consideration.\n\nTo answer your questions/comments:\n> Q1: Clarification on Differential Equation Models: Can you provide more details on how you determine the appropriate differential equation models for different datasets? How sensitive is the performance of MeLLoC to the choice of these models?\n\nThanks for your valuable comments. In our work, mechanism learning aims to discover the differential equation model that best describes the data. The identification of the mechanism is explained in the \"Model identification and its well-posedness\" section of the \"Author Rebuttal\". Please kindly refer to that. Through optimization, we obtain the optimal parameters $\\theta^*$ in Figure 1. These optimal parameters directly correspond to the coefficients in the second-order PDEs.\n\nWe also appreciate the reviewer's observation on sensitivity. Taking the CESM-ATM dataset as an example, when using fixed templates as shown in Figure 2(c), we observe the following compression ratios:\n1) Laplacian template: $2.67\\times$; \n\n2) Hyperbolic template: $3.29\\times$; \n\n3) Parabolic template: $1.45\\times$. \n\nAs shown by varying compression ratios, the performance is therefore sensitive to model choice. By MeLLoC, the model is optimized and the learned template approach consistently outperforms fixed templates, achieving 3.36x as shown in Table 1.\n\nIt also suggests that the underlying physical mechanisms in the CESM-ATM dataset align more closely with Hyperbolic Equations/Transportation Equations.  Such insights could potentially inform future modeling strategies and enhance our understanding of atmospheric dynamics in climate models.\n\n> Q2: Precision Control: Could you elaborate on the process for calibrating the precision control? Are there any guidelines or heuristics you follow to balance the compression efficiency and computational cost?\n\n We appreciate your question. Our method ensures lossless compression while optimizing computational efficiency through the following process:\n1) We observe that the source term $f = L(u) = \\sum_{i=1}^9 C_i u_i$ is computed with precision $10^{-(m+n)}$, where $m$ is the original data precision and $n$ is the model coefficient precision. We can maintain lossless while the solver for $K_\\mathcal{L}u^{in} = b_{u_{bd},f}$ has precision capacity of $10^{-(m+n)}$ (this is feasible with proper $n$). As $n$ increases, the admissible set for $C_i$ enlarges, contributing to a lower absolute value for $f$ but higher precision. The best compression ratio is reached when significant digits of $f$ reach a minimum. \n2) We optimize $n$ for coefficients $C_i$ by starting with high precision and gradually reducing it while monitoring reconstruction error and compression ratio. With several calibrations, $n$ can be fixed for the remaining dataset if the compression ratio for the following batches shows no significant fluctuation. \n3) The approach considers different scenarios to balance significant digits and value magnitudes, as shown in Figure 3(b). \n\nMeLLoC optimizes compression efficiency within lossless constraints, balancing perfect reconstruction with computational feasibility. The precision control is adaptive and can be tailored to different scientific datasets.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all the reviewers for their insightful comments and suggestions. Accordingly, we try our best to make substantial revisions. The revised article now contains the extensions of the proposed lossless compression framework, and the additional experiments on comparison studies. We have addressed the reviewers' questions in our respective responses.

### Additional Experiment Results{Reviewer XGH7,Reviewer VS2J}
During the rebuttal period, we have performed additional experiments as shown in Table 1 of the attached PDF, which will be the updated version of Table 1 of the main manuscript. The updated results demonstrate a comprehensive comparison between MeLLoC and several state-of-the-art lossless compression algorithms, including ALP, FPZIP, and ZFP, which are specifically designed for floating-point number compression. MeLLoC significantly outperforms general-purpose algorithms in terms of compression ratio. While its throughput may not match that of ALP or Blosc, it introduces a novel compression paradigm for scientific floating-point data. Figure 1 in the attached PDF visualizes these results, with the second graph showing Combined Performance (compression rate * log(speed)). MeLLoC demonstrates a clear advantage over other algorithms in this balanced metric, balancing high compression ratios with reasonable processing speeds.

### Differences between [Luo et al. (2024a)] and our work{Reviewer XGH7,Reviewer VS2J}
The method proposed by [Luo et al. (2024a)] is a near-lossless compression technique, as it incorporates noise separation. Consequently, it cannot achieve perfect reconstruction within the digit precision of the original data. In contrast, MeLLoC is a true lossless compressor. To achieve industry-standard high throughput, we have optimized MeLLoC's computational process based on second-order PDE theory. The well-posedness from PDE theory also ensures the feasibility of high-order data representation.

### Model identification and its well-posedness{Reviewer XGH7,Reviewer 5CqK}
Consider the underlying model of the data. For $u \in C^4(\Omega)$, $\Omega \subset \mathbb{R}^2 $,  the nine-point difference template is related to the second-order linear differential operator as
$$\sum_{k, l=-1}^1 C_{k, l} u(x+k h, y+l h) =  [2(c_1+c_5-c_6) h \partial_x+2(c_2+c_5+c_6) h \partial_y  +(c_3+\frac{1}{2} c_7+\frac{1}{2} c_8) h^2 \partial_{xx}^2+(c_4+\frac{1}{2} c_7+\frac{1}{2} c_8) h^2 \partial_{yy}^2 +(c_7-c_8) h^2 \partial_{xy}^2+c_9] u(x, y)+o(h^2).$$
where $C_{k,l}$ are the coefficients in Figure 2(a), the subscripts ${k,l}$ represent the relative position to the data point $(x,y)$.
The relationship between $C_{k,l}$ and $c_n$ can be represented as
$$\mathbf{C}=c_1 \mathbf{A}_1+c_2 \mathbf{A}_2+c_3 \mathbf{A}_3+c_4 \mathbf{A}_4+c_5 \mathbf{A}_5+c_6 \mathbf{A}_6+c_7 \mathbf{A}_7+c_8 \mathbf{A}_8+c_9 \mathbf{A}_9,$$

where $\mathbf{C}$ is the matrix of $C_{k,l}$, $\mathbf{A}_n$ are basis matrices, and $c_n$ are corresponding coefficients.
$\{\mathbf{A}_n\}$ are defined as
$$
\mathbf{A}_1 = [0,0,0;-1,0,1;0,0,0], \mathbf{A}_2 = [0,1,0;0,0,0;0,0,-1], 
\mathbf{A}_3 = [0,0,0;1,-2,1;0,0,0], $$
$$ \mathbf{A}_4 = [0,1,0;0,-2,0;0,1,0], \mathbf{A}_5 = [0,0,1;0,0,0;-1,0,0],  \mathbf{A}_6 = [1,0,0;0,0,0;0,0,-1], $$
$$
\mathbf{A}_7 = [0,0,1;0,-2,0;1,0,0],  \mathbf{A}_8 = [1,0,0;0,-2,0;0,0,1],
\mathbf{A}_9 = [0,0,0;0,1,0;0,0,0].$$

Based on this representation, encoding $u$ becomes encoding the sparser high-order term $o(h^2)$, i.e., the source term $f$. Therefore, the optimization objective is to obtain a minimized high-order term, which can be mathematically expressed as
$$\{C_{k,l}\}^{*} = argmin_{C_{k,l}} F({C_{k,l}};u) = argmin_{C_{k,l}} (\sum_{i, j}\sum_{k, l=-1}^1 C_{k, l} u(i+k h, j+l h))^2.$$

Once the template $\theta:=C_{k, l}$ is learned, one can calculate the coefficients of the differential operator, allowing one to classify the mechanism as elliptic, parabolic, or hyperbolic due to the reversibility of $C_{k, l}$ and $c_n$.

Next, we will briefly explain the solvability of the model identification problem. The well-posedness of the compression process is considered by the following formulation. The above minimization problem is equivalent to solving the least square problem: $$Ac = 0,$$

where $c = [C_1, \cdots, C_9]^T$, $A \in \mathbb{R}^{N\times 9}$, $N$ is the number of data points in domain $D$, $\mathcal{P}: D \to \mathbb{R}$, $k = \mathcal{P}(i,j)$, $(i,j) \in D$ is the index after rearranging the data into a one-dimensional vector, $(i,j) = \mathcal{P}^{-1}(k)$, $(k = 1, \cdots, N)$ is its inverse mapping.
$$A_{k,\cdot} = [u_{i-1,j-1}, \cdots, u_{i+1,j+1}], \quad (i,j) = \mathcal{P}^{-1}(k).$$

Therefore, to find non-trivial solutions is to obtain the null space (kernel) of $B = (A^TA)$. There is only a trivial solution if $B$ is rank full while otherwise there is no uniqueness. To address this issue, if we set the coefficient of $u_{i,j}$ to -1 (i.e., set $C_5$ to -1 in Figure 2 (a)), and fix the template size to 8, then the above problem becomes:
$$\tilde{A}\tilde{c} = b,$$

where $\tilde{c} \in \mathbb{R}^8$, $\tilde{A} \in \mathbb{R}^{N\times 8}$,
and $b = [u_{\mathcal{P}^{-1}(1)}, \cdots, u_{\mathcal{P}^{-1}(N)}]^T \in \mathbb{R}^N$. Then, the above problem has a unique least squares solution $\tilde{c} = \tilde{A}^\dagger b$, provided the data are not all degenerated. We assemble $\tilde{A}$ and directly solve the pseudo inverse, which serves as our fast solver for the compression process. 

Finally, we thank all the reviewers again for your insightful comments. We do believe that the revised article is much improved not only from the theoretical aspect but also from the experimental aspect. We are happy to answer any further questions the reviewer might have.

Meta-review: The reviewers had various questions and comments mainly related to differentiation relative to previous work by Luo 2024a as well as whether the target precision can be stated as lossless. The authors addressed every point raised. The authors also provided more experimental results as requested. Two reviewers, who took negative stances, were reluctant to change scores. One of these two reviewers had no counter argument. The other reviewer seemed to have only one issue left that had to do with the definition of the "lossless" property. Overall, the author rebuttals adequately addressed concerns raised. While likely to be effective for only certain types of data, the proposed method appears new and interesting.
Decision: Accept (poster)