Compressing multidimensional weather and climate data into neural networks
Authors: Langwen Huang
Torsten Hoefler
Venue: ICLR 2023 notable top 5% (2023)
Keywords: Weather and climate data
Compression

Abstract: Weather and climate simulations produce petabytes of high-resolution data that are later analyzed by researchers in order to understand climate change or severe weather. We propose a new method of compressing this multidimensional weather and climate data: a coordinate-based neural network is trained to overfit the data, and the resulting parameters are taken as a compact representation of the original grid-based data. While compression ratios range from 300x to more than 3,000x, our method outperforms the state-of-the-art compressor SZ3 in terms of weighted RMSE, MAE. It can faithfully preserve important large scale atmosphere structures and does not introduce significant artifacts.
When using the resulting neural network as a 790x compressed dataloader to train the WeatherBench forecasting model, its RMSE increases by less than 2%. The three orders of magnitude compression democratizes access to high-resolution climate data and enables numerous new research directions.

Published as a conference paper at ICLR 2023
COMPRESSING MULTIDIMENSIONAL WEATHER AND CLIMATE DATA INTO NEURAL NETWORKS
Langwen Huang
Torsten Hoefler
Department of Computer Science
Department of Computer Science ETH Zurich
ETH Zurich
langwen. huang@inf www.n.f..tt.cc...
torsten. hoefler@inf.ethhe. ch
ABSTRACT
Weather and climate simulations produce petabytes of high-resolution data that are later analyzed by researchers order to understand climate change or severe weather. We propose new method of compressing this multidimensional weather and climate data: a coordinate-bel
neural network trained to overfit the data, and the resulting parameters taken compact representation of the original grid-based data. While compression ratios range from 300x to more than 3,000x, method outperforms the state-of oaiee-of-thee af-ttheart compressor SZ3 in terms of weighted RMSE, MAE. It can faithfully preserve important large scale atmosphere struc- tures and does not introduce significant artifacts. When using the resulting neural network as 790x compressed dataloader to train the WeatherBench forecast- ing model, its RMSE increases by less than 2%. The three orders of magnitude compression democratizes access to high-resolution climate data and enables nu- merous new research directions.
INTRODUCTION
Numerical weather and climate simulations can produce hundreds of terabytes to several petabytes of data (Kay et al., 2015; Hersbach et al., 2020) and such data are growing even bigger as higher res- olution simulations are needed tackle climate change and associated extreme weather (Schulthess et al., 2019; Schär et al., 2019). In fact, kilometer-scale climate data are expected to be one of, if not the largest, scientific datasets worldwide the near future. Therefore is valuable to compress those data such that ever-growing supercomputers can perform more detailed simulations while end users can have faster access to them.
Data produced by numerical weather and climate simulations contains geophysical variables such as geopotential, temperature, and wind speed. They are usually stored as multidimensional arrays where each element represents variable evaluated ning space and time. Most compression methods (Ych et al., 2005; Lindstrom, 2014; Lalgudi et al., 2008; Liang et al., 2022; Ballester- al., 2018) blocks of data into compact representations that can later be decompressed back into the original format. This approach prohibits the flow of information between blocks. Thus larger blocks are required to achieve higher compression ratios, yet larger block sizes also lead to higher latency and lower bandwidth when only a subset of data is needed. Moreover, even with the largest possible block size, those methods cannot use all the information due to computation or memory limitations and cannot further improve compression ratio accuracy.
point in multidimensional grid span-
auto-encoder approach that compresses
We present new lossy compression method for weather and climate data by taking alternative view: we compress the data through training neural network to surrogate the geophysical variable as a continuous function mapping from space and time coordinates to scalar numbers (Figure 1). The input horizontal coordinates are transformed into three-dimensional Cartesian coordinates on the unit sphere where the distance between two points is monotonically related to their geodesic dis- tance, such that the periodic boundary conditions over the sphere are enforced strictly. The resulting Cartesian coordinates and other coordinates then transformed into Fourier features before flow- ing into fully- fully-connected feed forward layers so that the neural network can capture high- sigh-freuuency signals (Tancik al., 2020). After the neural network is trained, its weights quantized from

---

Published as a conference paper at ICLR 2023
(1p R1)
x b sin Φi
yi = cos i cos di
FCBlock
FCBlock d
2, = sin Y
Fourier Fe atures
Feed Forward
Batch Patch orrm Norm
Feed Forward
Batch Norm
Feed Forward
Batch Norm
Feed Forward
Batch Norm
f(to Uu de:
Pi)
Train
Quantize
(1v Py Ue 11)
MSE loss
0
Interpolate
Figure 1: Diagram of the neural network structure: Coordinates are transformed into 3D coordinates and fed to the Fourier feature layer (green). Then they flow into series of fully connected blocks (light blue) where each block consists of two feed forward layers (dark blue) with batch norms (orange) and a skip connection. Solid lines indicate flows of data in both compression (training) and decompression (inference) processes, and dash lines correspond to the compression-onl) process.
float32 into f to get an extra 2x gain in the compression ratio without losing much accuracy. This method allows compressing large data block and squeezes redundant information globally while still having low latency and high bandwidth when retrieving decompressed values. It per- forms well the high compression ratio regime: it can achieve 300x to more than 3,000x in com- pression ratio with better quality than the state-of-the-art SZ3 compressor (Liang et al., 2022) in terms of weighted root mean squared error (RMSE) and mean absolute error (MAE). The compres- sion results preserve large-scale atmosphere structures that are important for weather and climate predictions and do not introduce artifacts, unlike SZ3. The neural network produced by our method can be used as an 10-efficient dataloader for training machine learning models such as convolutional neural networks (CNNs) and only lead to a limited increase in test errors compared with training with original data. Although equipped with Fourier features, error analysis shows that our method poorly captures rarely occurring sharp signals such as hurricanes: it tends to smooth out extreme values inside the center hurricane. also suffers from slow training speed, especially in the low compression ratio regime.
Besides high compression ratios, our method provides extra desirable features for data analysis. Users can access any subset of the data with cost only proportional to the size of the subset because accessing values with different coordinates is independent and thus trivially parallelizable on modern computation devices such GPUs multi-core CPUs. In addition, the functional nature of the neural network provides "free interpolation" when accessing coordinates that do not match existing grid points. Both features are impossible impractical implement traditional compressors with a matching compression ratio.
1.1 RELATED WORK
Compression methods for multidimensional data Existing lossy compression methods com- press multidimensional data by transforming it into space with sparse representations after trun- cating, and then quantize and optionally entropy-encode the resulting sparse data. As one of the most successful methods in the area, SZ3 (Liang et al., 2022) finds sparse representations in the space of locally spanning splines. It provides an error-bounded compression method and can achieve a 400x compression ratio our test dataset. TTHRESH (Ballester-Rppoll Ripoll et al., 2018) compresses data by decomposing it into lower dimensional tensors. It performs well on isotropic data such as medical images and turbulence data with compression ratios of around 300x, but it is not the case for heavily stratified weather and climate data where it either fails to compress or results in poor compression ratios of around 3x. ZFP (Lindstrom, 2014) is another popular compression method that provides a fixed-bitrate mode by truncating blocks of orthogonally transformed data. It provides only low com- pression ratios that usually do not exceed 10x. While there are lossless methods for multidimensional data (Yeh et al., 2005), they cannot achieve compression ratios more than 2x because scientific data stored multidimensional floating point arrays rarely has repeated bit patterns (Zhao et al., 2020). SimFS (Girolamo al., 2019) is special lossless compression method that compresses the simu-
2

---

Published as a conference paper at ICLR 2023
Table 1: Comparison of methods for multidimensional compressing weather and climate data with the same RMSE bound. The decompression speed is measured in both continuous and random access patterns. SimFS' decompression speed depends heavily on the compression ratio.
Method
Comp. ratio
Comp. speed Decomp. speed Cont.
Rand.
SimFS (Girolamo et al., 2019)
Arbitrary
ZFP (Lindstrom, 2014)
10x
TTHRESH (Ballester-Ripp Ripoll et al., 2018)
300x
SZ3 (Liang et al., 2022)
400x
NN (Ours)
300x 3,000x
lation data by storing simulation checkpoints every N time steps and, whenever needed, recreating the discarded result by re-running the simulation starting from the nearest saved time step. It can reach arbitrary high compression ratios by simply discarding the respective amount of intermediate results, but reconstruction (decompression) may expensive because it always produces all the prognostic variables of simulation. Table summarizes the performance of the methods above when compressing weather and climate data.
Neural representation Using neural networks represent multidimensional objects is gaining popularity after the "spectral bias" of the vanilla MLP is resolved by Fourier features (Tancik et al., 2020) SIREN (Sitzmann al., 2020). Previous works of applying neural representation for compression focus on compressing 3D scenes (Mildenhall et al., 2021), images (Dupont et al., 2021), videos (Chen al., 2021). 3D volumetric data (Lu al., 2021), and isotropic fluid data (Pan et al., 2022). Instead of optimizing the compression ratio, it is also possible to optimize training time and learned details by adding a trainable embedding layer (Müller et al., 2022; Barron al., 2021). Our work differs because the target data is defined on manifold instead of the Euclidean space. Also, weather and climate data are typically larger, smoother, and much more anisotropic. COIN+ (Dupont et al., 2022) and MSCN (Schwarz& & Teh, 2022) applied meta-learning methods to compress time slices of climate data into latent vectors of several bits. The two methods can achieve an asympotic compression ratio of 3,000x assuming infinite number of time slices, yet the actual compression ratios are typically below 50x considering the size of neural network weights.
2 METHOD
A geophysical variable in weather and climate dataset can be seen as scalar function of four inputs [l, vi, Φ) with time I, pressure p, latitude 1 and longitude Φb Note that pressure used in metcorology the coordinate for vertical levels while the height of air parcel with constant pressure becomes a variable. We propose using a neural network with weights 0 as a compact representation of the function f(1, tù, d; 0). As Figure I shows, the neural network is trained with N random samples of the coordinates {[4,, Pi, vi, di)li E [1, N]} as inputs, and evaluations of the function [f(4., Pi, vh, i E []} at those coordinates as targets, and use the mean squared error (MSE) as the loss function. In practice, given a weather and climate data stored in a four- dimensional array, one can implement the function f(t, p, b Φ) by returning the interpolated value the input coordinate. Since the aim of the method is to "overfit" the neural network to the data, there is no need for generalization to unseen inputs. Therefore we use all the data available in the training process. After training, the weights O are quantized from float32 into float16 and then stored as the compressed representation of the original data. In the decompression process, the stored weights are cast back to float32 and loaded into the neural network along with coordinates of the data to produce decompressed values at those input coordinates. We remark that more complex quantization (Gholami et al., 2021) and sparsification (Hoefler et al., 2021) schemes could be used for further compressing of neural network parameters that we explain in the following.
3

---

Published as a conference paper at ICLR 2023
2.1 NEURAL NETWORK STRUCTURE
We the fully connected MLP with skip connections as the backbone of the neural network (Figure 1). It contains a sequence of fully connected blocks (FCBlocks), and each FCBlock contains two fully connected feed forward layers
Table Hyper-parameters
Symbol
Explanation
LR Cp
Learning rate Scaling factor for time Scaling factor for pressure Number of Fourier features
with batch norms and a skip connection in the end. We use the Gaussian Error Linear Unit (GELU) (Hendrycks & Gim- pel, 2016) as the activation function in feed forward layers to make the neural network smooth function. The widths of feed forward layers m and the number of FCBlocks d are configurable hyperpa- rameters (Table 2) that influence the compression ratio.
The standard deviation of Fourier features
d
Number of FCBlocks
W/s
Width of the i-th FCBlock
Firstly, the input coordinates are normalized and then transformed into Fourier features. During the normalization process, time and pressure coordinates are normalized by two constants: i= l/ce: p/cp. Latitude and longitude coordinates are transformed into 3D Cartesian XYZ coordinates on the unit sphere: x = (t) cos( y = cos(e) (b) sin()), = sin(v) such that two points close to each other on the sphere are also close each other on the XYZ coordinates. This transformation can enforce the periodic property along longitude strictly. Then, the five coordinates v [t. p. x. y, 2] are transformed into Fourier features 7(v) [cos B )oT"" sin(BrT)) where cos and sin are applied element-wise. B is blocked diagonal random matrix with 3m rows and 5 columns whose non-zero values are independently and randomly sampled from a Gaussian distribution with 0 mean and standard deviation a. The B matrix is blocked in way that there only interactions among x, y, and 2:
Bixx
B 3
B
B
Bmx3 ays
The values of the matrix B are fixed for neural network and setting it to trainable does not lead to a noticeable difference according to Tancik et al. (2020).
As weather and climate data is anisotropic, some geophysical variables may vary several magnitudes along the pressure and latitude dimensions. In that case, the neural network only captures variations along that dimension while ignoring variations along all other dimensions. Therefore, scale the neural network output to appropriate dynamic ranges at that coordinate. The scaling function can be computed by interpolating mean and range along pressure and latitude dimensions.
2.2 SAMPLING METHOD
As we train the neural network using a stochastic optimization method, it is important to feed the inputs randomly required by the stochastic optimization method to have a good convergence rate (Bottou, 2004). To achieve that, we randomly sample coordinates for latitude, longitude, and time within the data definition region. The target values on those coordinates are then interpolated such that coordinates are not restricted to grid point coordinates of the data. This makes our method independent from the input data grid and helps the neural network learn the underlying smooth function from the data instead of remembering values at each grid point and outputting meaningless values anywhere else. We use low discrepancy quasi-random generator (Owen, 1998) reduce the occurrence of closely clustered coordinates as well as reduce regions that are not sampled in each batch which also helps the neural network learn uniformly over the sampling region. addition, to avoid biased sampling over polar regions, the latitude & and longitude & are sampled according to the metrics on the sphere (Figure 1): &o = 3 cos (1-20 (1-2y), ø = 2 where (æ, y) is drawn from the low discrepancy quasi-random generator within range [0. 1] x [0, 1]. For pressure coordinates, we simply sample the exact grid points because they are usually very sparse with only tens of grid points, and interpolating on them introduces too much emror.
4

---

Published as a conference paper at ICLR 2023
3 EXPERIMENTAL EVALUATION
In this section, we test our method by varying model structure and data resolution, apply it to the training dataset of CNN, and compare the resulting test errors. The data used for experiments are extracted from the ERA5 dataset (Hersbach et al., 2020) and interpolated to regular longitude- latitude grids. When computing the mean of errois such as RMSE and MAE over the longitude- latitude grid, we assign weights to each grid point that are proportional to the cosine of latitudes to balance the denser polar regions and the thinner equator region (see Appendix A.I). Throughout the experiments, 128, and the standard deviation Fourier features a to 1.6. We use the Adam optimizer (Kingma & Ba, 2014) with learning rate of 3e-4. The neural networks are trained for 20 epochs where each has 1,046,437, ,220 randomly sampled coordinates (matching the total number of grid points in dataset 1) that are split into batches of 389,880 (matching the size of 3 horizontal slices in dataset 1) coordinates. All the experiments are performed on a single compute node with 3 NVIDIA RTX 3090 GPUs.
fix the number of FCBlocks d to 12, the number of Fourier features to
3.1 COMPRESSING DATA WITH DIFFERENT RESOLUTIONS
Table 3: Comparison of compressing data with different spatial and temporal resolutions.
Dataset
2
4
Variable
Geopotential
Time period
2016
2016
1979-2018
1979-2018
Spatial res.
0.50
0.25
5.625
2.8125
Temporal res.
24h
24h
1h
1h
Pressure levels
11
11
Data shape
(366,
(366.11,721,144)
(350640.1,32,64)
(350640,1,641128))
Data size
3.9GB
15.6GB
2.7GB
10.7GB
We test the performance of our method by compressing geopotential data from the ERAS dataset with various sizes and resolutions (Table 3). Geopotential selected because it the most important variable for weather and climate predictions (Rasp et al., 2020). We also compress the temperature data with the same formats with resulting error plots in Appendix A.4. With datasets and we test the performance for compressing 4D high spatial resolution data following the settings in Grönquist et al. (2021). Both datasets have 11 pressure levels which are meteorologically important (Ashkboos al., 2022). The year of 2016 is selected arbitrarily. We expect similar result when selecting different year as shown in the етгоr plots of compressing datasets with four different years (1998, 2004, 2010, 2016) in Appendix A.3. On the other hand, we test the performance for compressing high temporal resolution data on the 500hPa pressure level with datasets and 4. Datasets 3 and 4 also can be used for training machine learning models for weather prediction under the Weather- Bench (Rasp et al., 2020) framework. Note that 0.25 is the highest spatial resolution and hour is the highest temporal resolution available in the ERA5 dataset.
Table 4: Compression ratios of our method under different datasets and widths.
Dataset 3
Width
Model size
2
512
13.86MB
287.92
1,150.07
197.58
790.32
256
3,8 IMB
1,046.91
4.181.83
718.43
2,873.73
128
1.15MB
3,482.97
13,912.56
2.390.16
9,560.64
64
0,40MB
9,913.17
39,597.75
6,802.84
27,211.37
32
0.18MB
22,372.67
89,366.74
15,353.09
61,412.36
We use neural networks with different widths from 32 512 create spectrum of compression ratios (Table 4). Because larger models take more than 20 hours to train on our devices which not practical, only test method on the high compression ratio regime (>200x). As
5

---

Published as a conference paper at ICLR 2023
comparison, compress the same data using SZ3 (Liang al., 2022) with max absolute error tolerances from 10 to 2,000 and resulting compression ratios ranging from 10x to 700x. Addition- ally, We tested ZFP (Lindstrom, 2014) and TTHRESH (Ballester-Ripe Ripoll et al., 2018) but they cannot reach compression ratios larger than 10x and thus are not included the following analysis. For each compression result, we compute weighted RMSE, weighted MAE, max absolute error, and 0.99999-quantile of absolute error. The error is measured in the unit of m² m²/s² (the same as geopo- tential). We also analyze error distribution and perform case study for outputs on Oct 5th 2016 when hurricane Matthew emerged over the Caribbean Sea.
Weighted
Weighted
1,000
750
Dataset
in geopotentaal (mi/s")
500
250
250
0-
0-
0.99999 quantle of absolutee error
Max absolute error
8,000
7,500
Compression method
4,000
5,000
NN
2,500
10
100
1,000
10.000
100,000
100
10000
10,000
100,000
Compression ratio
Figure 2: Weighted RMSE (upper left), weighted MAE (upper right), 0. 0.99999-quantits of absolute error (lower left), and max absolute error (lower right) between the compressed and original data.
Error analysis As shown in Figure 2, our method can achieve at least 10 times more compression ratios with the same weighted RMSE and MAE compared to S73. The gap between the methods even larger in higher error regions where our method can achieve more than 10,000x while $Z3 capped at 600x. Comparing the results between datasets and 2, 3 and 4 where the only spatial resolution differs, our method gets free 4x gain in compression ratio without hurting any of the error measurements. This indicates our method can capture the real information underneath the continuous grid data. However, in terms of max absolute error, it is worse than SZ3 even with similar compression ratios due different error distributions, as shown in Figure 3 (left). The majority of emros in our method concentrate near O, but there exists small fraction that is relatively large. That small fraction around 0.001 % according to the 0.99999-quu -uanntlle of absolute error Figure where the two methods have similar errors at similar compression ratios. Mcanwhile, SZ3 distributes errors more evenly within the error tolerance. In terms of the geographic distribution of errors, we calculate the mean and standard deviation of errors over time and pressure levels from compressed dataset 2 using our method (Figure right). The mean values of errors of approximately -10 are distributed evenly the globe with exceptions of errors shows meridional difference where it grows from the equator to polar regions.
mountainous areas. The standard deviation
Case study Geopotential at 500hPa is one of the most important variables for weather prediction. Thus we select from Dataset 2 the 500hPa slice on Oct 5th 2016 when hurricane Matthew emerged over the Caribbean Sea and compare the compression results of the two methods with the original data. We use the biggest model with compression ratio of 1,150x for our approach. For $733, we select the one with absolute error tolerance of 1,000 and a compression ratio of 358x to match the 0.99t99-quantile of absolute error.
In the global plot (Figure 4), our method clearly preserves the original data better despite having much higher compression ratio: locations and shapes of minimum and maximum areas are nearly the 0" longitude line. The
identical to the original ones, and the data is perfectly continuous only noticeable difference is that some high- figh-frequency details are smoothed out. On the SZ3 side, the average value the equator region deviates from the original data, the structure of minimum

---

Published as a conference paper at ICLR 2023
Moan errors
120200 m wwwpdadddddd 10122
(358x)
1200
JaN 20-2
100
10153
gespatential 3,000
Figure 3: Left: Histogram of compression errors when compressing dataset 2 using method with a width of 512 and SZ3 with an absolute error tolerance of 1000. Right: mean and standard deviation of compression errors of our method.
and maximum areas is broken due to numerous artifacts, and there is discontinuity crossing the 0 longitude line. Moreover, we can easily identify from data compressed by our method two subtrop- ical highs over the North Atlantic and the North Pacific with slightly smoothed shapes compared to original. The two subtropical highs are important for predictions of precipitations and tracks of tropical cyclones (hurricanes/yphhonns) But from the SZ3 compressed data, one over the Pacific while the other breaks into small pieces, which is metcorologically incorrect.
only identify
Reference (1x)
NN (1.150k)
S23 (358x)
46,000
48,000 50,000
$2,000
$4,000
56,000
58,000
Geopotentiall (Imt/
Figure 4: Global plots of geopotential 500hPa on Oct 5th 2016 from original dataset 2 (left), compressed data using our method with a width of 512 (center), and SZ3 compressed data with absolute error tolerance of 1,000 (right). See Appendix A.2 for plots with varying compression ratios.
closer look at the hurricane region (Figure 5), our method failed to capture the extreme values in the center of the hurricane although a smoothed version of the surrounding structure is preserved. SZ3 better preserves extreme value in the hurricane center but largely deviates from the original in other regions, As the error histogram in Figure 3 shows, the errors of our method in the center of the hurricane fall into the "tail" the distribution that rarely occurs. Considering the hurricane center just few pixels wide, method can provide lot more details with fewer data compared with widely used lower resolution data that average out those extreme value regions.
3.2 COMPRESSING CNN TRAINING DATA
Training predictive machine learning models on large scientific datasets such as weather and climate data is often I/O bound (Dryden et al., 2021). Our method of compressing the dataset can potentially accelerate the training process by replacing the traditional dataloader with a trained neural network that generates training data demand. Yet, it is unelear how point-wise error estimates such as RMSE or MAE of the training data relate to changes test enror of the trained neural network. Here, apply method to the input data of CNN to show its effectiveness in this scenario.

---

Published as a conference paper at ICLR 2023
Reference (1x)
NN (1.150x)
523 (358x)
$6,000
57,
57,500
Geopotential (m/sy")
Figure 5: Local plots of geopotential at 500hPa on Oct 5th 2016 with hurricane Matthew in the center. The plots use data from original dataset 2 (left), compressed data using our method with width of 512 (center), and SZ3 compressed data with an absolute error tolerance of 1,000 (right). See Appendix A.2 for plots with varying compression ratios.
We choose the WeatherBench (Rasp al., 2020) baseline model the CNN. We use geopotential at 500hPa in datasets and 4 (Table 3) and temperature at 850hPa with the same range and shape. We select the biggest available neural network with width of 512 to compress the geopotential and temperature data, respectively. For SZ3, we set the absolute error tolerances to 900 m² /s² for geopotential and 5.0 "C for temperature to match the 0.99999-quantile of absolute error with our method. The outcomes are measured from the weighted RMSE error of CNN calculated using the test set.
Table 5: Weighted RMSE test error of CNN trained with original dataset and compressed dataset. C.R. represents compression ratio.
C.R.
Weighted RMSE error (test set)
Dataset 3
Geopotential at 500 hPa (m²/s²) Temperature at 850 hPa (?C)
Original
632.9
2.906
NN compressed
198x
637.3 (+0.7%)
2.944 (+1.3%)
SZ3 compressed
71x
650.6 (+2.8%)
2.985 (+2.7%)
Dataset 4
Original
688.8
2.834
NN compressed 790x
697.3 (+ (+1.2%)
2.888 (+1.9%)
SZ3 compressed
106x 702.9 (+2.0%)
2.887 (+1.9%
Results The results in Table 5 shows that our method leads to small increases in weighted RMSE test errors of not more than 1.9% while achieving high compression ratios of 198x and 790x for dataset 3 and 4. On the other hand, SZ3 results in larger test errors and lower compression ratios of 71x and 106x probably due to the additional artifacts observed in Figure 4 and 5. The high compression ratio and limited increase in test errors of our method make it possible to feed machine learning models with much more data which will eventually surpass the model trained with original data in performance.
3.3 ABLATION STUDY
We perform an ablation study by selectively disabling some features of the model and comparing the resulting weighted RMSE loss between compressed data and original data. We select output scaling, XYZ transform, Fourier feature, skip connection, and batch norm as the targets because they are the special parts added the fully connected MLP to compress data effectively. We also experiment changing the activation function from GELU to ReLU. We choose dataset in Table as the inputs and neural network with width of 512 which results in compression ratio of 288x.
8

---

Published as a conference paper at ICLR 2023
Table 6: Weighted RMSE between neural network compressed data and original data. In each experiment, parts of the model are disabled or replaced.Scaling means output scaling, XYZ means XYZ transform.
Scaling GELU XYZ Fourier feature Skip connection Batch norm
WRMSE
818
2
V
420
V
395
4
V
347
V
346
6
V
303
V
292
8
254
9
158
10
78
11
76
Results As Table shows, all of the features combined resulted the lowest error. Output scaling, Fourier feature, skip connections and batch norm can indeed reduce the compression error compared with experiments without them. The GELU activation function can help reduce the error compared with ReLU when combined with output scaling and skip connection. The XYZ transform part does not have much impact on the final result, however, it is important to enforce the physically correct hard constraint on the boundary condition of output data.
LIMITATIONS AND FUTURE WORKS
Despite having high compression ratios and relatively low error, method suffers from long com- pression/training time. takes approximately hours to train the network on dataset 2 (Table 3) with 4 NVIDIA RTX 3090 GPUs, whereas SZ3 takes just several minutes to compress with 32 CPU cores. The slow training process also limits us from using bigger models to get lower errors. Our method also struggles capture small-scale extreme values as discussed in Section 3.1. While error analysis shows that it rarely occurs, such events can relate to very important weather processes like hurricanes.
We are going to investigate potential ways to integrate techniques like trainable embeddings (Müller al., 2022; Barron al., 2021) and meta-learning (Finn al., 2017: Nichol & Schulman, 2018) to accelerate training and reduce extreme errors. In addition, we will expand our method to compress broader weather and climate data such satellite images and weather station measurements into neural networks hoping to act as a complement to the traditional data assimilation in the numerical weather prediction pipeline (Bonavita et al., 2016).
CONCLUSION
We present a new method of compressing multidimensional weather and climate data into neural network weights. We test our method subsets of the ERA5 dataset with different resolutions, and compare the results with the best available compressor SZ3 (Liang et al., 2022). Our method can achieve less weighted RMSE and MAE and similar -.uanle-quantile of absolute errors compared with SZ3 at compression ratios from 300x to 3,000x. While its max absolute error is worse than SZ3, the high emror shown be rare occurring (less than 0.001%). Through the case study the hurricane Matthew event, our method can reproduce high-quality data that preserve the large- scale structure compression ratio of 1,100x, but it struggled capture the extreme values in the hurricane center. It is also effective in replacing neural networking training data. Considering the compression ratio can be projected 3,000x even more on high-resolution data, our method has the potential to compress the whole ECMWF archive with hundreds of petabytes into tens of terabytes of data, fitting on a single hard drive, that can drive machine learning and other scientific applications.

---

Published as a conference paper at ICLR 2023
Reproducibility Statement We provide the source code implementing method well as instructions to reproduce the experiments Section 3. The data we used is also publicly avail- able through the download script in the source code. The source code is available in https: /
//github. www./o/w//aacacccccmcm /rge
Ethics Statement Our work can help reduce power consumption due to reduced data movement whenever loading the compressed data. It can make more weather and climate data easily available, which helps people better understand and mitigate climate change and severe weather.
This project received EuroHPC-JU funding under grant MAELSTROM, No. 955513. We thank the Swiss National Supercomputing Center (CSCS) for providing computing resources.
Acknowledgements
REFERENCES
Saleh Ashkboos, Langwen Huang, Nikoli Dryden, Tal Ben-Nun, Peter Dueben, Lukas Gianinazzi, Luca Kummer, and Torsten Hoefler. Ens-10 10: A dataset for post-processing ensemble weather forecast. arXiv preprint arXiv:2206.1477 2022.
Rafael Ballester-Ripoll, Peter Lindstrom, and Renato Pajarola. Tthresh: Tensor com-
pression for multidimensional visual data. 6 2018. doi: 10.1 10.1109/VCG.2019. 2904063. URL http: 1109/TVCCC 2019. 2904063
/ / /ttp/vaxivvv//////////e////aaav/ org/ hatpaaass/////// htt.://....552ttppp//. http://xx.doo..o.. wrg/I.....
Jonathan T Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualle and Pratul P Srinivasan. Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 5855-5864, 2021.
Massimo Bonavita, Elias Hólm, Lars Isaksen, and Mike Fisher. The evolution of the ecmwf hybrid data ussimilation system. Quarterly Journal of the Royal Meteorological Society, 182(694):287-- 303, 2016.
Léon Bottou. Stochastic learning. In Olivier Bousquet and Ulrike von Luxburg (eds.), Ad- vanced Lectures on Machine Learning. Lecture Notes in Artificial Intelligence, LNAI 3176, pp. 146-168. Springer Verlag, Berlin, 2004. URL http: //leon wottou. wwtp///g/p............
Sottou-m1sss200-
Hao Chen, Bo He, Hanyu Wang, Yixuan Ren, Ser Nam Lim, and Abhinav Shrivastava. Nerv: Neu- ral representations for videos. Advances in Neural Information Processing Systems, 34:21557- 21568, 2021.
Nikoli Dryden, Roman Böhringer, Tal Ben-Nun, and Torsten Hoefler. Clairvoyant Prefetching for Distributed Machine Learning I/O. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC21). ACM, 11 2021.
Emilien Dupont, Adam Goliñski, Milad Alizadeh, Yee Whye Teh, and Arnaud Doucet. Coin: Com- pression with implicit neural representations. arXiv preprint arXiv:2103.03123, 2021.
Emilien Dupont, Hrushikesh Loya, Milad Alizadeh, Adam Golinski, Y Whye Tch, and Arnaud Doucet. Coin++: Neural compression across modalities. Transactions on Machine Learning Research, 2022(11), 2022.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning, pp. 1126-1135. PMLR, 2017.
Amir Gholami, Sehoon Kim, Zhen Dong, Zhewei Yao, Michael W. Mahoney, and Kurt Keutzer. A survey of quantization methods for efficient neural network inference, 2021. URL https: //arxi www.iii..rrabbb//1111/ 13630.
10

---

Published as a conference paper at ICLR 2023
Salvatore Di Girolamo, Pirmin Schmid, Thomas Schulthess, and Torsten Hoefler. Simfs: A sim- ulation data virtualizing file system interface. 2019. doi: 00448550/axxxx1192.005555 URL
https: aariiv org/ aas//abs//// 1902. 03154.
Peter Grönquist, Chengyuan Yao, Tal Ben-Nun, Nikoli Dryden, Peter Dueben, Shigang Li, and Torsten Hoefler. Deep learning for post-processing ensemble weather forecasts. Philosophical Transactions of the Royal Society A, 379(2194)202200922 2021.
Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus). arXiv preprint
arXiv:1606.0844 2016.
Hans Hersbach, Bill Bell, Paul Berrisford, Shoji Hirahara, András Horányi, Joaquín Muñoz-Sabatee
Julien Nicolas, Carole Peubey, Raluca Radu, Dinand Schepers, et The era5 global reanalysis. Quarterly Journal of the Royal Meteorological Society, (197933):199920909 2020.
Torsten Hoefler, Dan Alistarh, Tal Ben-Nun, Nikoli Dryden, and Alexandra Peste. Sparsity in deep learning: Pruning and growth for efficient inference and training in neural networks, 2021. URL.
https: httpaaaxivvv..///ee/
org/ abs 2202.21222 00554.
Jennifer E Kay, Clara Deser, A Phillips, A Mai, Cecile Hannay, Gary Strand, Julie Michelle Ar- blaster, SC Bates, Gokhan Danabasoglu, James Edwards, et al. The community earth system model (cesm) large ensemble project: A community resource for studying climate change in the presence of internal climate variability. Bulletin of the American Meteorological Society, 96(8): 1333-1349, 2015.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 14X2v644/2.6980 2014.
H.G. Lalgudi, Ali Bilgin, Michael Marcellin, and Mariappan Nadar. Compression of multidimen- sional images using jpeg2000. Signal Processing Letters, IEEE, 15:393 396, 02 2008. doi:
10.1109/LSP.20088.997
Xin Liang, Kai Zhao, Sheng Di, Sihuan Li, Robert Underwood, Ali M Gok, Jiannan Tian, Junjing Deng, Jon Calhoun, Dingwen Tao, et al. Sz3: A modular framework for composing prediction- based error-bounded lossy compressors. IEEE Transactions on Big Data, 2022.
Peter Lindstrom. Fixed-rate compressed floating-point arrays. IEEE Transactions Visualization and Computer Graphics, 20, 08 2014. doi: 10. 109//TVCG..211223445555
Yuzhe Lu, Kairong Jiang, Joshua A. Levine, and Matthew Berger. Compressive neural rep- resentations of volumetric scalar fields. 4 2021. doi: 10448550axxi.21044.0333.. URL http:://rrx org/ /20oo//412445....
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes neural radiance fields for view synthesis. Communications of the ACM, 65(1):99-106, 2021.
Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics prim- itives with multiresolution hash encoding. 1 2022. doi: 10.485500/rxi.2221..00 URL
https:/ / arxiv..... org/ org/abs//////ooo.oooo.oo//...o.p 2200. 05989.
Alex Nichol and John Schulman. Reptile: scalable metalearning algorithm. arXiv: 02999, 2(3):4, 2018.
arXiv preprint
Art Owen. Scrambling lobol'and niederreiter-xing points. Journal of complexity, 14(4):466448, 1998.
Shaowu Pan, Steven I.. Brunton, and J. Nathan Kutz. Neural implicit flow: mesh-agnostic dimen- sionality reduction paradigm of spatio-temporal data. 2022. doi: 10.48550/ariv.220440216...
URL https:://///..m /tt////aaa /waaaiivvvoooo org/ abs /200/2204 03216.
Stephan Rasp, Peter D Dueben, Sebastian Scher, Jonathan A Weyn, Soukayna Mouatadid, and Nils Thuerey. Weatherbench: a benchmark data set for data-driven weather forecasting. Journal of Advances in Modeling Earth Systems, 12(1 )>e20200S000200,, 2020.

---

Published as a conference paper at ICLR 2023
Thomas Schulthess, P. Bauer, Oliver Fuhrer, Torsten Hoefler, C. Schaer, and N. Wedi. Reflect- ing on the goal and baseline for exascale computing: a roadmap based on weather and climate simulations. Computing in Science and Engineering (CiSE), 21(1), Jan. 2019. ISSN 1521-9615.
Jonathan Richard Schwarz and Yee Whye Teh. Meta-learning sparse compression networks. arXiv preprint FrXiv:2205.08955 2022.
Christoph Schär, Oliver Fuhrer, Andrea Arteaga, Nikolina Ban, Christophe Charpilloz, Salvatore Di Girolamo, Laureline Hentgen, Torsten Hoefler, Xavier Lapillonne, David Leutwyler, Katherine Osterried, Davide Panosetti, Stefan Rüdisühli, Linda Schlemmer, Thomas Schulthess, Michael Sprenger, Stefano Ubbiali, and Heini Wernli. Kilometer-scale climate models: Prospects and challenges. Bulletin of the American Meteorological Society, 100(12), Dec. 2019. Early Online Release.
Vincent Sitzmann, Julien N. P. Martel, Alexander W. Bergman, David B. Lindell, and Gordon Wetzstein. Implicit neural representations with periodic activation functions. 6 2020. doi:
10448550kmxx..220000666 URL /ttt:://axxi....o arxiv. httpabbb../...5555
Matthew Tancik, Pratul P. Srinivasan, Ben Mildenhall, Sara Fridovich-Kend Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan Barron, and Ren Ng. Fourier features let net- works learn high frequency functions in low dimensional domains. 6 2020. doi: 10.48550/arxiv. 2006.10739. URL httttp:/:atii...tt////t
www.a. abs 10739.
Pen-Shu Yeh, Philippe Armbruster, Aaron Kiely, Bart Masschelein, Gilles Moury, Christoph Schae- fer, and Carole Thiebaut. The new cesds image compression recommendation. In 2005 IEEE Aerospace Conference, pp. 4138-4145. IEEE, 2005.
Kai Zhao, Sheng Di, Xin Lian, Sihuan Li, Dingwen Tao, Julie Bessac, Zizhong Chen, and Franck Cappello. Sdrbench: Scientific data reduction benchmark for lossy compressors. In 2020 IEEE International Conference Big Data (Big Data), pp. 2716-2724. IEEE, 2020.
12

---

Published as a conference paper at ICLR 2023
A APPENDIX
A.1 WRIGHTS FOR RMSE AND MAE ON THE REGULAR LONGITUDE-LATTOON GRID
Both RMSE and MAE in this paper need to compute the average over the horizontal dimensions. The pointwise squared and absolute error can be seen as a function f(b,, +) mapping from latitude v) longitude Φ to scalar as the error that location. Ideally, the mean of the pointwise enror the sphere Ω2 is defined as:
[ f(e, o)ds/ / ids
(1)
Both integrations can be decomposed into sums of integrations over individual cells Ry in the regular longitude-latitude grid:
E,1,, f(v.. f(v.))ds) i. Riy
Σ Si
(2)
where denotes the cell index along the latitude dimension, denotes the index along the longitude dimension, Sij means the area cell Pey. By approximating integrals using the product cell
center value and cell areas, we have:
Eff(o
Σ Siz
(3)
2,5
If the grid size is large enough (which is the case for datasets in Table 3), the cell area can be further simplified:
Siy cos(C)) (vi)A
where r is the radius of the sphere, Aú and Ao the height and length of each cell. Canceling out r, A and á as they are constant values the regular longitude-latitude grid, the formula (1) becomes:
Σ f(er Ø,) cos( (1xs) Σ cos (p):) 1.7
(5)
1
can be seen as the sum of pointwise errors f(v,, (by) weighted by normalized cosine of latitudes cos( / E, cos()))).
A.2 PLOTS OF GEOPOTENTIAL ON OCT 5TH 2016
In this section, we add extra plots complement the case study Section 3.1.
CR=4,181
width-128 13-133
width-64 39.597
width CR-89.366
Geopotental (m/)a")
Figure 6: Global plots of geopotential at 500hPa on Oct 5th 2016 from compressed dataset using method with range of widths. CR means compression ratio.
13

---

Published as a conference paper at ICLR 2023
width-512
120 Ch-12,912
CR.39,597
CR-89.366
Figure 7: Local plots of geopotential at 500hPa on Oct 5th 2016 with hurricane Matthew in the center from compressed dataset using our method with a range widths. CR means compression ratio.
14

---

Published as a conference paper at ICLR 2023
abs CR-41
abs 101=100 CR=68
abs 200 10=
abs tol= CR=178
abs, 750 CB-
abs tol-1,000 CR- 358 CR-358
abs, 1o1-1,250 CR-510
abs_tol 1ol-1,500 CR-524
abs, 1o002,00 494
46,000
48,000
50,000
$2,000
54,000
56,000
58,000
Geopotentla Im?
Figure 8: Global plots of geopotential at 500hPa on Oct 5th 2016 from SZ3 compressed dataset with range of absolute error tolerances. CR means compression ratio.
15

---

Published as a conference paper at ICLR 2023
tol=10
toi-50
101
1o1-400
abs Jol--0- CR-287
116-1.0-0000 1,000 CR-156
abs Jol_), CR-510
abs 1,500 CR-524
abs, 200000 4R-494
Figure 9: Local plots of geopotential at 500hPa on Oct 5th 2016 with hurricane Matthew in the center from SZ3 compressed dataset 2 with a range of absolute error tolerances. CR means compression ratio.
16

---

Published as a conference paper at ICLR 2023
A.3
ERROR PLOTS FOR COMPRESSION EXPERIMENTS WITH DIFFERENT YEARS
Weighted RMSE
Weighted
1,000
750
750
500
Error geopolential (mf/s²
500
Dataset
250
0.99999 quantile of absolute error
Max absolute emror
6,000
Compression method
NN
4,000
523
2,000
2.500
0
10
1,000
10,000
100,000
1.0
100
1,000
10,000
100.000
Compression ratio
Figure 10: Weighted RMSE (upper left), weighted MAE (upper right), 0.99999-quantile of absolute error (lower left), and max absolute error (lower right) between the compressed and original geopo- tential data. Similar datasets with four different years (1998, 2004, 2010, 2016) are compressed.
A.4 ERROR PLOOTS FOR COMPRESSION EXPERIMENTS WITH TEMPERATURE DATA
Weighted
Weighted
Dataset
Erorr in temperature (K)
0-
0-
0.99999 quantle of absolute error
ax absolute error
20
30
Compression method
15
20
NN
10
S23
5
10
44
10
100
1,000
10.000
100,000
10
100
1,000
10,000
100,000
Compression ratio
Figure 11: Weighted RMSE (upper left), weighted MAE (upper right). Q.9999e- of absolute error (lower left), and max absolute error (lower right) between the compressed and original tem- perature data.
17


Reviews:
[{"review_id": "RAmyJVJgpy-", "reviewer": "Reviewer_5Wg4", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a compression method for weather and climate worldwide data using a deep learning model combined with a Fourier transformation in order to enforce periodicity of the sphere data. The compression ratio is very high, from 300 to 3000, and the residual error comparable to methods with compression ratios drastically lower. However, the computation time is still high and high frequency features, such as hurricanes, are not always reconstructed. The tests were made on ERA5 reanalysis data, which is the open source widely used meteorological data set. ", "summary_of_the_review": "Though I am not a specialist in compressing data, I did use climate datasets for machine learning applications and I think this paper is worth publishing.", "strength_and_weaknesses": "Strengths: \n- compression ratio (300 to 3000)\n- spherical transformation, which is indeed very important for earth data\n- usefulness of the method: while at first I was skeptical on the usefulness of such an approach for climate data, as clearly ERA5 will never be only stored in a degraded version, I was convinced by the usage of machine learning applications (and probably also others), where these amount of data are usually untractable for a local storage+usage.\n- comparison with state-of-the-art and on different settings\n- current limits of the method clearly analyzed\n\nSome clarifications would be valuable for the paper, and in particular the following points:\n- I think that in order to be used, this kind of approach would need to have an uncertainty quantification, at least based on the frequency of the signal to recover,  or based on the type of data, etc. in order for the user to be able to know when it can trust it and when in can't. How would you do it? \n- The compression consists in training a neural network. Do you think this is robust enough to work with all size, input field, etc.? Do you think this could be used by a non-machine learning expert?\n- In which type of applications would your error be negligible? It is good to have a very high compression, but the final quality needs to be satisfactory for targeted applications at least (since the hurricanes are for example not the case).\n- The compressed data representation are the neural network weight values directly. Could your compression be used directly to feed a machine learning model, or would users have to decompress the data before using it?\n- Would your method, and in particular the first part about the coordinate transformation, work in the case of non-worldwide data, i.e. only a part of the globe?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "3ARCD3QLh4", "reviewer": "Reviewer_Kcey", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The  authors proposes  their compression approach for multidimensional weather data leveraging Neural Networks. The authors argue that this could provide compression ratios from 300x to 3,000x which is quite important in complex high resolution data regime. They evaluate this on the ERA5 weather dataset benchmarked against the SZ3 baseline. The author's experiment on different network architectures and  provide a case study on a specific hurricane event.", "summary_of_the_review": "Overall, the work proposes an reasonably novel technique in an important domain. The authors conduct sufficient empirical evaluation. However, there are details which are not described clearly along with the technical novelty of the Neural network model.", "strength_and_weaknesses": "Strength:\n* The empirical evaluation and ablation studies are reasonably well detailed, showcasing how this is applied in real world scenarios.\n* The paper is well organized, motivated and details the related work.\n\nWeakness:\n* Some of the important details are quite unclear.\n     *It's quite unclear on what the described \"target\" scalar value is. How is the ground truth information sourced from? \n     * Why are the Fourier features used?\n     * Perhaps, I'm missing an obvious detail in information theory. How is the precision recovered during decompression  when converting model weights from float16 to float32?\n* Are the RMSE, RMAE metrics over grids the best choice of metrics? Is there a metric that compares blurriness of the decompressed output with the ground truth?\n* The technical novelty of the introduced Neural Network itself isn't quite significant.\n\n       \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "0xWJ9yTq86N", "reviewer": "Reviewer_U9H2", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper explores encoding high-resolution weather data using implicit neural representation (INR). In INR, a neural network takes as input a coordinate (here, transformed latitute, longitude, time and pressure) and output the compressed values (weather data here) at that coordinates. The approach is compared to several baselines and evaluated both in terms of compression error and in terms of performance of a model trained on the compressed data. The proposed approach significantly outperform the baselines. The paper also contains an ablation study.", "summary_of_the_review": "For me this paper describes an interesting application, could have widespread use to compress weather data and some limited novelty on the machine learning side, so I find the content interesting. However, the limited scope of the experiments makes it impossible to know how it will behave on larger data sets such as all historical weather data.", "strength_and_weaknesses": "On the plus side,\n* Weather data is an interesting application. The need for compression is also well motivated in the paper.\n* I like the that the authors explain the specificities of weather data and how it impact their model.\n* Compression is evaluated both in terms of reconstruction error and accuracy of a neural network trained on decompressed data. Results are good.\n\nOn the negative side,\n* Experiments are a bit limited in scope. Only four data sets and one weather variable are considered. Unless I am mistaken, this means only four encoding were done. I appreciate that the data sets of different characteristics and I am aware encoding time is long. The largest time frame considered is 24 hours. The results are good, but they did not convince me the proposed approach could extended to years of historical data, an application mentioned by the authors. I think discussing this point would be interesting. \n* Another application mentioned by the author is to compress simulation data. It would be interesting to compare compression time with simulation time.\n* The use of GELU is not really motivated and not included in the ablation study. \n\n-----------\n**Update**\nI had misunderstood the scope of the experiments. The authors pointed this out and were kind enough to add additional experiments. They also answered the other points I raised. I believe this work is worth publishing.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: notable-top-5%