Original: N EURAL DATA COMPRESSION FOR PHYSICS PLASMA SIMULATION
Revised: N EURAL DATA COMPRESSION FOR PHYSICS PLASMA SIMULATION

Original: A BSTRACT
Revised: A BSTRACT

Original: We present a VAE-based data compression method, called VAe Physics Optimized
Revised: We present a VAE-based data compression method, called VAe Physics Optimized

Original: Reduction (VAPOR), to compress scientiﬁc data while preserving physics constraints.
Revised: Reduction (VAPOR), to compress scientiﬁc data while preserving physics constraints.

Original: VAPOR is based on Vector Quantized Variational Auto Encoder (VQVAE) and extended with physics-informed optimization functions and reﬁnement layers, focusing on compressing and reconstructing scientiﬁc data with minimum loss of information under physics constraints.
Revised: VAPOR is based on Vector Quantized Variational Auto Encoder (VQVAE) and extended with physics-informed optimization functions and reﬁnement layers, focusing on compressing and reconstructing scientiﬁc data with minimum loss of information under physics constraints.

Original: We demonstrate VAPOR by using outputs from XGC, a massively parallel fusion simulation code running on the largest supercomputers.
Revised: We demonstrate VAPOR by using outputs from XGC, a massively parallel fusion simulation code running on the largest supercomputers.

Original: Key features of VAPOR are three-fold; i) ﬁnd a reduced representation of physics data, ii) reconstruct the data with a minimum loss, iii) preserve physics information (e.g., mass, energy, moment conservation).
Revised: Key features of VAPOR are three-fold; i) ﬁnd a reduced representation of physics data, ii) reconstruct the data with a minimum loss, iii) preserve physics information (e.g., mass, energy, moment conservation).

Original: We discuss challenges in XGC 5D data reconstruction and present our initial experiences and results on how we construct Deep Neural Network (DNN) of VAPOR to optimize the reconstruction quality of XGC data and integrate XGC’s physics constraints.
Revised: We discuss challenges in XGC 5D data reconstruction and present our initial experiences and results on how we construct Deep Neural Network (DNN) of VAPOR to optimize the reconstruction quality of XGC data and integrate XGC’s physics constraints.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: As data volume grows at an exceeding rate, several ﬂoating-point lossy data compressors, such as ZFP (Lindstrom, 2014), SZ (Di & Cappello, 2016) and MGARD (Ainsworth et al., 2019) have been actively developed and applied in many science applications.
Revised: As data volume grows at an exceeding rate, several ﬂoating-point lossy data compressors, such as ZFP (Lindstrom, 2014), SZ (Di & Cappello, 2016) and MGARD (Ainsworth et al., 2019) have been actively developed and applied in many science applications.

Original: Recently, researchers have started looking at deep learning-based methods for data compression as well.
Revised: Recently, researchers have started looking at deep learning-based methods for data compression as well.

Original: While most lossy compression methods are broadly based on numerical solutions for regression, interpolation and decompositions, data compression with deep learning is mostly based on developing a generative process for a given data set.
Revised: While most lossy compression methods are broadly based on numerical solutions for regression, interpolation and decompositions, data compression with deep learning is mostly based on developing a generative process for a given data set.

Original: A family of generative deep learning models, such as Variational Autoencoders (VAE) (Kingma & Welling, 2013) and Generative Adversarial Network (GAN) (Goodfellow et al., 2014), has been developed to reconstruct whole images or augment missing parts (Welander et al., 2018;
Revised: A family of generative deep learning models, such as Variational Autoencoders (VAE) (Kingma & Welling, 2013) and Generative Adversarial Network (GAN) (Goodfellow et al., 2014), has been developed to reconstruct whole images or augment missing parts (Welander et al., 2018;

Original: Yoon et al., 2018; Ganguli et al., 2019).
Revised: Yoon et al., 2018; Ganguli et al., 2019).

Original: We have been developing a VAE-based data compression method, called VAe Physics Optimized Reduction (VAPOR), with a scientiﬁc dataset from XGC, a fusion simulation code (Chang & Ku, 2008;
Revised: We have been developing a VAE-based data compression method, called VAe Physics Optimized Reduction (VAPOR), with a scientiﬁc dataset from XGC, a fusion simulation code (Chang & Ku, 2008;

Original: Ku et al., 2009).
Revised: Ku et al., 2009).

Original: VAPOR is based on Vector Quantized Variational Auto Encoder (VQ-VAE)
Revised: VAPOR is based on Vector Quantized Variational Auto Encoder (VQ-VAE)

Original: (Razavi et al., 2019), focusing on compressing XGC’s ﬁve-dimensional (5D) particle distribution data as well as preserving physics constraints.
Revised: (Razavi et al., 2019), focusing on compressing XGC’s ﬁve-dimensional (5D) particle distribution data as well as preserving physics constraints.

Original: Key features of VAPOR are three-fold; i) ﬁnd a reduced representation of physics data, ii) reconstruct the data with a minimum loss, iii) preserve physics information (e.g., mass, energy, moment conservation).
Revised: Key features of VAPOR are three-fold; i) ﬁnd a reduced representation of physics data, ii) reconstruct the data with a minimum loss, iii) preserve physics information (e.g., mass, energy, moment conservation).

Original: We will discuss challenges in the data reconstruction of the XGC 5D particle distribution function data and present our initial experiences and results on how we construct Deep Neural Network
Revised: We will discuss challenges in the data reconstruction of the XGC 5D particle distribution function data and present our initial experiences and results on how we construct Deep Neural Network

Original: [Figure]
Revised: [Figure]

Original: (DNN) for VAPOR to optimize the reconstruction quality of XGC 5D data and integrate XGC’s physics constraints, and share performance results.
Revised: (DNN) for VAPOR to optimize the reconstruction quality of XGC 5D data and integrate XGC’s physics constraints, and share performance results.

Original: Related work: Multiple GAN-based work, such as SRGAN (Nagano & Kikuta, 2018) and ESRGAN (Wang et al., 2018), focus on generating photo-realistic, high-resolution synthetic data.
Revised: Related work: Multiple GAN-based work, such as SRGAN (Nagano & Kikuta, 2018) and ESRGAN (Wang et al., 2018), focus on generating photo-realistic, high-resolution synthetic data.

Original: In contrast, we focus on reconstructing scientiﬁc data based on physics quantities.
Revised: In contrast, we focus on reconstructing scientiﬁc data based on physics quantities.

Original: Adversarial super-resolution for climate data (Stengel et al., 2020) explores generating highresolution climate data based on multi-level generative adversarial networks.
Revised: Adversarial super-resolution for climate data (Stengel et al., 2020) explores generating highresolution climate data based on multi-level generative adversarial networks.

Original: The work focuses on generating scientiﬁc data by preserving probability densities around, such as, kinetic energy spectrum, velocity gradients, etc.
Revised: The work focuses on generating scientiﬁc data by preserving probability densities around, such as, kinetic energy spectrum, velocity gradients, etc.

Original: Our work focus on data compression and reduction for fusion data with fusion physics constraints.
Revised: Our work focus on data compression and reduction for fusion data with fusion physics constraints.

Original:  XGC AND PARTICLE DISTRIBUTION DATA
Revised:  XGC AND PARTICLE DISTRIBUTION DATA

Original: XGC (Chang & Ku, 2008; Ku et al., 2009) is a gyrokinetic, particle-in-cell code used to simulate fusion reactor designs, focusing on the multi-scale physics at the edge of the fusion plasmas.
Revised: XGC (Chang & Ku, 2008; Ku et al., 2009) is a gyrokinetic, particle-in-cell code used to simulate fusion reactor designs, focusing on the multi-scale physics at the edge of the fusion plasmas.

Original: XGC is massively parallel, requiring supercomputers such as Summit in the Oak Ridge National Laboratory to run, and generates petabytes of data for large-scale runs.
Revised: XGC is massively parallel, requiring supercomputers such as Summit in the Oak Ridge National Laboratory to run, and generates petabytes of data for large-scale runs.

Original: Virtual particles are time-advanced or “pushed” according to governing gyrokinetic equations of motion, and then interpolated or “gathered” to discrete mesh points, for solving electric and magnetic ﬁeld equations which factor into the particle motion in subsequent steps.
Revised: Virtual particles are time-advanced or “pushed” according to governing gyrokinetic equations of motion, and then interpolated or “gathered” to discrete mesh points, for solving electric and magnetic ﬁeld equations which factor into the particle motion in subsequent steps.

Original: Particles are regularly histogrammed to form particle distribution functions, F , which is ﬁve-dimensional (2D velocity, 3D space)
Revised: Particles are regularly histogrammed to form particle distribution functions, F , which is ﬁve-dimensional (2D velocity, 3D space)

Original: (Figure. 1) and ranges in size from a few GBs up to a few TBs per each iteration in a large run.
Revised: (Figure. 1) and ranges in size from a few GBs up to a few TBs per each iteration in a large run.

Original: Due to I/O bottlenecks on the supercomputer, often various reductions are applied to F to form physically relevant quantities to output, such as density ( n ) and temperature ( T ), instead of writing out the entire F data.
Revised: Due to I/O bottlenecks on the supercomputer, often various reductions are applied to F to form physically relevant quantities to output, such as density ( n ) and temperature ( T ), instead of writing out the entire F data.

Original: But because these always involve a loss of information, methods to quickly and efﬁciently reduce or compress this F data (and in the future the particle data itself) are desireable, to enable scientists to capture richer physics from these simulations.
Revised: But because these always involve a loss of information, methods to quickly and efﬁciently reduce or compress this F data (and in the future the particle data itself) are desireable, to enable scientists to capture richer physics from these simulations.

Original:  VAPOR
Revised:  VAPOR

Original: To address the output bottleneck challenge in XGC’s F -data writing, we consider an auto-encoderbased lossy data compression approach where we can build a data-driven generative model to reconstruct the original data with customized error criteria.
Revised: To address the output bottleneck challenge in XGC’s F -data writing, we consider an auto-encoderbased lossy data compression approach where we can build a data-driven generative model to reconstruct the original data with customized error criteria.

Original: One of the key ideas is how to build a data-driven learning process to ﬁnd the generative process of XGC’s F -data and develop error criteria to meet XGC’s physics constraints.
Revised: One of the key ideas is how to build a data-driven learning process to ﬁnd the generative process of XGC’s F -data and develop error criteria to meet XGC’s physics constraints.

Original: x
Revised: x

Original: Enc(x) x '' x '
Revised: Enc(x) x '' x '

Original: Decoder Encoder
Revised: Decoder Encoder

Original: Autoencoder
Revised: Autoencoder

Original: Refiner I npu
Revised: Refiner I npu

Original: t
Revised: t

Original: R e f i ned ou t pu t r e c on s t r u c t ed (a) The VAPOR architecture.
Revised: R e f i ned ou t pu t r e c on s t r u c t ed (a) The VAPOR architecture.

Original: Origin 0.
Revised: Origin 0.

Original: VQVAE 0.
Revised: VQVAE 0.

Original: VQVAE+FNO 0.
Revised: VQVAE+FNO 0.

Original: | VQVAE-Origin | 0.1 0.
Revised: | VQVAE-Origin | 0.1 0.

Original: | VQVAE+FNO-Origin | 0.1 0.
Revised: | VQVAE+FNO-Origin | 0.1 0.

Original: [Figure]
Revised: [Figure]

Original: example output.
Revised: example output.

Original: The is to minimize the
Revised: The is to minimize the

Original: We have been developing a VAE-based data compression method called VAe Physics Optimized
Revised: We have been developing a VAE-based data compression method called VAe Physics Optimized

Original: Reduction (VAPOR), aiming at learning nontrivial data distribution in an unsupervised fashion and creating a DNN encoder to ﬁnd reduced or compressed data representations.
Revised: Reduction (VAPOR), aiming at learning nontrivial data distribution in an unsupervised fashion and creating a DNN encoder to ﬁnd reduced or compressed data representations.

Original: In VAPOR, we extend
Revised: In VAPOR, we extend

Original: VAE with custom error functions and reﬁnement processes to obtain the ﬁner reconstructed output.
Revised: VAE with custom error functions and reﬁnement processes to obtain the ﬁner reconstructed output.

Original: Figure 2 shows the VAPOR’s neural network architecture and an example output.
Revised: Figure 2 shows the VAPOR’s neural network architecture and an example output.

Original: In the following, we discuss our design decisions.
Revised: In the following, we discuss our design decisions.

Original: Vector quantization VAE: Using Gaussian prior distributions followed by Gaussian reparameterization is a well-known practice in VAE.
Revised: Vector quantization VAE: Using Gaussian prior distributions followed by Gaussian reparameterization is a well-known practice in VAE.

Original: However, recent reports (van den Oord et al., 2017; Razavi et al., 2019) demonstrate improved quality based on vector quantization (VQ).
Revised: However, recent reports (van den Oord et al., 2017; Razavi et al., 2019) demonstrate improved quality based on vector quantization (VQ).

Original: The deterministic nature of the VQ method is also favorable in data compression-reconstruction for its reproducibility.
Revised: The deterministic nature of the VQ method is also favorable in data compression-reconstruction for its reproducibility.

Original: Loss function: We devise ﬂexible, physics-informed loss functions, improving the accuracy of reconstructed data and maintaining physics constraints XGC imposed in the reconstruction.
Revised: Loss function: We devise ﬂexible, physics-informed loss functions, improving the accuracy of reconstructed data and maintaining physics constraints XGC imposed in the reconstruction.

Original: In short, we deﬁne the following composite loss function for training:
Revised: In short, we deﬁne the following composite loss function for training:

Original: [Equation]
Revised: [Equation]

Original: where L recon represents reconstruction errors based on the mean squared error measurement, L VQ represents the loss in the vector quantization, and L physics represents physics quantity loss.
Revised: where L recon represents reconstruction errors based on the mean squared error measurement, L VQ represents the loss in the vector quantization, and L physics represents physics quantity loss.

Original: We want to preserve mass, momentum, and energy derived from F -data after reconstruction.
Revised: We want to preserve mass, momentum, and energy derived from F -data after reconstruction.

Original: More details can be found in  Miller et al.
Revised: More details can be found in Ku et al. ; Miller et al.

Original: Finally, α , β , and γ are the weight control parameters.
Revised: α , β , and γ are the weight control parameters.

Original: Reﬁnement: We observe VAE’s generalization (blurring) effect.
Revised: Reﬁnement: We observe VAE’s generalization (blurring) effect.

Original: Inspired by super- and multilevel resolution approaches in (Ledig et al., 2017; Stengel et al., 2020), we add a reﬁner layer to improve point-wise accuracy.
Revised: Inspired by super- and multilevel resolution approaches in (Ledig et al., 2017; Stengel et al., 2020), we add a reﬁner layer to improve point-wise accuracy.

Original: After testing with a few candidates, such as GAN and SR-GAN (Nagano & Kikuta, 2018), we ﬁnd the Fourier Neural Operator (FNO) (Li et al., 2020) works well with XGC data.
Revised: After testing with a few candidates, such as GAN and SR-GAN (Nagano & Kikuta, 2018), we ﬁnd the Fourier Neural Operator (FNO) (Li et al., 2020) works well with XGC data.

Original: Fourier transformation helps to improve recovering detailed features in the data.
Revised: Fourier transformation helps to improve recovering detailed features in the data.

Original: A more detailed quantitative/qualitative study is scheduled for the next work.
Revised: A more detailed quantitative/qualitative study is scheduled for the next work.

Original:  E XPERIMENT RESULTS
Revised:  E XPERIMENT RESULTS

Original: We demonstrate the performance of VAPOR.
Revised: We demonstrate the performance of VAPOR.

Original: We use about 131,160 examples of XGC F -data from a DIII-D 1 simulation.
Revised: We use about 131,160 examples of XGC F -data from a DIII-D 1 simulation.

Original: The full simulation consists of over 600 timesteps.
Revised: The full simulation consists of over 600 timesteps.

Original: We choose a single
Revised: We choose a single

Original: -fusion/

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: timestep where F -data shows turbulent behavior, making it more challenging to compress.
Revised: timestep where F -data shows turbulent behavior, making it more challenging to compress.

Original: Each
Revised: Each

Original: F -data is a 2D histogram array of sizes of 39-by-39.
Revised: F -data is a 2D histogram array of sizes of 39-by-39.

Original: We designed VAPOR and FNO with the number of layers of 41 and 21 respectively, which contains about 1.8 and 2.4 million parameters to optimize.
Revised: We designed VAPOR and FNO with the number of layers of 41 and 21 respectively, which contains about 1.8 and 2.4 million parameters to optimize.

Original: In Table 1, we summarize the size of the XGC F -data, the size of encoded data generated by VAPOR, and the size of VAPOR model itself.
Revised: In Table 1, we summarize the size of the XGC F -data, the size of encoded data generated by VAPOR, and the size of VAPOR model itself.

Original: As common in many lossy compression methods (such as SZ and MGARD), we demonstrate how we can further reduce the size of encoded data and
Revised: As common in many lossy compression methods (such as SZ and MGARD), we demonstrate how we can further reduce the size of encoded data and

Original: VAPOR model parameters by using Gzip.
Revised: VAPOR model parameters by using Gzip.

Original: We achieved 8x reduction with VAPOR encoding and about 29.4x reduction after compressing the encoded data with Gzip.
Revised: We achieved 8x reduction with VAPOR encoding and about 29.4x reduction after compressing the encoded data with Gzip.

Original: To compare the size by using other lossy compression method, we chose ZFP and observed only 1.2x compression ratio.
Revised: To compare the size by using other lossy compression method, we chose ZFP and observed only 1.2x compression ratio.

Original: More detailed comparison study remains a future work.
Revised: More detailed comparison study remains a future work.

Original: Figure 3 shows a few examples showing the best and worst cases generated by VAPOR.
Revised: Figure 3 shows a few examples showing the best and worst cases generated by VAPOR.

Original: One of the challenges in compressing XGC F -data is to maintain detailed features after reconstruction.
Revised: One of the challenges in compressing XGC F -data is to maintain detailed features after reconstruction.

Original: While
Revised: While

Original: VAPOR works well for the F -data in the core area shown in Figure 3(a), F -data near the edge area (Figure 3(b)) are challenging due to its complex features.
Revised: VAPOR works well for the F -data in the core area shown in Figure 3(a), F -data near the edge area (Figure 3(b)) are challenging due to its complex features.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We present a VAE-based data compression method, called VAe Physics Optimized Reduction (VAPOR), to compress scientiﬁc data while preserving physics constraints.
Revised: We present a VAE-based data compression method, called VAe Physics Optimized Reduction (VAPOR), to compress scientiﬁc data while preserving physics constraints.

Original: Based on Vector Quantized
Revised: Based on Vector Quantized

Original: Variational Auto Encoder (VQ-VAE), we extend VAPOR with custom optimization functions to integrate physics constraints.
Revised: Variational Auto Encoder (VQ-VAE), we extend VAPOR with custom optimization functions to integrate physics constraints.

Original: In addition, we add an extra reﬁnement layer based on Fourier Neural
Revised: In addition, we add an extra reﬁnement layer based on Fourier Neural

Original: Operator (FNO) to improve point-wise accuracy by overcomeing the blurring generalization effect common in GAN/VAE-based methods.
Revised: Operator (FNO) to improve point-wise accuracy by overcomeing the blurring generalization effect common in GAN/VAE-based methods.

Original: We discuss design decisions in VAPOR and demonstrate VAPOR by using real-world scientiﬁc outputs from XGC, a massively parallel fusion simulation code running on DOE supercomputers.
Revised: We discuss design decisions in VAPOR and demonstrate VAPOR by using real-world scientiﬁc outputs from XGC, a massively parallel fusion simulation code running on DOE supercomputers.