Learning Efficient Hybrid Particle-continuum Representations of Non-equilibrium N-body Systems
Authors: Tailin Wu
Michael Sun
H.G. Jason Chou
Pranay Samala
Sithipont Cholsaipant
Sophia Kivelson
Jacqueline Yau
Zhitao Ying
E. Paulo Alves
Jure Leskovec
Frederico Fiuza
Venue: Submitted to ICLR 2023 (2023)
Keywords: multi-scale
hybrid representation
particle-continuum
n-body
plasma

Abstract: An important class of multi-scale, non-equilibrium, N-body physical systems deals with an interplay between particle and continuum phenomena. These include hypersonic flow and plasma dynamics, materials science, and astrophysics. Hybrid solvers that combine particle and continuum representations could provide an efficient framework to model these systems. However, the coupling between these two representations has been a key challenge, which is often limited to inaccurate or incomplete prescriptions. In this work, we introduce a method for Learning Hybrid Particle-Continuum (LHPC) models from the data of first-principles particle simulations. LHPC analyzes the local velocity-space particle distribution function and separates it into near-equilibrium (thermal) and far-from-equilibrium (non-thermal) components. The most computationally-intensive particle solver is used to advance the non-thermal particles, whereas a neural network solver is used to efficiently advance the thermal component using a continuum representation. Most importantly, an additional neural network learns the particle-continuum coupling: the dynamical exchange of mass, momentum, and energy between the particle and continuum representations. Training of the different neural network components is done in an integrated manner to ensure global consistency and stability of the LHPC model. We demonstrate our method in an intense laser-plasma interaction problem involving highly nonlinear, far-from-equilibrium dynamics associated with the coupling between electromagnetic fields and multiple particle species. More efficient modeling of these interactions is critical for the design and optimization of compact accelerators for material science and medical applications. Our method achieves an important balance between accuracy and speed: LHPC is 8 times faster than a classical particle solver and achieves up to 6.8-fold reduction of long-term prediction error for key quantities of interest compared to deep-learning baselines using uniform representations.

Under review as a conference paper at ICLR 2023
LEARNING EFFICIENT HYBRID PARTICLE-CONTINUUM REPRESENTATIONS OF NON-EQUILIBRIUM N-BODY SYSTEMS
Anonymous authors Paper under double-blind review
ABSTRACT
An important class of multi-scale, non-equilibrium, N-body physical systems deals with an interplay between particle and continuum phenomena. These in- clude hypersonic flow and plasma dynamics, materials science, and astrophysics. Hybrid solvers that combine particle and continuum representations could provide efficient framework to model these systems. However, the coupling between these two representations has been a key challenge, which is often limited to inac- curate or incomplete prescriptions. In this work, we introduce method for Learn- ing Hybrid -erticl-Cooniiuum (LHPCC models from the data of first-principles particle simulations. LHPC analyzes the local velocity-space particle distribution function and separates it into near-equilibrimm (thermal) and far- Tar-from-equilirrinnn moon-thermal) components. The most computationally-ineen intennive particle solver is used to advance the non-thermal particles, whereas neural network solver is used to efficiently advance the thermal component using continuum representa- tion. Most importantly, an additional neural network learns the particle-contnnumm coupling: the dynamical exchange of mass, momentum, and energy between the particle and continuum representations. Training of the different neural network components done an integrated manner to ensure global consistency and sta- bility of the LHPC model. We demonstrate method in intense laser- aaser-plasma interaction problem involving highly nonlinear, hur-fro--quiiiirium dynamics associated with the coupling between electromagnetic fields and multiple particle species. More efficient modeling of these interactions is critical for the design and optimization of compact accelerators for material science and medical appli- cations. Our method achieves an important balance between accuracy and speed: LHPC is 8x faster than classical particle solver and achieves up to 6.8-fold re- duction of long-term prediction error for key quantities of interest compared to deep-learning baselines using uniform representations.
INTRODUCTION
The dynamics of physical systems is often nonlinear and involves the competition of different pro- cesses across a wide range of spatial and temporal scales. This gives rise to local non-equilibrium conditions (in the thermodynamic sense) that results in the failure of common numerical approaches. While continuum models (e.g., based on fluid equations) can provide accurate and computationally efficient descriptions of near-equilibrium systems at large scales, they break down when significant departures from local equilibrium are encountered (often at small scales) and give rise to important N-body phenomena. Kinetic (e.g., particle-based) numerical methods can accurately describe these non-equílibrium phenomena but are very computationally intensive, limiting their practical applica- tion to small scales. Over the last decades, this has motivated efforts to develop hybrid algorithms that can more efficiently couple continuum and particle representations a variety of fields, includ- ing hypersonic gas dynamics (Schwartzentrubeer & Boyd 2006). high-energy-densine physics (Fiuza et al. 2011), and plasma physics (Bai et al. 2015).
a particularly challenging class of com- plex physics systems, where long-range electromagnetic interactions inevitably drive multi-scale and nuuffrom-equllibrium dynamics. Indeed, plasma research associated with controlled nuclear fu-
Plasmas hot ionized gases of charged particles

---

Under review as a conference paper at ICLR 2023
(9)
kneek partidies (oumubred to 0
knete partides
lónotic
Clasccll Salver g
newly ijjezted particies (OK:0))ir")
newly injected parcides (M))-)
(3)
Bockground (chermal) [M*)
()) gifond
background
background
(thermal)
(thermal)
Flid 2
Badiground
(b)
Fujjosh
Disorbution Momeramm (MSPP)
Sample
Newyy rnjected parsiee (cxu))a)
pe')
(Mernn
Network
(a) Schematic of our LHPC architecture. consists of three components: (1) neural network Uraid for evolution of thermal uubpoppultiion with fluid representation; (2) solver g for the evolution of the non-thermal energetic sub-population with particle representation; (3) neural networks that model the injection of particles from fluid state (Fijeaa). (b) and (c) illustrate the architectures of the components (1) and (3).
Figure 1:
sion (Zylstra al. 2022). advanced laser- aaser-plasma particle accelerators (Haberberger et al. 2012),
and most space and high-energy astrophysical environments (Drake et al.) (006) have long stim- ulated the development of hybrid canticle-contnnumm representations to accurately and efficiently capture the nonlinear dynamics of these systems (Winske 1985) Fiuza et al., 2011 Bai et al. 2015) Liu et al. 2022). all these cases, small-scale kinetic processes can accelerate a very small group of particles to energies significantly above the mean (thermal) energy, driving the system out of equilibrium. Importantly, this small group (few %) of non-thermal particles can carry away large fraction (up to 50%) of the system energy and thus impact its global evolution. This motivated the recent development of hybrid representations that use a fluid solver model the near-equee eeur-equilibrim (thermal) part of the particle distribution and particle-basee
kinetic solver to model the evolution of high-energy particles (Kowal et al., 201T I Bai et al.) 2015) Guidoni et al., 2016). However, the cou- pling of the two representations has been based over-simplified phenomenological prescriptions that can limit their validity and applicability.
In this work, introduce method for Learning Hybrid Particle-Continuum (LHPC) models to address the key challenge of efficiently and accurately coupling fluid and kinetic representations of far- har-from-en www.--rrruulllle from-equilibrium N-body systems. LHPC combines classical particle-in-cell (PIC) solver to advance the non-thermal particle distribution with a neural network surrogate model that efficiently advances the thermal component using a continuum representation. Most importantly, our key tribution is the use of an additional neural network to learn the self-consistent particle-coninuum coupling: the dynamical exchange of mass, momentum, and energy between the particle and con- tinuum representations. This coupling is learned from the data of first-principles PIC simulations, providing an accurate and physics-informed informed description that addresses the main limitation of previ- ous hybrid methods.
While the combination of classical numerical solvers and deep learning has been explored before (Um et al. 2020 Vlachas et al. 2022), these have been primarily based on uniform representations. The use machine learning techniques learn efficient and accurate coupling between continuum and particle representations introduced in this work is promising and important route for addressing the multi-scale and multi- peltii-phyics challenge modeling N-body non-equilibrimm systems.
We demonstrate our method a challenging non-linear, far-from-equilibrium N-body system: the interaction of an intense laser with a solid-density plasma and resulting particle acceleration. Present-day high-power lasers reach intensities in excess of 10 /cm², which nearly instanta- neously vaporize and ionize solid-state matter upon interaction. resulting in bigh-energy-denstee plasma. These interactions give rise nonlinear and kinetic processes that establish strong elec-
2

---

Under review as a conference paper at ICLR 2023
tric fields in the plasma and accelerate particles to high energies over very compact distances. In the last two decades, there has been great interest in exploiting these laser-plasma based accelera- tors for wide range of applications, including material science (Patel et al.) 2003), imaging (Rygg) eet al. 2008), and medical therapy (Kroll et al. 2022). Due to the need to capture detailed kinetic physics associated with these interactions, numerical modeling has relied primarily on fully kinetic PIC simulations, which are too computationally intensive in fact, one-to-one modeling of experi- mental systems is computationally prohibitive even on the largest supercomputers. Effective hybrid particle- marticeecontinuum representations have not been previously established for this problem. This is thus a prime example for testing our model and evaluating the ability of LHPC to learn efficient and accurate hybrid representations that can have a transformative impact in the modeling of these complex laser-plasma interactions.
Our results show that LHPC achieves an important balance between accuracy and speed: LHPC is times faster than a classical particle solver, and achieves up to 6.8-fold reduction of long-term prediction error for key quantities of interest when compared to deep-learning baselines using uni- form representations. This opens up a novel route to develop more efficient and accurate particle- continuum descriptions across different domains involving non-equilibrium N-body systems, and represents a key step for data-driven multi-scale algorithms for nonlinear physical systems.
2 RELATED WORKS
Deep learning has recently emerged as powerful tool to complement
Um et al. (2020) or serve
as a surrogate for classical solvers (Sanchez-Gonzalez et al. 2020 They accelcrate simulation of
physical systems through coarser spatial resolution (Um et al. 2020 Kochkov et al.] [02TT) or tem-
poral intervals (Li et al. 2021: Vlachas et al., 2022), explicit forward method (Tang et al. 2020
Wu al. 2022b), or via reduced representations
Baachez-Gonzalee et 2020 Wu et al. 2022a)
Two important examples are LED (Vlachas al. (022) and solver-in-the-loop (Um et al.) 2020)
The former optimizes the temporal evolution by employing a solver to simulate the full system in some interval of time and uses latent evolution to evolve in other parts of time, with a pre-defined alternative schedule. The latter uses a fluid solver on a coarser spatial grid and a neural network to correct for the fluid solver's 's error. However, almost all works employ uniform representation, based on either all particles or all fluid. This is not appropriate when trying to optimize far-from- equilibrium, N-body systems where particle description is too computationally expensive for the near-ther thermtl population (where most particles reside) and a fluid description cannot accurately de- scribe the complex distributions associated with the -or--rro-uuuibbiuum population. Our method aims to address this long- standing challenge by learning an accurate and efficient coupling between different representations involving different physical processes and scales, namely a hybrid particle- continuum representation which is critical to describe non-equilibrium N-body systems. To the best of our knowledge, machine learning methods have not yet been used to address this problem.
PROBLEM SETUP AND PRELIMINARIES
Task setup. In general, simulations of the time-evolution of dynamical system can be described as follows. The state of the system at time t is s², and there exists a ground-truth evolution g that evolves this state:
g (s*) t= 0,
The ground-trute evolution g" can either be the physical world which is challenging to predict, or a first-principles numerical solver which can be slow and expensive for large-scale systems. Assume that we have a pre-defined mapping h which maps the state sf to a representation S² S' = h(s*), t = 0, 2, such that S" captures the essence of the system. For example, con- sider an original state st represented by particles, and St a fluid representation which describes the system with statistics of the particles within each cell (see below). Alternatively, h may be iden- tity mapping resulting in St = s'. A hybrid h mapping will apply both of these representations to different groups of particles. Given the states {s"), t T, the task is to design proper representation St = h(s'), and learn a surrogate model ga. typically in terms of a neural network, which approximates the ground truth evolution of 5t:
3

---

Under review as a conference paper at ICLR 2023
(B)
electron
1-.55%
1- 16R
10
eeocdroo 0/n urh))
unie
-.- 5.5
16
1.
target
(x
fiald
energeoc
particies
2
00
02
04
D6
to
0.06
:5556 16
0.04
Flan Plonbra a/d
plontarb.unt
10
0.3
10
Freqy evyy
0.0
100
00
2
0 [ohAn]
00
04
08 Energy (Hel))
Figure 2: (a) Intense laser-plasma interactions generate energetic particles with rich dynamics. (b)
Projection of the particle dynamics onto the longitudinal momentum (ug) (ug)-position (x) phase spaces, the laser E-field (orange), and the corresponding energy spectra of the particles at two different times.
(*) particle representation of full plasma
(b) hybrid partidd--ooninuum representation
partide representation offer
concinuum (fuid momenes) off
oleuron (ueb and)
www
02
fur U)
18 8 8
[ec]] c de o vp 8
-01
(c.M
(clus)
[closs
[clus]
Figure 3: Representations of plasma illustrated with electrons. (a) Particle representation of the non- thermal (JNT) and thermal (/T) populations. (b) Hybrid particle-continuum representation showing the same particle representation of fNr and example velocity moments of the thermal electron fluid.
$1+1 ge( 0,1,
Here S1+1 the model go's prediction of S1+1, Our goal is to have the long-term autoregressive evolution of the learned surrogate model Št+ go go (St) (composing & times) to closely match the ground-truth St++k
Physical problem overview. this work, we consider the interactions of intense laser pulses with plasmas, where the laser field accelerates the plasma electrons the front surface, which in turn sets up a charge separation, generating electric (E-) and magnetic (B-) fields that accelerate and deflect the ions (Fig- [ see Appendix A for details). These interactions of the charged particles with E- and B-fields (collectively EM-fied) occur on ultra-fast time scales (~ 10-18 s), and can produce electrons and ions with energies exceeding 106 electronvolt (= MeV) with the electrons moving at close to the speed of light c. These are prime examples of non-equilibrium systems, for which computationally expensive first-principles simulations are necessary accurately capture the associated non-linear dynamics.
Particle and continuum representations. The two dominant representations of the plasma system considered in our work are particle (kinetic) and continuum (fluid) representations. A particle repre- sentation ((rf, uf. Qs)} describes the position æ/., momentum vector u', and other static properties Q: (e.g., mass or charge) of each particle i. The particles interact with each other, and change their position and momentum accordingly. In contrast, a fluid representation M defined on a fixed grid or mesh. The values on each vertex represent the moments (statistical averages) of the mo- menta of the particles residing around the vertex. These statistical quantities summarize the states of large number of particles, permitting more compact representation. Their physical meaning well established. For example, the first three moments correspond to the fluid observables mass density, fluid velocity, and pressure. We note that in general exact representation of the full par-
4

---

Under review as a conference paper at ICLR 2023
ticle distribution requires infinite number of fluid moments (see Appendix A.1 However, if the particle distribution is near local thermodynamic equilibrium it can be described by a compact set of fluid moments. A hybrid particle-continumm model aims to provide efficient description of more general distribution of particles by coupling a partiele representation for far-from-equilibrium particles with continuum representation for particles
equilibrium.
this context, we refer to the group of particles a near equilibrium state as thermal, for which the momentum distribution can be well described by Gaussian (i.e. requiring only the mean and variance). On the other hand, we refer to the particles that are far from equilibrium as non-thermal. These are the outliers from this Gaussian distribution. In terms of the total particle distribution function, f. this allows us to decompose it into a thermal, fr, and non-thermal, JIT components, with f= fr + [NTT- See Fig. for an illustration of this separation in physical problem of intense laser-plasma interactions considered in this work; details of the separation algorithm are presented in Appendix
Thermal and non-thermal population.
METHOD
4.1 METHOD OVERVIEW
Here we introduce Learning Hybrid Particle-Comtinuum Models (LHPC) to address the key chal- lenges in hybrid representations of non-equilibrium dynamical systems. Specifically LHPC learns two key components, for the evolution of the fluid, and coupling between the particle and fluid rep- resentations. In Figs. I 3 and the following sections, we detail the data generation and preparation procedure, the main architecture, and its application to the study of intense laser- plasma interactions.
4.1.1 DATA GENERATION AND PREPARATION FROM FIRST-PRINCIPLES SIMULATIONS
Although computationally expensive, full PIC simulations provide ground truth data from first- principles particle-based evolution of plasma dynamics. We run full PIC simulations of aaser-plasma interactions one spatial dimension, three velocity dimension (ID3V) setting, where the laser in- tensity is varied to generate multiple trajectories. A trajectory is composed of a simulation with 2000 time steps of evolution, involving 1.5 million (numerical) particles evolving on 1=1kk-cll grid, and is 200 GB in size (see Appendix A.3 for details).
To train the two learned components of LHPC, we transform the ground truth particle data into fluid and particle representations at each time step. This is achieved by first applying algorithm that separates the particles into thermal and non-thermal populations (e.g. Fig. 3 for details see Appendix B). The labeled particle data is then transformed, with the non-thermal population keeping the original particle representation, and using the velocity moments (up to 2nd order) to represent the thermal fluid. As the system evolves, small fraction of the partieles from the thermal population will gain a significant amount of energy to get detached from the thermal distribution, becoming non-thermal. They are said to be "injected" into the non-thermal population. The transformed ground truth data therefore provides the information of the evolution of both representations, and the coupling (injection) between them.
4.1.2 ARCHITECTURE
LHPC evolves the system state St at time to state 5t+ I at time ++1 with three components (Fig.
(1) A neural network Inmid,o with parameters 0 which evolves the (thermal) fluid (Fig. Tb).
(2) A ground-truth solver g" which evolves the non-thermal particles.
(3) A neural network Iinject.pp with parameters &p which models the interplay of the fluid and particle populations (Fig. [e... In particular, it models the injection of new non-thermal particles from the fluid, and updates the existing fluid and particle populations accordingly.
At inference, these processes are iterated autoregressively to predict the state of the system into the long-term future.
5

---

Under review as a conference paper at ICLR 2023
4.2 LHPC FOR LASER-PLASMA INTERACTIONS
Here, we employ LHPC to model laser-plasme simulations. The space in æ is divided into C' con- secutive cells: c E (Clet, Clet+1...... Cright) in the æ direction. The state S consists of:
S" = ((M', MM, ,Nrr",, (INT', unr', sPic", uric"), (E', BB))):
CE (c, (t)...))).
where the velocity moments of different populations M and the non-thermal particles (æ, interact through the fields (E, B). The full pipeline in Fig. i Appendix C further visualizes the depen- dencies amongst these components and documents what each quantity represents. The following sequence of operations is used to evolve the system from time to time + 1:
M++ + (M, BY, M")
Advance fluid moments according to electromaanetic (EM-) field on the grid.
(la)
9M,9 is an instantiation of the I introduced in Sec. 4.1.2
M + 9 y (M*+),M*
Compute moments of new non-thermal particles to be injected
(Ib)
9p is an instantiation of the Pinject,pp introduced in Sec. 4.1.2
(z, + + -N((
(1c)
Inject new non-thermal particles by sampling from distribution with M
1 +-
(INT, uNr, M)
(Id)
Update fluid moments to conserve mass, momentum, and energy
(x, wii + [[G((z, W/i (E, B)));
(z, u]+]]
(1e)
Advance existing particles and append newly injected non-thermal particles
je+1 + (M', u,) u)++1
(1
Deposition of electric current from particles and fluid grid
(F,, B)(+1
J--1
(1g)
Advance EM-field on the grid using Maxwell's equations
The operators 9M.0 and 9M are instantiated from Inuid, and Pinject. and are deep neural network models learned from data. The operator G and the field solver advances the non-thermal particles with the classical PIC solver. A more detailed description of each component a-j can be found in Appendix C.I and detailed architecture, learning procedure and objective is Appendix B
4.2.1 BASELINES
We compare with state-of-the-art deep learning-based surrogate model of Fourier Neural Operator (FNO: [i et al. 202T,, and a baseline CNN that has the same architecture as our fluid evolution model without hybrid representation. Both architectures are evaluated on two representations: (1) an all fluid representation which models partieles as fluid defined on each cell; (2) bi-Gaussian fluid representation, that models thermal particles and non-thermal particles separately but each as Gaussian. Baseline (1) treats the entire system as a fluid and advances the system by learning g as:
M++1 (E, B) B), M²).
The improved "bi-Gaussian" bascline (2) models the thermal population and the non- con-thermal parti- cles each as Gaussian, and advances the system by learning g as:
M1 M (F., B)++1 g(EF, B) M Maa)

---

Under review as a conference paper at ICLR 2023
To evaluate the importance of the coupling network 9Ms..p also compare hybrid model with an ablation where we remove 9Mrr.ep while keeping all other parts the same ("LHPC (no- coupling)"). For all models, we care especially about the error on the moments of the non-thermal subpopulation Mpic to understand the ability of the respective methods to correctly capture the most dynamic subpopulation of particles. This is important because in many of the relevant applications of frerri--euuiliiium, N-body systems we are interested in the spectrum of accelerated particles.
5 EXPERIMENTS
In this section, we aim to answer two questions: (1) does our LHPC model with hybrid representa- tion achieve better accuracy than learning the evolution with a pure fluid representation? (2) Does LHPC offer speedup, compared to the full-PIC solver that evolves the full system in first principles with particle representation? We evaluate LHPC and baselines in the N-body laser-plasma inter- actions system described above. We evaluate both single-step prediction error and auto-regressive rollout error over 50 steps, where the state of the system changes significantly. We use relative L2 as the error metric the ratio between L.2 norm of error over L.2 norm of ground-truth values.
5.1 DATA AND EXPERIMENT
Both the baseline and main pipeline are trained over 9 trajectories of PIC simulation. 10 trajectories were generated that vary over the normalized laser intensity ag from [2, ,,, 20] (ao 10) is held-out for testing (see Appendix[A]for details). In this way, we test the generalization of the models to novel initial conditions and environment setup. Each dataset consists of T = 1500 time steps and uses the separation strategy (details in Appendix [B). For the nine datasets, train with single-step loss on the time-range [0, (40], and validate on [1400, 1500]. We iterate on hyperparameters to obtain the best individual settings for both the baseline and hybrid pipelines respectively (detailed in Appendix). Then, the best models are finetuned using the multi-step loss with data from timeranges (980, 1000] U [1080, 1100] U [1180, 1200] U [1280, 1300] U [1380, 1400), and validated on [1400, 1500]. We report the confidence intervals of the 1-, 20- and 50-step relative 1.2 errors on the held-out dataset and report the results in Table T which shows the error for the EM-field, the non-thermal particles (Mric) and the full fluid, respectively. Runtime for ground-trute (GT) solver and the compared models are also provided in Table We provide additional results in Appendix D with an ablation study on trajectory 10 with the most intense laser (ao 20).
and trajectory
5.2 RESULTS
Error comparison. From Table we see that our LHPC typically outperforms the baselines in terms of error by a wide margin. Among the three aspects we evaluate, the EM-field error and the non-thermal particles (Mric) are the most important, since the EM-field is most sensitive to the local particle dynamics, and the non-thermal particles are typically the most important observ- able/deliverable of laser-plasma accelerators. An all-fluid description can be very efficient and accu- rate at describing the thermal (M) population, as is common for traditional fluid solvers. However, it fails to capture non-thermal particle acceleration, manifested in large errors for the field evo- lution (Field) and nhn-thermal population ( (Mpic). Specifically, for Field, our LHPC achieves an error reduction of 91.0%, 91.3% and 85.4% (6.8-fold) for 1-, 20- and 50-step autoregressive rollout, compared to the best models among the baselines. Similarly, in the evaluation of MPIc. our LHPC achieves an error reduction of 63.5%, 36.2%. 25.8% for 1-, 20- and 50-step predictions, respec- tively, demonstrating the ability of our LHPC to model accurately the dynamics of those energetic non-thermin particles. The Baseline CNN, although having the same architecture as our LHPC, can- not model accurately the non-thermal particles via either all-fluid or Bi-Gaussian representations, thus achieving much larger error. Specifically, while the use of separate fluid description for the non-thermal population (Bi-Gaussian model) can help capture its impact on the system evolution, it still leads to significantly larger errors in the field evolution when compared to the LHPC model and cannot be used to fully describe the complex distribution of the non-thermal particles (see also Sec. (.3). Comparing baseline and state-of sfate-of-the-ar FNO, we find that FNO performs comparable or slightly worse than Baseline CNN, likely due to the simpler architecture of CNN being more appro- priate to capture the local fluid evolution. The LHPC (no-coupling) ablation shows that the coupling component is essential in ensuring good long-term accuracy, especially for Field and Mptc- On
7

---

Under review as a conference paper at ICLR 2023
Method
Component
Firor 6
Brror step 20
Frmrr step
Speed (s/step)
GT Solver (fuil PIC)
9.21E-01
Field
5.77E-02 1,526-02
9.36E-01 2.00E-01
2.57E+00 8.56E-01
FNO: All-fuid
Mлc
7.79E-02
M
5.53E-03 + 1.16E-03
6.79E-02 + 1.47E-02
3.07E-01 = 6.71E-02
Field
2.98E-02 + 2.36E-03
4.69E-01 + 5.49E-02
9.65E-01 = 1.51E-01
FNO: Bi- Ba-Gaussian
3.25E-02 + 8.10E-03
3.83E-01 + 9.30E-02
1.03x+00 4.75E-01
1.66-001
M
6,92E-03 + 1.17E-03
8.63E-02 + 1.01E-02
2,38E-01 = 3,45E-02
Field
1.97E-02 7.53E-03
183E-01 = 5.86E-02
7.16E-01 = 3.666-01
Baseline: All-fluid
Mис
4.61K-02
M
2.62E-03 4.63E-04
4.86E-02 9.02E-03
1.31E-01 3.55E-02
Field
1.12E-02 3.68E-03
1.31E-01 2.61E-02
3.67E-01 = 4.77B-02
Baseline: Gaussian
Mac
2.24E-02 + 9.21E-03
182E-01
7.70E-02
5,00E-01 = -8.01E-2
1.10E-01
5.05E-04
6.07E-02
8.691-03
1.39F-01 1.08E-02
Field
L00E-03 1.72E--4
140E-02
1.66E-03
1,385-01
4.52E-02
LHPC (no-coupling)
Mpc
8.18E-03 2.02E-03
1.53E-01
5.97E-02
5.15E-01
1.15E-01
1.0SE-01
M
4.51E-03 7.12E-04
9.29E-02
2.00E-02
3.26E-01 = 7.00E-02
Field
L01E403 1.57F.04
1.14F.02
6.76F-04
5,34K-02
6.68F-03
LHPC
Mac
8.18E-03 2,02E-03
1.16--01
+ 6.69E-02
3,71E-01 + 7,83E-02
1.15E-01
M
4,51E-03 7.12E-04
7.44E-02 + 8.80E-03
1.80E-01 2.88E-02
Table I: Results for the N-body laser-plasma interactions system on the held-out dataset. We eval- uate three aspects of prediction: EM-field (Field), non-thermal particles (Mpic). and the fluid mo- ments M. Mean and 90% confidence interval are reported. The results are average performance on the held-out dataset over 15 independent rollouts with initial conditions at t = 0, 100, .... 1400. Our LHPC achieves significant improvement on the EM-field predictions in both short- and long-terms. In terms of EM-field, LHPC achieves 91. 3100% 91.3%, and 85.4% (6.8-fold) reduction of error for 1-, 20- and 50-step rollouts, respectively, compared to the best-performing baseline. Similarly, for Mric LHPC achieves error reduction of 63.5%, 36.2%, 25.8% for 1-, 20- and 50-step predictions. Note that the performance is N/A for all-fluid representation models for MrIc as they cannot model the non-the oon-thermal particles separately. For M prediction, our LHPC has comparable performance with the baselines, since both use similar architectures to model the thermal part of the distribution as fluid, but the baselines also receive M as an input whereas LHPC does not.
predicting the fluid moments (M), our LHPC model performs comparably to Baseline that has the same CNN architecture, showing both models can model reasonably the dynamies of the fluid.
Runtime comparison. While achieving significant error reduction on predicting the EM-field and the non-thermal particles, our LHPC also achieves significant speedup compared to the Ground- truth (GT) full PIC solver, with reduction of runtime by 8.0-fold. We also see that the runtime of LHPC much higher than full deep learning based baselines, showing that although our LHPC involves GT solver evolve the non-thermal particles, the increase in runtime is negligible since non-thermal particles only constitute a small fraction of the system.
5.3 VISUALIZATION OF RESULTS
Full-PIC
LHPC
time shopss
[ofie]]
[[/oa]]
Figure 4: Comparison of the evolution of non-thermal electron current JeNT between ground truth (Full-PIC) and LHPC across steps of rollout.
In this section, we visualize the predictions of LHPC. An accurate evolution of the system dynamics relies the correct EM-field (Eq. Tg). which is dictated by the electric current J (Fqq. Tf). Figure
8

---

Under review as a conference paper at ICLR 2023
shows that model predict the evolution of J contribution from the non-thermal population accurately over 50 time steps rollout. This is further illustrated in Fig. where for the different contributions of J, good agreement between the predictions of our model and the ground truth is ob- served. Similarly, our model predictions for components of the E-field and the thermal fluid density match well the ground truth after 20 steps of rollout. These results demonstrate faithful evolution of the important quantities that describe the dynamics of the fluid, the non-thermal particles, and the coupling between them. Importantly, our model able to accurately capture the critical aspect of laser-plasne interactions trivial energy distribution of the accelerated (nontheermal) particles cannot be properly described with few fluid moments and must rely on the accurate evolution of the particle representation.
non- non-thermal particle acceleration (Fig. 6). Note that the highly non-
RUHFIC
Jan (arudd
wond
x Johoo]
[oloe)
x [2000)
Figure 5: Comparison of ground truth (Full-PIC) and LHPC model predictions after 20 steps of rollout, for E-field components (left column) Eg. Ey (middle) total density of the fluid n, and J contribution from the fluid electrons Je, (right) non-thermal ions JIN and electrons JeNT.
electron
10
LHPC
10
dlccronn (arb. uni
10
10
electrnn
101
10
0.2
0.4
0.8
0.0
0.4
0.5
1.0
Energy (MeV)
Energy (MeV)
Figure 6: Comparison of energy spectrum of non-thermal electrons (left) and ions (right) between ground truth (Full-PIC) and LHPC after 20 steps of rollout.
DISCUSSION AND CONCLUSION
This work helps address a key challenge in modeling multi-scale, non-equilibrium. N-body systems by providing novel hybrid particle-continumm model, whose coupling is learned from the data of first-principles simulations. LHPC outperforms the classical first-principles solver by an order of magnitude in speed, and the baseline method that models the entire system fluid by up to order of magnitude in accuracy. Further speed-ups are expected in multiple dimensions as the non-therma population constitutes much smaller fraction of the system representation (see Appendix A.4).
Our method can be extended towards finding more efficient and accurate particle-cone continuum descrip- tions across different domains involving non-equilibrium N-body systems, and represents key step towards the development of advanced data-driven algorithms for multi-scale modeling of nonlinear physical systems.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Xue-Ning Bai, Damiano Caprioli, Lorenzo Sironi. and Anatoly Spitkovsky.
METHOD FOR COUPLING COS- MIC RAYS WITH A THERMAL PLASMA: APPLICATION TO NON-RELATIVISTIC
doi: SHOCKS. The Astrophysical Journal, 809(1):55, aug 2015. ISSN 1538-4357.
URL http:// iog. 000p:/0004-0000 0000///0000333/80005 637X//09/ 10:0088000446377780991555 w1w..aaa06000000f6666e0aaaaaaaaa
C. K. Birdsall and A. B. Langdon. Plasma Physics via Computer Simulation. McGraw-Hill, Elec- trical Engineering and Computer Sciences Department, University of California, Berkeley, 1991.
Johannes Brandstetter, Daniel Worrall, and Max Welling. Message passing neural pde solvers. arXiv preprint 2xXiv:2202.03370 2022.
B.I. Cohen, A.J. Kemp, and L. Divol. Simulation of laser-plasma interactions and fast-clectron transport inhomogeneous plasma. J. Comput. Phys., 229(12):4591-4612 jun 2010. ISSN 0021-9991.
John M. Dawson. Particle simulation of plasmas. Rev. Mod. Phys., 55:403 Apr 1983. doi: 10.103//wMooPhys.5..4
URL
RevModPhys. 55.
JF Drake, M Swisdak, H Che, and MA Shay. Electron acceleration from contracting magnetic islands during reconnection. Nature, 443(7111):553-550 2006.
Fiuza, M. Marti, R. A. Fonseca, L. O. Silva, Tonge, May, and W. B. Mori. Efficient modeling of laser-plasma interactions in high energy density scenarios. Plasma Physics and Controlled
Fusion, 53(7):74004-4405,, jul 2011. ISSN 07413335.
doi: 10. 40-108880741-335555777700000
URL
374004/met.a
S. E. Guidoni, C. R. DeVore, J. T. Karpen, and B. J. Lynch. Magnetic-island contraction and par- ticle acceleration in simulated eruptive solar flares. The Astrophysical Journal, 820(1):60, 2016. doi: 10.364770000-637788221/16 URL
htt:::/ooi..ooo 100
2F82032F182F61
Dan Haberberger, Sergei Tochitsky, Frederico Fiuza, Chao Gong, Ricardo A Fonseca, Luis O Silva, Warren B Mori, and Chan Joshi. Collisionless shocks in laser-produced plasma generate monoen- ergetic high-energy proton beams. Nature Physics, 8(1):95-99, 2012.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
Yu. L. Klimontovich. The Statistical Theory of Non-Equilibrium Processes in a Plasma. MIT Press, Cambridge, 1967.
Dmitrii Kochkov, Jamie A Smith, Ayya Alieva, Qing Wang, Michael P Brenner, and Stephan Hoyer. Machine tearning-accelerateed computational fluid dynamics. Proceedings of the National Academy of Sciences, 118(21). 2021.
G. Kowal, E. M. de Gouveia Dal Pino, and A. Lazarian. Magnetotydrodynamic simulations of reconnection and particle acceleration: three-dimensional effects. The Astrophysical Journal, 735 2011, doi: 10. 000100880000443337777/7/220 URL #B004-6778827332222222
Florian Kroll, Florian-Emanude Brack, Constantin Bernert, Stefan Bock, Elisabeth Bodenstein, Ker- stin Brüchner, Thomas E. Cowan, Lennart Gaus, René Gebhardt, Uwe Helbig, Leonhard Karsch, Thomas Kluge, Stephan Kraft, Mechthild Krause, Elisabeth Lessmann, Umar Masood, Sebastian Meister, Josefine Metzkes-Ng. Alexej Nossula, Jörg Pawelke, Jens Pietzsch. Thomas Püschel, Marvin Reimold, Martin Rehwald, Christian Richter. Hans-Petee Schlenvoigt, Ulrich Schramm, Marvin E. P. Umlandt, Tim Ziegler, Karl Zeil, and Elke Beyreuther. Tumour irradiation in mice
10

---

Under review as a conference paper at ICLR 2023
with laser-accelerated proton beam. Nature Physics, 18(3):316 322, mar 2022. ISSN 1745-
2481. doi: 10. 10000088/411677-00--1555333333
https/::::// www. nat com/ articles/
s 6=1567-022-01520-1
Zongyi Li, Nikola Borislavov Kovachki, Kamyar Azizzadeneshelii Burigede liu, Kaushik Bhat- tacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. In International Conference on Learning Representations, 2021. URL
https://gpenreviier net Forumridd----o..ooo... C8P
Chang Liu, Stephen C. Jardin, Hong Qin, Jianyuan Xiao, Nathaniel M. Ferraro, and Joshua Bres- lau. Hybrid simulation of energetic particles interacting with magnetohydrodynamies using slow manifold algorithm and gpu acceleration. Computer Physics Communications, 275: 108313, 2022. ISSN 0010-4655, doi: http:///o...rrg//l..o////ooooppp 108313, URL httpss
www.ssiie.eee com http:///ccccatrra...t..e////////
Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.0390 2016.
Patel, A. Mackinnon, M. Key, T. Cowan, M. Foord, M. Allen, D. Price, H. Ruhl, P. Springer, and R. Stephens. Isochoric heating of solid-density matter with ultrafast proton beam. Physical Review Letters, 91 1250444 9 2003. ISSN 0031-9007. doi: 10. Re. 155000 URL https: /tttps://link aps wttp:o////ooooooe/.oo wtt.::.1111//////.eeeee
91 125004
J. R. Rygg, F. H. Séguin, C. K. Li, A. Frenje, M. J.-E. Manuel, R. D. Petrasso, R. Betti, J. A. Delet- trez, O. Gotchev, J. P. Knauer, D. D. Meyerhofer, J. Marshall, C. Stoeckl, and W. Theobald. Proton radiography of inertial fusion implosions. Science, 31 (1915667) 9203. doi: 10.1 10.1116//ccence.1 152640. URL http:httt///t /wwp://wwwww./..m ncbi n 1m1 nih gww..oo//ubbeee.eoooo 1830907
2 2008. ISSN 1095-
Alvaro
Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovee, and Peter Battaglia. Learning simulate complex physics with graph networks. In International Confer- ence on Machine Learning, pp.
PMLR, 2020.
T.E. Schwartzentruber and L.D. Boyd. A hybrid particle-contnnumm method applied shock waves. Journal of Computational Physics, 215(2))402-416, 2006. ISSN 0021-9991. doi: https://doi. org/10. 11t.10000//..cc.200..0.0.
URL hittps://wwe www.:ssciinccrre.........m.. www/science
art 0et.ccee/p1//00222999115550
R. A. Snavely, M. H. Key, S. Hatchett, T. E. Cowan, M. Roth, T. W. Phillips, M. A. Stoyer, E. A. Henry, T. C. Sangster, M. S. Singh, S. C. Wilks, A. MacKinnon, A. Offenberger, D. M. Pennington, K. Yasuike, A. B. Langdon, B. F. Lasinski, Johnson, M. D. Perry, and E. M. Campbell. Intense high-energy proton beams from vetawatt-lasee irradiation of solids. Phys. Rev. Lett., (5(14))2945-2944) 10 2000. doi: 10.1 10.P 85..555
URL htttp:://iinn
aps org. wwtp///g//ooooo 1103 www3:/1ysseeeeett..ccmmmm 85.885..95 85.245
Meng Tang, Yimin Liu, and Louis J Durlofsky. A deep-learning-based surrogate model for data as- similation in dynamic subsurface flow problems. Journal of Computational Physics, 413:109456, 2020.
Kiwon Um, Robert Brand, Yun Raymond Fei, Philipp Holl, and Nils Thuerey, Solver-in-the-lopp: Learning from differentiable physics to interact with iterative pde-solvers. Advances in Neural Information Processing Systems, 33:611 1-222, 2020.
Pantelis R Vlachas, Georgios Arampatzis, Caroline Uhler, and Petros Koumoutsakos. Multiscale simulations of complex systems by learning their effective dynamics. Nature Machine Intelli- gence, 4(4):359-366, 2022.
S. C. Wilks, A. B. Langdon, T. E. Cowan, M. Roth, M. Singh, S. Hatchett, M. H. Key, D. Pen- nington, A. MacKinnon, and R. A. Snavely. Energetic proton generation in ultra-intense laser-solia interactions. Physics of Plasmas, 542-5444,,4 2001. doi: 10.1063/1.133369.. URL 00..1.....66...0 1063 13336697
https:://di..or//00

---

Under review as a conference paper at ICLR 2023
D Winske. Hybrid simulation codes with application shocks and upstream waves. Space Science Reviews, 42(1):53-66, 66, 1985.
Tailin Wu, Takashi Maruyama, and Jure Leskovec. Learning to accelerate partial differential equa- tions via latent global evolution. Advances in Neural Information Processing Systems, 35, 2022a.
Tailin Wu, Qinchen Wang, Yinan Zhang. Rex Ying, Kaidi Cao, Rok Sosič, Ridwan Jalali, Hassan Hamam, Marko Maucec, and Jure Leskovec. Learning large-scale subsurface simulations with hybrid graph network simulator. Proceedings of the 28th ACM SIGKDD Conference on Knowl- edge Discovery and Data Mining (KDD '22), 2022b.
AB Zylstra, OA Hurricane, DA Callahan, AL Kritcher, JE Ralph, HF Robey, JS Ross, CV Young, KL Baker, DT Casey, et al. Burning plasma achieved in inertial fusion. Nature, (7894))) 2022.
12

---

Under review as a conference paper at ICLR 2023
Appendix
A FIRST-PRINCIPLES SIMULATIONS OF LASER-PLASMA INTERACTIONS
A plasma is a collection of unbound, moving ions and electrons, interacting through the electro- magnetic (EM-) fields. It can be produced through for example the irradiation of a high-intensityy (1 > 10 W/cm²) laser pulse onto solid material such as a metal foil. Through the nonlinear in- terplay between the electrons, ions, and the EM-fields, these laser-plasma interactions can generate high-energy (> McV) ions for applications in material science (Patel al., 2003), imaging (Rygg et al. 2008), and medical therapy (Kroll et al. (022). To understand these nonlinear processes, is necessary to use kinetic treatment of the plasma.
A.1 PARTICLE AND CONTINUUM DESCRIPTIONS OF PLASMAS
One example of kinetic treatment of plasma is the particle-in-cell (PIC) method (Dawson 1983 Birdsall & Langdon 1991). In this method we solve the Klimontovich equation (Klimontovich 1967) for finite size particles, coupled to Maxwell's equations for the EM-fields. The numerical procedure consists of solving Maxwell's equations on a spatial grid using the current and charge densities that are obtained by weighting the discrete plasma particles onto the grid. The particles are then advanced via the Lorentz force associated with the EM-fields. This warticle-based simulation technique captures the kinetic microphysics of plasmas, and to the extent that quantum mechanical effects can be neglected, provides first- trinciprns description of plasma dynamics.
plasma can also be described exactly as a continuum, by fluid equations the evolution of the ve- locity moments of the distribution function f of the plasma particles (Mx (x) == I d of df v)v. the N-th order velocity moment). The fluid equations can derived from the kinetic equations and form infinite hierarchy of exact coupled conservation equations for each fluid moment (with the n-th moment depending explicitly on the (n + 1)-th). In practice, this infinite hierarchy needs to be truncated after the first few moments, through the so-called closure relation presses the evolution of the highest-orde order moment considered in terms of the lower-order moments. For plasma near local thermodynamic equilibrium, such the thermal population of the electrons and ions studied in our work, these closures allow us to describe the plasma ceff-consistently at larger spatial and temporal scales (relative to the PIC scales), using finite number of moments. Indeed, we use the first three moments the mass, momentum, and energy densities of the particles - to describe the dynamics of the thermal fluids, and learn their coupling with the kinetic (non-thermal) particles and the EM-fields.
relation that
A.2 LASER- LASER-PLASMA INTERACTIONS
The generation of high- ion beams from intense laser-solid interactions has been an active area of research due to the potential of producing high-energy. high-charge, high-current ion beams in much more compact systems than solid-state based accelerators (e.g. linear accelerators, cyclotrons). The enormous accelerating gradients (TV/m; ~5 orders of magnitude larger than solid-state based accelerators) can be sustained in plasma (Wilks et al. 2001), accelerating ions to MeV energies within millimeters.
For these laser intensities it is useful to define the normalized vector potential 40
0.85 / /
where Ån is the laser wavelength. The laser electric field can accelerate electrons to relativistic speeds in one cycle if ag > For ion acceleration, dolid-density target is typically used, for which the laser E-field accelerates the electrons in the small, skin depth, layer near the front surface, producing very energetic electrons. In addition to the acceleration near the front surface, these energetic electrons cross the dense target and escape into the vacuum on the rear side, setting up a strong charge separation E-field that accelerates the ions from the back surface. As result, the ions will be laminar and possess small divergence angle, and exhibit an energy spectrum typically characterized by an exponentially decreasing distribution (Snavely et al. 2000 Wilks et al. 2001).
13

---

Under review as a conference paper at ICLR 2023
A.3 SIMULATION PARAMETERS
In our PIC simulations we consider an intense laser interacting with planar, solid-density target. In our ID simulations a laser with frequency we is launched along the a direction from the left boundary and irradiates an electron-proton plasma (i.e. m, my 1836 m.). The laser pulse duration is = 15 fs, with intensity 2 & ag & 20. The plasma density follows a step-like profile with thickness jmm and electron number density 1023 cm- (considering laser wavelength of Jam), corresponding to a solid-density target such as metal foil.
The target is simulated with 1000 partieles per cell per species, and the total simulation domain of = 80 umm is resolved with a spatial resolution (cell size) of 0.03 c/un. The time step is chosen according the Courant-FFiidrrichs--Lww condition, and the system is evolved over 2000 time steps (or 8500 for ag 20). Periodic boundary conditions for both particles and fields are used.
A.4 COMPUTATIONAL CHALLENGES AND OPPORTUNITIES
Studying these laser- plasma interactions using the particle-in-cell (PIC) method requires resolving the fastest and smallest oscillations of the electrons in the plasmas. As a consequence, modeling these interactions with first-principles simulations is very computational demanding. For example, 3D one-to-one simulation of laser-plasma interactions typically involves evolving the dynamics of billions of (numerical) particles on a billion-cell grid, requiring millions of CPU hours to compute. In this work, we consider only one spatial dimension æ, while retaining the three dimensions in momentum. This is identical to simulating a particle distribution uniform in the y and z directions. See Appendix A for details of laser-plasma interactions and the PIC method.
It is important to note that in more realistic 2D and 3D geometries a significant enhancement of speed-up can be achieved, due to the much smaller proportion of the non-thermal particles. This is result of the finite spot size of the laser, which will only interact with finite volume of the plasma, and accelerate a small fraction of the particles to non-thermal. In a 3D simulation of these laser-plaee laser-plasma interactions, the non-thermal particles compose typically 0.1% of the total number of particles while encompassing > 50% of the system energy. Compared with the ID simulations presented in this work (~ 20%), one expects speed-up of > 1000x in 3D.
B DETAILS FOR SEPARATION METHOD
Here we describe the details of the separation method for preparing the labeled (thermal and non- thermal) data. The algorithm separates the particles (from the original simulations) into thermal and non-thermal populations. This is done by computing locally the moments of the particle velocity dis- tribution f (v) and considering non-thermal particles those with velocities v that exceed a threshold a'iin - a given multiple (ox) of the thermal velocity Whh (= the velocity spread o, for non-relativistic velocities: Cohen al. 2010 Fiuza al. 2011). At each time step, we study the velocity distribution function of the particles in the local neighborhood in space N₁ of each particle P,. Without knowing which particles belong to the thermal population priori, we begin by identifying population that is likely thermal at iteration 1. The velocity spread de, computed from this population provides a first estimate of the threshold ao,, allowing us remove particles with v ad, from the distribu- tion. The procedure is repeated (on the updated distribution) until a convergence of the value of the threshold is reached iteration N. The remaining particles now constitute the thermal population, characterized by the fluid velocity en (uy) V E A' and thermal velocity Vn, = Jen ((...) denotes the average value). These are the mean and sigma of the Gaussian. The corresponding threshold a'tin is then used to determine the population P; belongs to. Note that only P. has been given a label, the other particles in MG were only used to calculate vn and Vin-
Figure 7 illustrates an iteration in detail. From the example velocity distribution function shown in Fig. Aa), we recognize an absolute peak located at Vabs with height hass- Among the peaks (Un,, hy A hn < hhaa for a constant X, pick the one with Un, closest to zero the first guess of thermal population and denote the peak location as (Vo, hv) (Fig. 7p). Search down and outwards
from (No. ho) (i.e. in both directions of u with U4 > Uo v_) until the heights h4 = h(v = u4)
Yho and h_ = h(v v_) = Yhg (Fig. (7). The population enclosed by (olu, > v > v_ is
regarded as the population to estimate the threshold. Transform all the particles in A into the local
14

---

Under review as a conference paper at ICLR 2023
(d)
Vans
Tio-vshh)
XPow
69
Sx10*
.ho)
4x10
3x104
2x10*
Yhu
Lx10
Figure 7: The separation method identifies the thermal and non-thermal populations from the local velocity distribution of particles.
fluid frame according to en (Fig. 71; for non-relativistic velocities this amounts to i - 0/ Un) Calculate a. obtain the threshold (Fig. 7e).
The size of N₁, values of X, Y, and a are tunable parameters determined by the physics and fine- tuned empirically. We have found that Nf 60 cells, X 0.8, 0.1, and 3 and 5 for the water and laser-plasma systems lead to the most physically accurate separation.
The accuracy of the algorithm is evaluated by the pressure tensor P:j of the thermal population, where Ps = (m,uy) - (vz)(ry) characterizes the second-order velocity moments and a directly related how well the distribution described by Gaussian. Figure 7 exemplifies that indeed, for the identified thermal population of the ions for the laser-plasma system, the diagonal terms are all comparable and the dia-oiag terms are negligibly small. Namely, the velocity distribution can be described by Gaussian.
C FULL PIPELINE
C.1 COMPONENT DETAILS
C.1.1 COMPONENTS (A) AND (B)
For both M or MNT, there's 20 moment directional components that needs to be predicted - 10 for ions, 10 for electrons of the zero (1), first (3), and second (6) moment orders. Thus, for both components, we instantiate 20 models with the same hyperparameter configuration in far or fM We also train Component (1) with the forh-forward trick and multi-step loss, shown Brandstetter et al. (222) to greatly enhance capability of generalization.
C.1.2 COMPONENT (C)
In order to parameterize the distribution to sample, a three-dimensionnl Gaussian is taken to have mean Elu] as the first order moments in 1/t11 and covariance matrix E[(u - E[u])(u - -[u))"] e x3x3 informed by the second order moments in M To be able to sample, we require valid (positive definite) covariance matrix. order enforce this condition, we obtain the ground-truth L using the Cholesky decomposition E((u ((u))(u E(u))T] LLTT and train fas to predict approximate E((u Efu])(u E'u))" = LiT, and use the valid covariance matrix to sample a set of particles which then "injected". The cumulative set of injected particles constitute the PIC loop.
L. During rollout,
15

---

Under review as a conference paper at ICLR 2023
C.1.3 COMPONENT (D)
After obtaining the sample of injected particles, we subtract their moments from Mr++ to obtain the thermal population's 's moments M for the next time step.
C.1.4 COMPONENTS (E) (F) (G)
Each of these components by default use the respective components from the imitation solver. The imitation solver implements the equations used in the OSIRIS PIC simulation, and is tested match the simulation to machine precision error on any single timestep. These components run efficiently because only advance the much smaller sub-population of non-thermal particles (z, u) u)*+!" Figure Souure8lllusse illusttaae the full pipeline of our method, concretized into the laser paser-plasma simulation.
Fluid/
E, B
J
E, B
(9)
Current deposit field advance
(e) PIC particle advance
(x, uk
u]
(c)
Sample NT
(x, u
(x, u)
(a)
M
Fluid advance
M'
Calculate M (d)
NT Injection
time step
t+1
Figure 8: Schematic of LHPC for the multi-scale laser-plasma interaction, which is the application of the general pipeline (Fig. [ to the current problem. The components correspond almost exactly Eqs. la except current deposit and field advance are separated.
D ADDITIONAL RESULTS FOR MULTI-SCALE LASER-PLASMA INTERACTION
Here we provide additional results for our experiments, as shown in Tables and Table uses the same dataset split as Table [ but focus on rollout starting from & = 1200, where there is most injection and dynamical activity. From the table, we see that the conclusion is similar as Table in main text: LHPC achieves significant error reduction for key quantities of interest: EM field and MPIC (energetic particles).
To ablate different aspect of our model, we perform additional experiment with different train: test split, where the test dataset in the future the training (Table We take the 10 trajectory with the highest laser intensity that consists of T' 8480 time steps, where the training time-range range [1000, 1900] and the testing (rollout) time-range 1900 onwards. We explore an ablation study various strategies to the additional data [1800, 1900]. Namely, we compare finetuning with simply retraining the model on [1000, 1900]. We also use the multi-step loss with the push-forward trick, reported by [Brandstetter al.) (2022) and other works enhance the generalization of the model, and because it can accurately mimic the input distribution during rollout.
E ADDITIONAL DETAILS FOR ARCHITECTURE AND TRAINING
We use a convolutional network (CNN) as the base architecture for modeling fluid network I (Fig. 9) and injection network Oinject. For both networks, the same layers, with the same
16

---

Under review as a conference paper at ICLR 2023
Method
Component
Error & step
Error 10
Error @ step 20
Eroor @ step 50
Speed (s/stop)
GT Solver (full PIC)
9:21E-01
Field
4.02E-02
3,45E-01
6,45E-01
1,284+00
FNO: All-fluid
8.45E-02
M
8.91E-03
6.64E-02 02
9,58E-02
4,20E:01
Field
3.17E-02
2.49E-011
4.00E-01
3.80E-01
FNO: Bi-Gaussian
Myc
3.16E-02
1.31E-01
2.17E-01
4.40E-01
1.69E-01
M
0312E-03
5.85E-08 02
9.88F-02
2,98E-01
Field
7,391:03
4.271:-22
9.05E402
2,71E-01
Baseline: fl--flui
Myc
4.88E-02
M
5.23F:01
3.16F:02
6.62N:02
1,46:-01
Field
6.62E-03
4.631--2
1.99-001
2,98E-01
Baseline: Bi Gaussian
Myc
1.53E-02
4.95E-02
8.66E-02
4.52E-01
1.27E-01
M
S.22E-43
3.50E-02
7.18E-02
1.62E-01
Field
1.10E-03
9.277--3
1,391402
7.32E-022
I.HC. (no (no- coupling)
Myc
1.02:422
4.16E-02
1.0SE:01
M
6,22E-03
7.05E-02
1,48E-02
3.80E-01
Field
1,10E-03
8.08E-003
L077:00
7,33E-02
LHPC
Myc
1.024-02
1.15E-01
M
6.22E-03
4.31E-02
9.65E-02
2.86E-01
Table Results for laser-plasma interactions for time range t 1200-1250. Trajectory 5 (ag = 10) is held out for testing, and the model is trained on the other 9 datasets. We report performance Dataset 5 and rollout at = 1200, where there the most injection and dynamical activity. As shown, the model remains stable within this chaotic, unseen time range, showing generalization to an unseen trajectory.
Croeadaszi Sdlerda RC))
N0A
3BA
SOA
OGIA NEAI
DNAA NIAI
DOUL NIA
DIAA, NAA
DOUL NIAI
1-70000
5005-000
NGA
SDN
GLARE-S 5.E--7))
ELOME-01- 01E-00
ASE-01
2485 20
196 1aee
189) 1900
11.3 65361
CLABR 20 1080
ALTEE 3116
9318-40
0-3---1 206
2438-00
538-1065
AA335-0 4318-48
(LIEE42 19R8-23
12:12-002
14412/12 8815-22
18425-922 2438-311
1285-41
1901 1000
1831 1m0
11 3DC
(L149-00 1.10-40)
(1.19-43 1199-40
Table 3: Ablation study for multi-physics laser-plasma interactions for the 10th dataset that has highest laser intensity (ag 20). We compare amongst: (a) train only single-step model from [1000, 1800] vs. (b) from [1000, 1900] vs. (c) taking (a) and finetuning with single-step loss [1800, 1900] vs (d) taking (a) and finetuning with multi-step push-forward trick loss on [1800, 1900]. We compare with the All-fluid baseline, and report result from the best hyperparameter configuration. Each cell reports the relative L.2 error for the EM-field [ B¹), for M
kernel sizes of 1,1,3,7,3,1, with feature sizes of 64, allowing feature extraction and local information exchange among neighboring cells. For each feature of the moments, we use different feature head as shown in Fig. 9
Training. We use Adam (Kingma & Ba 2014) optimizer, with starting learning rate of 10-3, The training consists of two stages. the first stage, we train both gar,a and gM with single-step loss:
T = E, [eMtt
M(+1 [(Nttt,,M4) + e(Mtt) MC+1)
In the second stage, we fine-tune with predicting N = 4 steps into the future with push-forward trick (Brandstetter et al. 2022 that stops the gradient the input. The loss is given by:
L2 E
Σ
(
Σ
all(M M
For both losses, have
17

---

Under review as a conference paper at ICLR 2023
Nr+ 921,0 (sg((E, (ggh-++)))((((M++-))) ,1= 11,,,.... M M (g((M(tii))) sgg sg((((+ii))) ,1 1,2, ...,...N
(4)
(5)
Notice the hat notation in the input arguments, denoting that they are autoregressive prediction of the LHPC at the previous time step, but the gradient is stopped (the "sg" notation). Essentially, we rollout the pipeline of LHPC for multiple steps, and use it to provide realistic input that contains the rollout error, requiring the model to not only predict well, but able to adapt to noise due to the rollout error. We find that this significantly improve the long-term prediction performance. For the coefficient of the multi-step loss in Eq. 4 we set (ar1, 022 03, (t4 (1, 0.5, 0.1, 0.1) with decreasing weight for longer time steps. This put more emphasis on the single-step loss to make training more stable, and also have weight longer-term future improve long-term prediction.
For the loss function :.: in Eq. and
wC use
( v) 1/
(6)
We find that this achieves better performance than the alternative of MSE loss (with exponent of 2) and MAE loss (with exponent of 1). MSE will give very small gradient if the loss is small, and able to encourage that the prediction be exactly in the vacuum. On the other hand, MAE harder to train and do not penalize more for larger errors. Our choice of loss function E(,,) strikes a good balance between the two and enjoys the benefit of both loss functions.
For single-step and multi-step training, we both train 500 epochs, with and cosine learning rate scheduling (Loshchilov & Hutter 2016).
Species Feature Extractor
Conv Elu
Conv C
Conv C 64, Act: Elu
Conv 64, Act: Elu
Conv
Linear
Act:
Input: M' Field
parallel blocks (corresponds ions' output
features)
Electron Species Feature Extractor
Conv 7, C 64,
Conv 3, 64, Act:
Conv I, C 64,
Conv Act: Elu
Conv 3, 64,
Act:
Linear
Act: Elu
Act: Elu
parallel blocks (corresponds to output features)
Figure 9: Schematic of the architecture of Gavid.
18


Reviews:
[{"review_id": "xipFpviIPa", "reviewer": "Reviewer_gLTB", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a hybrid framework for modeling N-body systems. It uses a classical solver for the non-thermal particle distribution. The neural networks are used in two components: (1) in modeling the thermal part using a continuum representation, (2) in coupling the continuum model and the particle model. The proposed method has been shown effective in modeling laser plasmas. Overall, this paper is interesting as it combines classical physical approaches with neural networks to address a challenging and important physical problem. ", "summary_of_the_review": "Overall, this paper proposes an interesting idea of combining the classical solver of physical systems and neural network models, and evaluates this algorithm on a challenging problem. \nIt would be more convincing to compare with (1) different design of NN models (e.g., using NN for non-thermal part as well), and (2) different NN architecture. Also, it can be more convincing to show the effectiveness in multiple physical applications.\n", "strength_and_weaknesses": "Strengths\n1.\tThe proposed method combines classical approaches with neural networks models in addressing important scientific problems.\n2.\tA separate neural network is used to couple the classical solver for particle distribution and the NN model with continuum representation. \n\nWeaknesses\n1.\tWhy not use NN to simulate the non-thermal part? I assume a NN-based surrogate model for both thermal and non-thermal component can further accelerate the speed. \n2.\tIt would be better to evaluate the method in more than one application. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "mvk0i-QpCe", "reviewer": "Reviewer_mprf", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work studies a new learning system to model a non-equilibrium N-body system. In particular, the new method combines two learning components to jointly model the near-equilibrium (thermal) flow and far-from-equilibrium (non-thermal) particles. The experiments show that this hybrid model outperforms a model that only describes the system in a single mode (thermal or non-thermal). ", "summary_of_the_review": "Overall I think the main focus of this work is to solve a modeling problem in physics. But I don't have enough knowledge to judge the novelty and correctness of this contribution. On the machine learning side,  I feel that the proposed model is very typical and does not bring much innovation. I also worry that the work deviates too much from the main theme of this conference. ", "strength_and_weaknesses": "Strength:\n+ the motivation is clear, and the proposed method seems to work.\n+ The writing of the paper is generally clear. I can read the paper well though I don't have much background knowledge in physics. \n\nWeakness:\n\n- The novelty of the proposed method is limited in terms of machine learning techniques. This type of system is typical in machine learning. \n- This work may not be able to draw much interest from the wide audience of this conference, whose main focus is on innovations in learning systems. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "hUPFLS9LN-y", "reviewer": "Reviewer_qT1Y", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes to jointly formulate particle and continuum for model multi-scale, non-equilibrium, and no-body physical systems. Specifically,  this model contains three components, 1) a classical solver for kinetic particles, 2) a neural network for the evolution of thermal sub-population with fluid representation, and 3) an injection neural network from a fluid state to a particle state. \n\nCompared with existing methods on non-equilibrium, and no-body physical systems, it uses the hybrid representation instead of the uniform representation. Though the method is simple, it achieves a good balance between accuracy and speed, i.e. faster than classical particle solvers and more accurate than the deep-learning baselines.\n\n", "summary_of_the_review": "Balancing the strength and weaknesses, I think this paper would be better when more comparisons and analyses are conducted. \nBesides, the technical contribution of adding a neural network to learn particle-continuum coupling is limited. ", "strength_and_weaknesses": "Strength:  \n1）This paper focuses on an important and interesting problem that models the non-equilibrium, N-body nonlinear， which has the potential for complex physical systems analysis.\n2) The hybrid representation makes sense and works well in the simulations. \n3) The basic idea and technical details are clear.  More details of the network architecture are attached in the supplementary details.\n\nWeakness:\nThe main weaknesses of this paper lie in experimental comparison and technical contribution:\n1）There seem to be some existing methods in the field of nonlinear physical systems modeling, such as LED[1] and  Solver-in-the-loop[2]\nIt is suggested to compare the proposed method with these existing methods. \n2)  There are some methods to fuse the representation of different views, such as Cross-Attention and CCA. Do these information-fusing methods work well in physical systems modeling?\n3) The whole method contains three components.  Which one is most important? It is suggested to add a detailed ablation study.\n4) Using a neural network to learn particle-continuum coupling is not difficult. Did existing methods use the neural network for coupling? If yes, which is the advantage of this method? If not, it is better to analyze the effect of different deep-learning-based coupling methods.\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject