Learning Macroscopic Dynamics from Partial Microscopic Observations
Authors: Mengyi Chen
Qianxiao Li
Venue: NeurIPS 2024 poster (2024)
Keywords: Dynamical system
Neural closure model
Macroscopic dynamics
Partial observation
Primary area: machine_learning_for_physical_sciences

Abstract: Macroscopic observables of a system are of keen interest in real applications such as the design of novel materials. Current methods rely on microscopic trajectory simulations, where the forces on all microscopic coordinates need to be computed or measured. However,  this can be computationally prohibitive for realistic systems.  In this paper, we propose a method to learn macroscopic dynamics requiring only force computations on a subset of the microscopic coordinates. Our method relies on a sparsity assumption: the force on each microscopic coordinate relies only on a small number of other coordinates. The main idea of our approach is to map the training procedure on the macroscopic coordinates back to the microscopic coordinates, on which partial force computations can be used as stochastic estimation to update model parameters. We provide a theoretical justification of this under suitable conditions. We demonstrate the accuracy, force computation efficiency, and robustness of our method on learning macroscopic closure models from a variety of microscopic systems, including those modeled by partial differential equations or molecular dynamics simulations.

Learning Macroscopic Dynamies from Partial Microscopic Observations
Mengyi Chen', Qianxiao Li
Department of Mathematics, National University of Singapore',
Institute for Functional Intelligent Materials, National University of Singapore? chenmengy @u. nus edu, qiamxiaočnus edu.sg sg
Abstract
Macroscopic observables of system of keen interest in real applications such as the design of novel materials. Current methods rely on microscopic trajectory simulations, where the forces on all microscopic coordinates need to be computed or measured. However, this can be computationally prohibitive for realistic systems. In this paper, propose method to learn macroscopic dynamics requiring only force computations on a subset of the microscopic coordinates. Our method relies sparsity assumption: the force each microscopic coordinate relies only on a small number of other coordinates. The main idea of our approach to map the training procedure the macroscopic coordinates back the microscopic coordinates, on which partial force computations can be used as stochastic es- timation update model parameters. We provide a theoretical justification this under suitable conditions. We demonstrate the accuracy, force computation efficiency, and robustness of our method on learning macroscopic closure models from a variety of microscopic systems, including those modeled by partial dif- ferential equations or molecular dynamics simulations. Our code is available /ttps::/qqttus
Introduction
Macroscopic properties, including thestructural and dynamical properties, provide a way to describe and understand the collective behaviors of complex systems. In wide range of real applications, researchers focus mainly on the macroscopic properties system, egg. the viscosity and ionic diffusivity liquid electrolytes for Li-ion batteries (Dajnowiez usually depend on the whole microscopic system, e.g., the calculation of mean squared displacement requires all microscopic coordinates during the simulation. With growing simulation and experimental data, data-driven learning of macroscopic properties from microscopic observations has become an
2022). Macroscopic observables
active area of research (Zhang al. 2018 Wang
2019 Husic et 2020 Lee al.
2020 Fu
et al. 2023 Chen et al. 2024).
Accurate caleulation of macroscopic properties requires large-scale microscopic simulation. How- ever, accurate force computations on all microscopic coordinates for large systems are extremely
expensive (Jia al. 2020 Musaelian al. 2023). For example, in ab initio molecular simulations, accurate forces need to be calculated from density functional theory (DFT). The computational cost of DFT limits its application relatively small systems, typically ranging from few hundred atoms several thousand atoms, depending on the level of accuracy and computation resources (Hafner et al. 2006 [uuo al. 2020). This poses dilemma: Accurate macroscopic properties obtained from singe-scale microscopic simulation, but the computation of forces on all the microscopic coordinates extremely challenging.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Data Generation
Macroscopic Dynamics Identification
X
(z*, 2))
latent dynamics
encoder
microscopic force computer
(x)
(Ø (
loss
Figure 1: Overview of our method. Left. Data generation workflow. For each configuration x, forces subset of all the microscopic coordinates calculated by the microscopic force calculator. Right. Macroscopic dynamics identification. The macroscopic dynamics is mapped to the microscopic space first, then compared with the forces subset of the microscopic coordinates.
To solve the dilemma, the corresponding question is: Can we still obtain accurate macroscopic observables even though only access to forces subset of the microscopic coordinates? In this work, we develop method to learn the dynamics the macroscopic observables directly, while only forces subset of the microscopic coordinates are needed. Efficient partial computation of microscopic forces relies on the sparsity assumption, where the computation cost of forces on a subset of microscopic coordinates does not scale with the microscopie system size. To learn the dynamics of the macroscopic observables, we first map the macroscopic dynamics back to the microscopic space, then compare it with the partial microscopic forces. Our key idea is summarized in Fig.
Our main contributions are as follows:
We develop a novel method that can learn the macroscopic dynamics from partial computa- tion of the microscopic forces. Our method can significantly reduce the computational cost for force computations.
We theoretically justify that forces on a subset of the microscopic coordinates can be used as stochastic estimation update latent model parameters, even if the macroscopic observables depend on all the microscopic coordinates.
We empirically validate the accuracy. force computation efficiency, and robustness of our method through a variety microscopic dynamics and latent model structures.
2
Related Work
Learning from Partial Observations Several works have sought learn dynamics from partially
observed state & utilizing machine learning (Ruelle & Takens [971] Sauer et al. 1991 Takens 2006
Ayed et al. 2019; Ouala et al., 2020 Huang et al.) 2020; |Schlaginhaufen al. 2021: Lu et al. 2022 Stepaniants et al. 2023). For training, these methods reconstruct the unobserved state X first and (x, x). Our work assumes full state x but partial forces Furthermore,
model the dynamics
we do not model the dynamics state directly, but rather the latent space because the dimension of x would be extremely high for large systems.
Reduced Order Models By modeling the dynamies in the latent space and then recovering full states from them, reduced order models (ROMs) substitute expensive full order simulation with cheaper reduced order simulation (Schilders al. 2008 Fresea al. 2020 Lee Carlberg 2020
Hernandez et al.
2021
Fries al. 2022).

---

Our method can be thought to fall in the range of closure modeling. Unlike ROMs, we aim to model the dynamics of some given macroscopic observables directly and are not interested in recovering the microscopic states from the latent states.
Equation-free Framework The equation-free framework (EFF) has sought simulate the macro- scopic dynamics efficiently (Kevrekidis et al., 2003] Samaey et al. 2006 Liu et al., 2015). The EFF usually applied partial differential equation (PDE) systems, and the macroscopic observables are chosen to be the solution of the PDE at the coarse spatial grid. In EFF, the macroscopic observables depend locally on the microscopic coordinates, allowing the macroscopic dynamics to be directly estimated from the microscopic simulations performed in small spatial domains. In contrast, the macroscopic observables may depend globally on the microscopic coordinates in our method, and the macroscopic dynamics may not be easily estimated from microscopic simulations performed in small spatial domains.
Another difference is that our method explicitly learns the macroscopic dynamics, while EFF can bypass explicit derivation of macroscopic evolution law by coupling microscale and macroscale dynamics. During simulation, EFF still requires microscopic simulation to be performed in small spatial domains and for short times, but our method can enable fast macroscopic simulation without requiring any microscopic simulation However, for systems where the macroscopic evolution equa- tions conceptually exist but are not available in closed form, EFF can efficiently handle such but the learned dynamics in our method may involve approximation or statistical errors that are often challenging to estimate.
cases,
Problem Setup
We consider microscopic system consisting of n particles. Let the state of the microscopic system be x (x1, , x,) R. X; E R" .N mn, where x; TR. is some physical quantity associated with the i-th particle, such the position and velocity. Assume the dynamics of the microscopic system can be characterized by an ordinary differential equation(ODE):
dx(() dt
f(x(1))
where f(x) (fi (f), fx, E RN,, We will call x, the microscopic coordinate of the i-th particle and f; the force acting on the microscopic coordinate x,. In many real applications, we are interested in the dynamics of some macroscopic observables z² co(x)). Here o is given beforehand and describes the functional dependence z* on x. For example, 2 can be chosen to be the instantaneous temperature or mean squared displacement in Lennard-Joner system.
The goal is to learn the dynamics of x from microscopic simulation data. Existing methods that try to learn the macroscopic or latent dynamics require microscopic trajectories or forces all the microscopic coordinates for training (Champion et al.] 2019; Fries et al., [022] Fu et al., 2023] Chen et al.) 2024). The problem is: When the microscopic system size N is very large such that the force computations on all the microscopic coordinates are impossible, these methods are no longer applicable. Instead, method aims to learn from partial computation of microscopic forces.
Consider we are given a microscopic force calculator for computation of partial microscopie forces. Let the microscopic coordinate x be sampled from a distribution D. For each x, the microscopic force calculator S will first sample an n dimensional random variable I(x) = (x). .I, Px E {0, 1)" according to certain strategy. Next S will calculate the corresponding partial forces f1(x) == (fi, (x): fx)())) For notation simplicity sometimes we will simply write f(xx) as fr.
(x) indicate whether partial i is chosen to calculate the force or not. If particle chosen,
I, (x) = 1, (x)))))) = f, otherwise (x) = 0, (x) = O. We require the sampling strategy Px to
satisfy:
1. For each I(x) ~ Px. exactly n items are equal to I and the rest are 0.
2. Each particle can be chosen with equal probability p. i.e. ((I = 1) = pp P(, =
0) = 1 p,
n.
This means that the microscopic force calculator S can calculate forces on n p microscopic co- ordinates, and 0 < < 1 limits the computation capacity of the microscopic force calculator S.
3

---

The above requirement is consistent with real applications since it is difficult to calculate all the microscopic forces due to computational cost. Furthermore, for efficient calculation of the partial microscopic forces, we will assume f satisfies the following sparsity assumption:
Assumption: For given error tolerance e O, there exists constant M n, such that for any x D and E {I,-.-, ,n], we can always find an index set (xx) c {i 1, nn), [xx.)] M
which satisfies:
[[ ,,,) -
<E
(2)
Intuitively, the assumption implies that the computational cost of foree f. is independent of the microscopic system dimension N. Thus our microscopic force calculator S can compute partial forces in an efficient way. This assumption is prevalent in real-world applications. To better illustrate this, we give two examples here. The first example is about molecular dynamics. In molecular dynamics, each x; represents the position r; and velocities v, of the i-th atom, ie., x; = (r,, V;) E R. Then Eq. (T) becomes the Newton's 's law of motion:
dr; Vi
(3)
dt
dv; dt
rn),
my
is common to limit the range of pairwise interactions to cutoff distance (Allen et al. 2004 Zhou & Liu 2022 Vollmayr-Lee/ ).20). To calculate the force atom, we only need to consider its interaction with other atoms that are within the cutoff. The second example is about systems modeled by partial differential equation (PDE), We consider time-dependent PDE and apply finite difference scheme to discretize the spatial derivatives. Then, the resulting semi-discretized equation takes the form of Eq. ( and each x, is the value at the i-th grid. f, only depends on those grids that used for finite difference approximation of the spatial derivatives.
Let the training data generated by the microscopic force calculator S be (x', Fiox)k=1,... K. The data generation procedure provided in Algorithm [ We will introduce the next section how we can learn the macroscopic dynamics from the training data with partial forces.
4 Method
Existing works for macroscopic dynamics identification consist of two parts: dimension reduction and macroscopic dynamics identification (Fu et al. 2023) Chen et al., 2024). We will follow these two parts. We start with most standard parts of closure modeling with an autoencoder, next, we turn to the main difficulty of macroscopic dynamics identification from partial forces.
4.1 Autoencoder for Closure Modeling
We will use an autoencoder to find the closure 2 (xxx to z
-x*(xx such that z = (z", 22) forms a closed system. Here we define z as forming closed system if its dynamics à depends only on not any external variables. Note that x* (xx),, yo* determined beforehand and contains no trainable parameters. This ensures 2* represents the desired macroscopic observables and remains unchanged during the training of the autoencoder.
Denote the encoder by 4p (40*. b) and the decoder by æb, we will minimize the following reconstruction loss:
Cree = R Σ ||x sp(x)) )|3
(5)
We also want (' T to be well-conditions (see Section 4.2). then we impose constraints on the condition number of sp (x))
Loond R ΣI |x(()(((((()))))) (x0((=(()))9((xx)))) - 1
(6)
By enforcing (p'("))<p (x)) ) to be well-conditioned, we are also enforcing (xxx)) RdxN, & N to have full row rank, which will be used later. The overall loss to train the autoencoder is
LAE
Lres + Noond Loond

---

Algorithm Data generation.
Algorithm 2 Training procedure.
Require:
Require:
D:configuration distribution
{x (x (=x..) Ki: data
S: microscopic force calculator
B: minibatch size
K: training data size
0y: model parameter
P: partial index sampling strategy
opt: optimizer
1: for to K do
1: while stopping criterion is not met do
2:
xt~
2: sample J = {1. KKIJJ B
3:
((x*)>P= calculate fi(x))x (x') using S
3: Calculate Lx.p Eq.
12) with fx', fo (x))
4:
4: 0z+1 opt(O,, V.x.p))
return ( (x' (i-,,.
return model parameter
Acond a hyperparameter to adjust the ratio of foood and is chosen to be quite small in our experiments, e.g., 10 or 10 6, The aim of training the decoder v is help the discovery the closure variables. The decoder ab will not be used for further macroscopic dynamics identification. To facilitate comparison between models tainted with all and partial forces, will train the autoencoder first and freeze it for macroscopic dynamics identification.
4.2 Macroscopic Dynamics Identification
We address the diffieulty of learning from data with partial microscopic forces. Substitute Z (x) into equation Eq. (1) and make use of chain rule, we get the dynamics of z:
dz dt ('()xx(xx)) z(0) (x(x))
(8)
here we use yo'(x) to denote the Jacobian (x)((x) notation simplicity. If the dynamics of closed, the right-hand side of Eq. (8) will only depend on z, and we parametrize it using neural network ge (z) s sp (x)) Since we are only interested in macroscopic dynamies identification,
the loss would be naturally defined on the macroscopic coordinates:
(0) : Σ || (x' f(x') ge (z())
Eq. (9h is used commonly in existing work (Champion et al. 2019 Fries et al. 2022 Bakarji et al. 2022
Park et al. 2024).
The main difficulty of La is that it includes the matrix-vector product f(x)) Note that the
i-th entry of yp' (x) can be written as Σy P (x) and is difficult find an unbiased estimation of the i-th entry using a subset of (f, (x)),-1.... .N- Thus the accurate calculation of fz requires the forces (f, (x) j-),,,, ,N on all the microscopic coordinates.
The main idea of our method is to map the loss the macroscopic coordinates back the microscopic coordinates:
Lx (
1 K K (==]1(((')) [|f(x*)) (xd((x))) (g
(10)
where (xp((x))) Nxd is the Moore-Penrose Penrose inverse. Since px(xt) is of full row rank, (xp((x)) (x in fact the right inverse of ('(xx", ie.. ('(x')(vp(xx")) is identity matrix. Below will show our main theoretical result:
Theorem 1. Assume for any x ~ D, the eigenvalues of ('()))x' are lower bounded by b and upper bounded by bz, 0 by & bz. Then:
()) + C) < (x( & (2( +C)
(11)
here C does not depend on hence does not affect the optimization.
The proof relies singular value decomposition of o'(xx) and we provide the full proof in Ap- pendix A.1 Theorem states that by minimizing (x(O), we are actually narrowing the range of Cz ()) Hence want bj and by to be close possible, this is the reason why we constrain the
5

---

condition number of ( in Eq. 6 In the very special case where b1 b2, minimizing
(0) just equivalent minimizing C, (0).
Note that in loss Lx. the term ||f(x) (x)) go (z)ll can be rewritten as شーー1 | (x)
(oo (z))l and , - [x) (x)())), [x) (z)il can be regarded as its unbiased stochastic
estimation. Then, we can introduce our loss defined for partial forces:
xx pk Σ (
(40( Mixe Bo Bo
(12)
Here a constant 1/p is multiplied to Cx.p to guarantee:
Ext,-- x )xxx)),- _(x*) -(x*)xxxxxxxxxx (x. Bx,,,...
x (x(
(13)
We provide the full proof of Eq. (13) in Appendix A.2 By training with the model with Lx.p- use data with partial forces as stochastic estimation to update model parameters. The full training procedure is provided Algorithm
Note that in Algorithm D during the data generation, I(x') is also sampled from its distribution. generated. In the limit, the
Thus, Lx.p is deterministic once the samples {x', (x)x)(x")) estimation is unbiased:
Theorem 2 (informal). Let (x) = TELx(0), 0* arg (0), 0k,p arg ming (0),
then under certain conditions:
[x(0 (x*) a.e. 0
(14)
The proof utilized Rademacher complexity and a crucial assumption used in the proof is the uniform boundedness of Lx.p- The formal version of Theorem and the complete proof is provided in Appendix A.3 Theorem theoretically justifies the expected risk (xp) at the optimal parameter found by Cx.p. converges to the optimal expected risk (x* as goes to infinity.
Experiments
In this section, experimentally validate the accuracy, force computation efficiency, and robustness through a variety of microscopic dynamics.
5.1 Force Computation Efficiency
We first consider one-dimensional spatiotemporal Predator-Prey system, mainly to validate the correctness and force computations efficiency our method.
Predator-Prey System
The simplified form of the Predator- Prey system with diffusion
Murray
2003 is
du
Eu
ōt u(1 - v) D
2I1
(15)
du av(u b) +
22
I E = [0,1], te (0,
where denote the dimensionless populations of the prey and predator respectively, a, b, D are three parameters. The complex dynamics Predator- Preyy interaction, including the pursuit of the predator and the evasion of the prey in ecosystems, can be described by Eq. (15).
We discretize the spatial domain of Eq. 15) into 50 uniform grids with x; 0.02, & i & 50. Let u(t) (m(21, ..... u(Iso, t)), v(t) (v(x1, ... (. t)), then (u(t), v(t) 10000 are treated as the microscopic states. After approximating the spatial derivatives in Eq. 15 with the finite difference method, we consider the semi-discrete equation which is an N 100 dimensional ODE as our microscopic evolution law (see Appendix[B..
A), Ar
We choose the macroscopic observable of interest to be z predator and the prey's 's population:
(ū. G), the spatial average of the
50
50
i
u u(x,,)) t),
8
D xr, t)
(16)
50
50

---

We find another 2 closure variables using the autoencoder, then the total dimension of the latent space is 4. We choose D to be the trajectory distribution of the state x. For the data generation with partial forces, given x we randomly choose forces on 100 p microscopic coordinate. For example, if p 1/5, then for each configuration, forces on 20 coordinates are calculated for training.
Results Fig. 2 shows the results on the test dataset which consists of 100 trajectories (for more detials, see Appendix Table 2). For models trained with partial microscopic forces, we report the equivalent number of training data with full forces throughout the paper. For example, for model trained with px = 1/5) on 3 x 10a data, we will report the number of training data to be x 10 x 0.2 = 600. The test error is defined to be the mean relative error of the macroscopic observables between the ground truth and the predicted trajectories in Ap- pendix Eq. 43)
4x
10-2
(xpp(p=3/4)
fxpp(p=122
Test error
(x=(p=114)
(p-(pp=1/5)
10-3
103
104 # of training data
Figure 2: Mean relative error on the test dataset of the Predator-Prey system. The black dashed line represents test error = 3 x 10-3
First, we observe that the mean relative error of all the models is around 10-4 when the number of train- ing data is large enough. This tells us that training with Cx.p is correct and accurate, which is consistent
with Theorem 2 The predicted trajectories fit quite well with the ground truth trajectories (see Appendix Fig. and Fig. 7
We can conclude from Fig. that, under the same number of training data, fx.p with smaller p (1/4, 1/5) performs better. Similarly, to achieve the same performance, fx.p with smaller p requires less training data. We set the error tolerance to be Gpo training data is required to reach the error tolerance. In Fig. 2 the r-coordinate of the intersection point between the black dashed line and the other curves indicates the minimum data size required If we arrange each model according to their minimal required training data, then Cp = 1/5) = (p 1/4) < (p = 1/2) Lp 3/4) < Lx. Model trained with (x requires less data to reach Cul, equivalently, less force computations. This validates the force computation efficiency of our method. One explanation could be that there are many redundant information in the forces acting all the microscopic coordinates. By using partial microscopic forces, Lx.p can explore more configurations x given the same size of training data, thus can make use of more useful information. Another observation from Fig. 2 is that the training data size increases, the gap between models trained with different narrows down. This is because as more data provided, these data contain almost all the information of the Predator- Peeey system, thus more information will not lead to significant improvement.
10 and investigate how much
1/5)
5.2 Robustness to Different Latent Structures
Having validated the correctness and force computation efficiency of our method, we are ready to apply our method variety of latent structures. We will tackle the Lennard-Jones system this subsection, and validate the robustness of our method to different latent model structures. Three latent model structures are considered: MLP, OnsagerNet (Yu et al. 202I). GFINNs (Zhang et al. (022).. Both the OnsagerNet and GFINNs endow the latent dynamical model with certain thermodynamic structure to ensure stability and interpretability. The specific implementations of these two models slightly different.
The Lennard-Jones system is widely used in molecular simulation study phase transition, crystallization and macroscopic properties of system (Hansen & Verlet 1969 [Bengtzelius] 1986 Lin et al.) 2003 Luo al. 2004). The Lennard-Jones potential describes the interaction between two atoms and] through the potential of the following form:
Lennard-Jones System
Vi((r) = { 4ccs((ar//))2 -aa)))))))(000000 if r & Yeuls
(17)
if r > Teut-

---

Table 1: Summary of the results on each system. Results of the Predator-Pres Preyy and Lennard-Jone (small) system are taken from Section 5.1 Section 5.2 For each system, Lz and Lx.p are trained with the same size of data.
Micm dim N'
Ohservables
Latent dim d' Partial lahels
Axcp
Predater- Prey system
N,
3.19 sa x10-
2nu x10
Allen-Cahn system
free energy E(e)
693
x10-
2r9 x10-
Leunard-Jones system (small)
4800
temperature T
32
1/16
4,45 12m x10
1.17 10
Lennard-Jones system (large)
3U7200
temperature T
1/1024
4,96
uue x10
Az cxpop 1/16)
All the results in this experiment will be shown in reduced Lennard-Jones units. We consider three- dimensional Lennard-Jones fluid with Natons = 800 atoms of the same type. The microscopic state con- sists of the positions and velocities of the 800 atoms, thus the microscopic dimension is N 4800. We simulate the Lennard-Jones system under NVE en- semble using LAMMPS (Thompson al. 2022
10-22
Test
10-3
We choose the instantaneous temperature (T) as our macroscopic observables:
MLP
OnsagerNet
GFINNs
Nagu
miuf
Figure 3: Results on the Lennard-Jones sys- tem with 800 atoms and N 4800. Forces on atoms are used to train Cx.p for all the latent model structures. Each model is trained with ten repeats.
X
(18)
3 3(N Nattoms
2
here is the velocity of the i-th atom and my = We find another 31 closure variables using the autoencoder, then the latent dimension is 32. In our experiment, we also adopt the trajectory distribution of microscopic statex for D. For data generation with partial forces, we choose p = 1/16. For each x we randomly choose 50 atoms for force computations.
Results All the models are trained with the same size of data. Fig. 3 shows the test error on 10 test trajectories. The test errors of using (p 1/16) are relatively small (~ 10 10-"), which validates the accuracy of our model on the Lennard-Jones system. It is easy to observe from Fig. thtt for all the latent model structures, models trained with Cx.p can always outperform those trained with Lz This validates Lx,p is robust over different latent model structures.
5.3 Robustness to Different Microscopic dynamics
We have already validated the accuracy and force computation efficiency of fxx.p on the Predatory- Prey system and the Lennard-Jones system, but their microscopic dimension is still not big enough. In this subsection, we focus on the robustness of our method different systems including those with much larger microscopic dimension. We will consider two large systems: the Allen-Cahn system and larger Lennard-Jones system with 51200 atoms.
The Allen-Cahn equation is widely used to model the phase transition process in binary mixtures (Allen & Cahn 1979 Del Pino et al.) 2008 (Shen & Yang 2010; Kim et al.) 2021 Yang et al.) 2023). We consider the 2-dimensional Allen-Cahn equation with zero Neumann boundary condition on bounded domain:
Allen-Cahn System
d:== = 2 - z F'() on Ω = [0,1] x [0.1]
(19)
2n = 0 on 2n,
where v--1 S S 1) denotes the difference of the concentration of the two phases. F(v) is usually
chosen to be the double potential taking the form of F(v) = (02 -1)2. The Allen-Cahn equation
is the 2 gradient flow of the free energy functional E(v) R in Eq. 20 (arttlls 2015).
E(v) = S ( F(u) VulB dr dy
(20)
8

---

The free energy functional z" = E(v) is a macroscopic observable of wide interest, hence we choose (()) as our target macroscopic observable.
The spatial domain is discretized into 200 x 200 grids, then the dimension of the microscopic system is N = 40000. We find another 31 closure variables using the autoencoder hence the total dimension of the macroscopic system is 32, which is much smaller compared to the dimension of the microscopic system. We consider D be the trajectory distribution of x. We choose p 1/25, each time the forces on 1600 grids are calculated for training Lx.p-
Lennard-Jones System (large) To further demonstrate the capacity of our method, we scale up the Lennard-Jones system in Section 5.2 encompass 51200 atoms, then N 307200. The size of the simulation box is increased from 10 x 10 x 10 to 40 x 40 40 to keep the density unchanged.
Results For a summary of the experiments and the results, we refer the reader to Table [ Note that for the Lennard-Jones system (large), we still use the forces on 50 atoms for training, thus = 1/1024. For the training of Cx.p (p= = 1/ 10144,) only 5000 configurations with partial forces are used due to memory limit, which equivalent to 5000/1024 a 5 training data with forces on all the atoms. Obviously, training data with size 5 is way too small, hence the results of Lg are not reported for this system.
From the results shown in Table one can observe that for a variety of problems, including those modeled by partial differential equations molecular dynamies simulations, Lx.p can always outperform Lz This shows the robustness of our method to a variety of systems. Moreover, the success of our method on the Lennard-Jones system (large) demonstrates the ability and efficiency of our method when scaled to very large systems.
We also compare the number of force computa- tions that are required for models trained with Cg. and Cx.p to reach test eπrоr Col = 3 x 10 Fig. 4 shows the results on the Lennard-Jones sys- tem with different sizes. Lennard-Jones system with 800, 2700, 6400, 21600 atoms are considered and the density is 0.8 for all the systems. The number of force computations here refers to the total number of forces on atoms that are used. For example, if Lx.p uses 100 configurations to train, and for each configuration, forces on 50 atoms are calculated, then the number of force computations would be 100 x 50 = 5000. From Fig. we can observe that as the system size increases, the number of force computations required by fa continues increasing. In the experiment, we find that the number of training data does not change a lot. The increase in the number of force compu- tations is mainly due to the system size increases, then for each configuration, more force computations are required. However, the number of force computations even decreases a bit. One possible explanation is that, as the system size increases, the dynamics become less fluctuating and are easy to learn.
force computations
10?
Lz
10
Lx.p
10a
5000
10000 15000 20000 of atoms
Figure 4: Number of force computations re- quired to achieve God = 3 10-3 on Lennard- Jones system with different sizes. Forces on 50 atoms are used train Lx.p for systems of different sizes.
6 Conclusion
We present a framework for modeling the dynamics of macroscopic observables from partial compu- tation of microscopic forces. We theoretically and experimentally demonstrate the accuracy, force computation efficiency, and robustness of our method through different problems. Finally, we apply method to very large Lennard-Jone system which contains 51200 atoms.
While our method can learn the macroscopic dynamics from partial computation of microscopic forces, it relies on the sparsity assumption. For systems that do not satisfy the sparsity assumption, the calculation of partial computation of microscopic forces is not efficient, thus it is not beneficial to learn from partial forces computation. For example, in the McKean- Vlasov system, the force on

---

each microscopic coordinate depends on the collective behavior of all the other coordinates Mééléadd 1996).
Another limitation the structure of the autoencoder. For particle systems such as the Lennard-Jones system, ideal encoder should be permutation-invariany Currently, we use MLP for the encoder, which can be improved. Additionally, our method assumes the microscopic state to be sampled from a distribution D. We choose D to be trajectory distribution in the experiments, but reality trajectory distribution of large systems may be impossible to obtain. Active learning is commonly applied to efficiently select microscopic configurations for training (Ang et al. 2021 Zhang et al. 2019 Farache et al., 2022] Kulichenko et al. 2023] Duschatko et al., 2024). It is of interest to combine active learning and our proposed method overcome the difficulty of the choice of D. Furthermore, our method assumes the microscopic dynamics to be deterministic, another future direction could be generalizing our method to stochastic systems.
Acknowledgments
This research is supported by the National Research Foundation, Singapore under its Al Singapore Programme (AISG Award No: AISG3-RP-2022-028).
References
Michael P Allen et al. Introduction to molecular dynamics simulation. Computational soft matter: from synthetic polymers proteins, 23(1):1-28, 2004.
Samuel M Allen and John W Cahn. A microscopic theory for antiphase boundary motion and its
application to antiphase domain coarsening. Acta metallurgica, 27(6):1085-1095 1979.
Shi Jun Ang, Wujie Wang, Daniel Schwalbe-Kode Simon Axelrod, and Rafael Kōmez-Bombarellff Active learning accelerates ab initio molecular dynamics reactive energy surfaces. Chem, 7(3); 738-751, 2021.
Ibrahim Ayed, Emmanuel de Bézenac, Arthur Pajot, Julien Brajard, and Patrick Gallinari. Learning dynamical systems from partial observations. arXiv preprint arXiv: /902. 136, 2019.
Joseph Bakarji, Kathleen Champion, Nathan Kutz, and Steven I. Brunton. Discovering gov- erning equations from partial measurements with deep delay autoencoders. arXiv preprint arXiv:2201.05136. 2022.
Sören Bartels. The Allen-Caie Equation, pp. 153-182. Springer International Publishing, Cham, 2015.
U Bengtzelius. Dynamics of lennard-jones system close to the glass transition. Physical Review A, 34(6):5059, 1986.
Kathleen Champion, Bethany Lusch, Nathan Kutz, and Steven L Brunton. Data-driven discovery of coordinates and governing equations. Proceedings of the National Academy of Sciences, 116(45): 22445-22455 2019.
Xiaoli Chen, Beatrice W Soh, Zi-En Ooi, Elconore Vissol-Gaudin, Haijun Yu, Kostya Novoselov, Kedar Hippalgaonkar, and Qianxiao Li. Constructing custom thermodynamics using deep learning. Nature Computational Science, 4(1):66-85, 2024.
Steven Dajnowicz, Garvit Agarwal, James M Stevenson, Leif D) Jacobson, Farhad Ramezanghorbani, Karl Leswing, Richard A Friesner, Mathew D Halls, and Robert Abel. High-dimensional neural network potential for liquid electrolyte simulations. The Journal of Physical Chemistry B, 126(33): 6271-6280, 2022.
Manuel Del Pino, Michal Kowalczyk, and Juncheng Wei. The toda system and clustering interfaces in the allen-cahn equation. Archive for rational mechanics and analysis, 190(1):141-18) 2008.
Blake Duschatko, Jonathan Vandermause, Nicola Molinari, and Boris Kozinsky. Uncertainty driven active learning of coarse grained free energy models. npj Computational Materials, 10(1):9, 2024.
10

---

David E Farache, Juan C Verduzco, Zachary D McClure, Saaketh Desai, and Alejandro Strachan. Active learning and molecular dynamics simulations find high melting temperature alloys. Computational Materials Science, 209:11 1386, 2022.
Stefania Fresca, Andrea Manzoni, Luca Dedè, and Alfio Quarteroni. Deep learning-baae based reduced order models in cardiac electrophysiology. PloS one, 15110)a0239416, 2020.
William D Fries, Xiaolong He, and Youngsoo Choi. Lasdi: Parametric latent space dynamics
identification. Computer Methods in Applied Mechanics and Engineering. 399:115436, 2022.
Xiang Fu, Tian Xie, Nathan J Rebello, Bradley Olsen, and Tommi S Jaakkola. Simulate time- integrated cuuse-grained molecular dynamics with multi-scal graph networks. Transactions on Machine Learning Research, 2023.
Jürgen Hafner, Christopher Wolverton, and Gerbrand Ceder. Toward computational materials design: the impact of density functional theory on materials research. MRS bulletin, 31(9):659-66 -6866,996666-- 2006.
Jean-Pierre Hansen and Loup Verlet. Phase transitions of the lennard-jones system. physical Review, 184(1):151, 1969.
Quercus Hernandez, Alberto Badias, David Gonzalez, Francisco Chinesta, and Elias Cueto. Deep learning of thermodynamics-awaar reduced-order models from data. Computer Methods in Applied Mechanics and Engineering, 379; 13763, 2021.
Zijie Huang, Yizhou Sun, and Wei Wang. Learning continuous system dynamics from irregularly- sampled partial observations. Advances Neural Information Processing Systems, 33:16177- 16187, 2020.
Brooke E Husic, Nicholas E Charron, Dominik Lemm, Jiang Wang, Adrià Pérez. Maciej Majewski, Andreas Krämer, Yaoyi Chen, Simon Olsson, Gianni de Fabritiis, et al. Coarse graining molecular dynamics with graph neural networks. The Journal of chemical physics, 153(19), 2020.
Weile Jia, Han Wang, Mohan Chen, Denghui Lu, Lin Lin, Roberto Car, Weinan E, and Linfeng Zhang. Pushing the limit of molecular dynamics with ab initio accuracy to 100 million atoms with machine learning. In SC20: International conference for high performance computing, networking, storage and analysis, pp. 1-14. IEEE, 2020.
Ioannis G Kevrekidis, C William Gear, James M Hyman, Panagiotis G Kevrekidis, Olof Runborg. Constantinos Theodoropoulos, et al. Equation- free, coarse grnse-grained multiscale computation: en- abling microscopic simulators to perform system-level analysis. Commun, Math. Sci, 1(4):715-762, 2003.
Yongho Kim, Gilnam Ryu, and Yongho Choi. Fast and accurate numerical solution of allen-cahm equation. Mathematical Problems in Engineering, 2021:1-12, 2021.
Maksim Kulichenko, Kipton Barros, Nicholas Lubbers, Ying Wai Li, Richard Messerly, Sergei Tretiak, Justin S Smith, and Benjamin Nebgen. Uncertainty-driven dynamics for active learning of interatomic potentials. Nature Computational Science, 3()):200-339, 2023.
Kookjin Lee and Kevin Carlberg. Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders. Journal of Computational Physics, 404: 108973, 2020.
Scungjoon Lee, Mahdi Kooshkbaghi, Konstantinos Spiliotis, Constantinos Siettos, and Ioannis G Kevrekidis. Coarse-scale pdes from fine-scale observations via machine learning. Chaos: An Interdisciplinary Journal of Nonlinear Science, 30(1). 2020.
Shiang- Tai Lin. Mario Blanco, and William A Goddard III. The two-phase model for calculating thermodynamic properties of liquids from molecular dynamies: Validation for the phase diagram of lennard-jones fluids. The Journal of chemical physics, 119(22): 119/22 1792-1182088 2003.
Ping Liu, Giovanni Samaey, C William Gear, and Ioannis G Kevrekidis. On the acceleration of spatially distributed agent-based computations: A patch dynamics scheme. Applied Numerical Mathematics, 92:54-69. 2015.

---

Peter Y Lu, Joan Ariño Bernad, and Marin Soljačić. Discovering sparse interpretable dynamics from partial observations. Communications Physics, 5(1):206, 2022.
Sheng-Nias Nian Luo, Alejandro Strachan, and Damian C Swift. Nonequilibrium melting and erystalliza- tion of a model lennard-jones system. The Journal of chemical physics, 120044::164001116: 2004.
Zhaolong Luo. Xinming Qin, Lingyun Wan, Wei Hu, and Jinlong Yang. Parallel implementation of large-scale linear scaling density functional theory calculations with numerical atomic orbitals in honpas. Frontiers in Chemistry, 8:589910, 10, 2020.
Sylvie Méléard. Asymptotic behaviour of some interacting particle systems; McKean Vlasov and Boltzmann models, pp. 42-95. Springer Berlin Heidelberg, Berlin, Heidelberg, 1996. ISBN 978-3-540-68513-55 doi: 10.1007/BB600993177 BFb0093177
JD Murray. Multi-specier waves and practical applications. Mathematical Biology: II: Spatial Models and Biomedical Applications, pp. 1-70, 2003.
Albert Musaelian, Simon Batzner, Anders Johansson, and Boris Kozinsky. Scaling the leading accuracy of deep equivariant models to biomolecular simulations of realistic size. In SC23: International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 1-12. IEEE, 2023.
Said Ouala, Duong Nguyen, Lucas Drumetz, Bertrand Chapron, Ananda Pascual, Fabrice Collard, Lucile Gaultier, and Ronan Fablet. Learning latent dynamics for partially observed chaotic systems. Chaos: An Interdisciplinary Journal of Nonlinear Science, 30(10), 2020.
Jun Sur Richard Park, Siu Wun Cheung, Youngsoo Choi, and Yeonjong Shin. tlasdi: Thermodynamics- informed latent space dynamics identification. arXiv preprint arXiv:2403.05844 2024.
David Ruelle and Floris Takens. On the nature of turbulence. mathématiciens de StrasbourgRCP25, 12:1-44, 1971.
Les rencontres physiciens-
Giovanni Samaey, Ioannis G Kevrekidis, and Dirk Roose. Patch dynamics with buffers for homoge- nization problems. Journal of Computational Physics, 213(1):264-287, 2006.
Tim Sauer, James A Yorke, and Martin Casdagli. Embedology. Journal of statistical Physics, 65: 579-616, 1991.
Wilhelmus HA Schilders, Henk A Van der Vorst, and Joost Rommes. Model order reduction: theory, research aspects and applications, volume 13. Springer, 2008
Andreas Schlaginhaufen, Philippe Wenk, Andreas Krause, and Florian Dorfler. Learning stable deep dynamics models for partially observed or delayed dynamical systems. Advances in Neural Information Processing Systems, 34:1 11:118888 18701180111111 1882, 2021.
Jie Shen and Xiaofeng Yang. Numerical approximations of allen-cahn and Discrete Contin. Dyn. Syst, 28(4):1669-166) 2010.
equations.
George Stepaniants, Alasdair D Hastewell, Dominic J Skinner, Jan F Totz, and Jöm Dunkel. Discover- ing dynamics and parameters of nonlinear oscillatory and chaotic systems from partial observations. arXiv preprint arXiv:2304.0488 2023.
Floris Takens. Detecting strange attractors in turbulence. In Dynamical Systems and Turbulence, Warwick 1980: proceedings of a symposium held at the University of Warwick 1979/80, pp. 366-381, Springer, 2006.
A. P. Thompson, H. M. Aktulga, R. Berger, D. S. Bolintineanu, W. M. Brown, P. S. Crozier, P. J. in Veld, A. Kohlmeyer, C. Trott, and S. J. Plimpton. LAMMPS flexible simulation tool for particle-based materials modeling the atomic, meso, and continuum scales. Comp. Phys. Comm., 271:10817, 2022. doi: 10. 10.10166..pp.22011108111
Moore, D. Nguyen, R. Shan, M. J. Stevens, Tranchida,
12

---

Katharina Vollmayr-Lee. Introduction to molecular dynamics simulations. American Journal of Physics, 88(5):401-422, 2020.
Martin J. Wainwright. Uniform laws of large numbers, pp. 98-120. Cambridge Series Statistical and Probabilistic Mathematics. Cambridge University Press, 2019.
Jiang Wang, Simon Olsson, Christoph Wehmeyer, Adrià Pérez, Nicholas E Charron, Gianni De Fabri- tiss, Frank Noé, and Cecilia Clementi. Machine learning of coarse-grained molecular dynamics force fields. ACS central science, 5(5):755-767, 2019.
Junxiang Yang, Yibao Li, Chaeyoung Lee, Yongho Choi, and Junseok Kim. Fast evolution numerical method for the allen-cahn equation. Journal of King Saud University-Science 35(1):102430, 2023,
Haijun Yu, Xinyuan Tian, Weinan E, and Qianxiao Li. Onsagernet: Learning stable and interpretable 1):1 14402, 2021.
dynamies using generalized onsager principle. Physical Review Fluids, 6(11):1
Linfeng Zhang, Jiequn Han, Han Wang, Roberto Car, et al. Deepeg: Constructing coarse-grained models via deep neural networks. The Journal of chemical physics, 149(3), 2018.
Linfeng Zhang, De- Ye Lin, Han Wang, Roberto Car, and Weinan E. Active learning of uniformly accurate interatomic potentials for materials simulation. Physical Review Materials, 3(2)0023804
2019.
Zhen Zhang, Yeonjong Shin, and George Em Karniadakis. Gfinns: Generic formalism informed
neural networks for deterministic and stochastic dynamical systems. Philosophical Transactions of the Royal Society A, 380(2229):20210200 2022.
Kun Zhou and Bo Liu. Chapter 2 potential energy functions. Kun Zhou and Bo Liu (eds.), Molecular Dynamics Simulation, pp. 41-65. Elsevier, 2022.
13

---

Appendix
E θ here θ the set of our model X = (x1,". 1. E E R y = f(x) E y c IRN. AE
parameters. Write DR fx', y') i. then L2(0), (x can be written as:
1
(x
Σ
11x0((x) go(2)1
K
(x.))ERR
(21)
(0)
Σ
Ily (x)((x)) go(2))1
K
(xy))DK
In our method, we map the training procedure from the macroscopic coordinates to microscopic coordinates and use partial computation of the microscopic forces to train. We treat (z as baseline and give detailed theoretical analysis of the possible error introduced by using Lx.p (). The error
can be controlled by two parts: (i) Convert loss from space to space, i.e., the error introduced by using (0) (ii) Use partial labels, i.e., the error between Cx.p (0) and (x))..
We will analyze the first part and the second part of the error accordingly (Appendix A). More experimental details are provided Appendix B
A Theoretical Analysis
A.1 Proof of Theorem
Theorem. Assume for any x D, the eigenvalues of ('()x' are lower bounded by bj and upper bounded by b2, O < bi & bz. Then:
bz( + & (z) (0) + C)
(22)
here C does not depend on hence does not affect the optimization.
Proof. We can write p'(x)) in the following form by leveraging singular value decomposition:
''(x) UEVT.
(23)
where (x(x) E RaxN E RdxN is a rectangular diagonal matrix, e Rdxdd and V e RNN two orthogonal matrices, d N. E. V actually depends on but for simplicity
omit the dependence in notation. During the training of the autoencoder, we enforce pp((x) to have full rank, then the diagonal items all nonzero, i.e., Σi, Ax(x) * 0, 1,... dd. We define Σ TR Nxd be the rectangular diagonal matrix such that the diagonal items
(E'),, = (x) * 0, = ,, and all the remaining items are zero. Actually Σ is the Peooree inverse of Σ.
Then (T)( can be calculated by:
(x'(x)) V
(24)
If we denote the i-th column of V by v,, the i-th column of U by u, then V (V1,'" "V (u Ua), and we can rewrite L. (x) and (0):
(0)
1 Ex (x)y {z)
= R EIxYJED UEV y {0(2)
N/ E(XY)ED UEVTY y UUT go(2))1
= 1
K LIXY)ER ΣVT y - U U - go)))
(25)
K E(XYJEDX ΣVT y U gggo(z)) (z)
1 Dd.
K E(XY)DD (xxxv u go (z)
14

---

(x ExyyEED by (x)( go 2)12
R LiXyJED yy VE go go
R LIXYJED VV VE'UU
K IXNEED (V go-"" -
= LIXyEED V"y - U- go(z)]
= R LT (x,)EDD lIV - x,
+ & L. Exxyje IIy
= * E. E(X,)EDD N, (xxx))) - uf go(z) 123 + F Ei=d+1 ((x)))DD |
(26)
We define:
(x) K Ei= [(X))EDX N (x) - uf go(2)
(27)
C= - Ei (x v
then (x) = (x) C, C does not depend on 0 and:
min (x) = min (x)
(28)
Comparing (0) and (0), we observe that the only difference between (0( and Lx is that
for every term [ f(x) - uu go(z) (r(z))ll, 11.. there is a constant 2x) multiplied to it. Hence Lx (0) a weighted version of f. (0). Note that if the eigenvalues of (x(xx) are (x), i 1, ..... then the eigenvalues of xx [0(())((]))))))))))) (x)? are (x), i = 1, ... d.
Since the eigenvalues of up (x)) are lower bounded by bj > 0 and upper bounded by b2, i.e.
Vx E X < b1 & {² & b2, then:
Lx() < (x) + C (0)
(29)
or equivalently,
(c( + C) < (x) (2((x(0) +
(30)
By minimizing (x((),
actually narrowing the region of (0). Hence want by and by to be as close as possible. In the extreme case where b1 by, minimizing (0) is just equivalent to minimizing (0))). Another observation is that (x(() is a weighted version (x(0), (0), and if there exists i such that ), is too small compared to the others, the weighted sum will be dominated by in Eq. (27)
The above insights guide us to constrain the condition number of (''xx)(((x during the training of autoencoder, i.e., we require x'(xx)''(xx) to be well-conditioned through Eq. (6
A.2 Proof of Eq. 13
Exi x Fi(x)). I(xx (x)
E. pK Σ [ (x*) (xx(x))) (zx
(4xx)) [[((xc) (x) - (xx(x)) Bo (z)
1 Ex 1
Σn= [(x) [|f((x) - (4' (x))
p | Σx= L, [lf((x) - (x))
(31)
Ex M [[((() - (4 (oo
-Ex(]f((x) (xx( (x)) (z) (2)
Ext,.. xx (0)
15

---

A.3 Proof of Theorem 2
In this section we will prove the behavior of the minimizer found by Cx.p in the limit. Our proof relies the statistical learning theory and especially Rademacher complexity. We will provide some background information first.
Let H. be family of real-valued functions with domain W and integrable w.r.t. IP, here I is a proba- bility over W. W (w", ,w") is collection of i.i.d. samples from probability distribution P defined over W.
We will the tool of Rademacher complexity:
Definition 1. Let H. W be defined as before. The empirical Rademacher complexity of with respect to W is defined as:
1 n
Rw (H) Eg
sup - hEW TI
Σ
ach(w"))
(32)
σ = (01, .............................. , are independent random variables uniformly chosen from (-1, 13, with P(a. P(o: -1] 0.5. Taking the expectation with respect W yields the Rademacher complexity of the functional class H.:
].
1
R., Ewn Eo
sup has
Σ o,h(w")
(33)
Then one can derive the generalization bound in terms of the Rademacher complexity (Wainwright 2
Theorem Assume H uniformly bounded by (i.e., ll & b. Then for all > and 0, we have
sup kEH
h(w') E[h] 22),(1> +
(34)
n2
with probability least 1
2exp
nga 882
Consequently, as long as R, (xx) o(1), we have
1 - - - BIf] 0.a, 0, Vf E Hx.
Now in our problem, let Hx.p be the following one function class:
Hx.p = (ha.p: X y I (x. I(x)) = |1 (v0( (ox e 0) (35)
Then
Cx (0) K
Σ
hx,
(36)
(x.))DD
Now we can prove Theorem 2
Theorem. Let (2(0) (O, 0* arg ming (0), 0k,p argming (), if Hx.p is
uniformly bounded by bx.y and Rx (Hx,)) o(1), then:
_x(0k,p) (0*) a.e. 0
(37)
Proof. We define (x,((0) = EExxp((), then Ex = Ex.p by Appendix A.2 Applying Theorem get
3
sup A
(x) (x) & (Rx(Hx.)) +
(38)
with probability at least - 2 2exp
K8 8b
and (x) a.e, (x,
(), 40 O.
16

---

Note that 0* arg (x) E arg ming (x.p then
0 < (xOK,p)) (x()*) (0pp)) (x.p(OKp)) + Lxp((0pp) - Lx,p(
(39)
+ Lx,p(0%) (x))
thus (x,)) (0*)
a.R. 0.
B Experiment Details
(a) Predator-Prey system
(b) Allen-Cahh Cahn system
(c) Lennard-Jones system
Figure 5: Visualization of the microscopic state of each system
B.1
Predator-Prey System
We consider the Neumann boundary condition S,uu = 0, δ,v = on aQ and the following initial conditions
u(r 0)
cos(5 0(2, 0) μ cos( (n, [0, x 0.64,0.6]
(40)
The Neumann boundary condition commonly used in mathematical models of ecosystems, restrict- ing any movement of the species out of the boundary.
We approximate the spatial derivatives in Eq. (15) with finite difference method:
220
o( 1)
20(zi, 1) v(x;-1.() 2 2 i < 49
dx2 t)
Ax
2
0(32, () v(x1 ()
11 t)
(41)
Ax
v( 1)
v v(230:0)
(550 1)
Ar?
Let (u, = w(1- u u v), (u, v) = av u b). and hu (u, v), hy v) denote the element-wise
application of h, by each u( t), ( t), & i i 50. Then
du
dt h (u,
(42)
dv = h (u, v) Av
di
here A R-50550 is a matrix defined according to Eq. spatial discretization can be written in the form of Eq.
42). Hence the predator-prey system after
The training parameter set Timin pairs (μ, a) sampled uniformly from [0, 0.2] [0.4, 0.6]. For testing, Tes is also sampled uniformly from [0, 0.2] x [0.4, 0.6]. but with a different random seed from Train- The mean relative error is defined as
In our experiment, we choose a 3, b 0.4,
Σn Liiuue (tro p4, 0) gord (tni o)
e Treat
Σ
(43)
Tress)
Σn - (!ni ,4, o)
(p.
here use z(-; to denote the dependency of the solution the initial condition.
17

---

Table 2: Results on the Predator-Prey system. Models are trained with different training metric
Lx Cx, Cp = 3/4, 1/2, 1/4,1/5) 1/5). Mean and standard deviation are reported over three repeats.
# of truining data
Lx
Cxp(p = 3/4)
Lxp(p 1/2)
Lxp(p = 1/4)
Lx - 1/5)
6.0 x 10
1.09 ast x10
1.79+ x10
9.22 +44x 10
3,50 +13 x10
4.00 La x10
1,5 x 10
4.70 046 x10
4,56+
191x10
3.30- um x10-3
2.19 +051x 10-
1.92 1.92-0xx100
3.0 102
2.90
x10
2.07+04: x10-
1.57 x10
1.23+ x10
1.34- x10-
6.0 x 10a
1.24 1oux 10-3
1.07 x x10
1.46 100 x x00-
1,08
x10
1.13 x10-3
1.2x x 104
9.20 200 x x00
8.13 x10-4
8.23 x10
7.66
x x10
8.36 x x10-
2.4 x 10
6.76 x x00
6.85 10
6.98 1130x
5.64 1004 x10
5.70 x10
Data Generation The microscopic equation is solved with uniform time step At 0.01 from t = 0 to t = 30 using the Euler method. We subsampled every tenth snapshot for training. During testing, the microscopic evolution equation is solved with the same At 0.01 using Runge-Kutee Kutta 4-order RK4 solver. Then we encode the microscopic trajectories to obtain the ground truth latent trajectories. The predicted latent trajectories are obtained by encoding the initial microscopic state first, then solved using RK4 solver with At 0.1, A 0.5 on [0, 30]. Fig. 6 and Fig. show the true and predicted trajectories.
Predictions uf
Predictions of v
Predictions in the phase plane
Error
A19
A/t
Gup (p 1.5)
Cap (p- 1/5)
Cep
3.00
Ground truth
Ground truth
Ground truth
0.0004
0.000
555%
Figure 6: Latent trajectories with initial condition / 0.02, o 0.52 and At 0.1 in the Predator- Prey system.
Predictions of
Predictions of V
Predictions the phase plane
Error
4301
Fuu (p 10 18)
Cap (p 1/5) truth
Pxp lp 1/5)
Cun In- 1/5)
Ground truth
460
9
2000
0.000
0.0
3.10
340
6.50
Figure 7: Latent trajectories with initial condition = 0.02, a = 0.52 and At = 0.5 in the Predator- Prey system.
B.2 Allen-Cahn System
In our experiment, we consider the initial condition of a torus
(Kim et al. 2021
ulr,y, y. 0) = - + tanh( F1 - d(x, v)
tanhi r2 d(x, y)
(44)
2ec
/2c
here d(x, y) / (a - 0.5) 0.5) r1 E [0.3,0.4] the cireumscribed circle radius and T2 E (0.1,0.15) is the inscribed circle radius. The initial condition is visualized in Fig. (b).
The free energy in Eq. (20) also tends to decrease with time, following the energy dissipation law in Eq. (45). Then the minimization of the free energy drives the evolution of the system towards equilibrium.
(0(2) Øt
||0uu113 dx dy
(45)
18

---

Data Generation For both training and testing, the microscopic evolution law is solved using RK4 method with At 1/N 2.5 10 from t= 0 to min(ty. 1ere ł is the time when the Allen-Cahn system reaches the equilibrium. We subsample every hundredth snapshots for training. We choose in Eq. 19 and Eq. (44) to be 2v2tanh-1(0.0) as (Kim et al. 2021).
100
and we report the mean relative error. The test parameter set Test contains 50 parameter points (r1, (2) are chosen uniformly from [0.3, 0.4] x 0.15,, but are sampled with different random seed.
For testing, 50 parameter points (r1, are chosen uniformly from [0.3, 0.4) x
B.3 Lennard-Jones System
We consider Lennard-lones systems containing different atoms in this paper: Natons 800, 2700, 6400, 21600, 51200. We use periodic boundary conditions and fix the density to be 0.8, then the corresponding box side lengths are 10, 15, 20, 30, 40. We simulate the Lennard-Jones system under the NVE ensemble using the LAMMPS (Thompson al. (022). In our experiment, Ei Fij 1, Vi,j j and Yeur 2.5. The integration step is 0.001 and each trajectory is integrated for 250 steps. We sample the initial temperature randomly from [0.5, 1.5]. The initial velocities
then sampled from the has the same atom positions and velocity direction. For testing, the initial temperatures are also randomly sampled from [0.5, 1.5] but with a different random seed to the training data.
distribution. For each system, the initial configuration
C Implementation Details
All the experiments are run on single NVIDIA GeForce RTX 3090 GPU. For all the experiments, we use the multilayer perceptron (MLP) for both the encoder and the decoder. The autoencoders are trained with LAE in Eq. 7 The condition number is the maximal eigenvalue Amax divided by the minimal eigenvalue Amin of 4 (x)))
K0 (x)((x) = Ammax (xx)((()) 2
(46)
DAmia (x'(())p(xx))")
Rdxd, and d is small, the condition number ((x'((x)p'(xx (x)7) can be calculated efficiently. In our experiments, we calculate Amax and Amin with torch. 1 diinllg.sva svd. To better compare fx and Lx.ps once finish the training the au- toencoders, we freeze them and use the encoder for macroscopic dynamics identification. For the macroscopic dynamics identification, MLP and GFINNs are used for the latent model in Section 5.2 For the rest experiments, we adopt the structure of OnsagerNet to enhance the stability for latent dynamics prediction for ge Yu et al. 2021).
Since (x) e RdxN (x((x)x'(xx)
D Additional Experiments
D.1 Loss Curve of Lz, Lxs Lx.p
To give the readers a better idea of the behaviors of the loss Le, Ex. 1/4) trained on the same number of training data. Fig. X shows the training and test loss curve of different training metrics. Note that the training metrics of these models are different, but they are tested with the same metric Eq.
43
D.2 Ablation Analysis of Acond
To evaluate the influence of the hyperparameter Asond on the performance of the loss Cx,p- we conduct experiments with different values of Acoed and show the test error in Table
From Table we can observe when Acond increases from 0 to 10-6 the test error gradually decrease. When Agond further increases from 10-6 to 10 2 the test error gradually decrease. Among all the Acond that we tried, the test error has the minimal value when Acond 10-6
Theoretically, if Acond is too low, since LAE Lree + Aeond there may not have enough constraint on Loond and the condition number of (x)xp'(x) may be very large. By Theorem
19

---

training loss fx
training loss Cx
training loss Ex.0
Test error
10-3
10
Lx
Cx.0
10-3
10"2
10-4
10-4
10-1
10-1
500
1000
2000
1000
1500
2000
1500 Epoch
2000
500
Epoch
Epoch
Epoch
Figure 8: Loss curve of the Lx, Lx, (p 1/4) on the Predator-Pres Prey system. Models are trained
with different loss functions Lzz Lxx, Lx 1/4) on the same number of training data.
Table 3: Results on the Predator-Pre Prey system. We train the autoencoder with different Acond and then train the macroscopic dynamies model with loss (x, 1/5). The mean relative error of the macroscopic dynamics model is reported over three repeats.
Avard
6
5
Test errrr of Lx p 1/5) 6.42 x 10-2 2.62 x 10-3 2.84 x 10-3 8.36 x 10 3,68 2.60
10-3 4.24 x 10-3
small condition number of yp (x)) guarantee the effectiveness of Lx. But when the condition number large, there is no guarantee that Lx thus Lx.p can perform well. If instead Acond is large, Cae will be dominated by Lcona- Then the autoencoder may reconstruct the microscopic dynamics well and, hence may not capture the closure terms well. If the latent space is not closed
enough, learn the macroscopic dynamics well.
20

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The main claim clearly reflect our contributions and scope both in theory and experiments.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
In Section 6 we discuss the limitations.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
21

---

Full set of assumptions and complete proof of our theoretical results are provided Appendix
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proof' provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification:
We explain our method in details in Section 4.2 Algorithm and Algorithm
Guidelines:
The answer NA means that the paper does include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
22

---

Answer: [Yes]
Justification:
Our
code is available
at https://githu. hom.tthh.comoommcccommmmmmmmm com MLDS NUS/
Learn-Partial
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (htttps://nips..
public publicc/uiddes//eeeeee /ewdedeeee Codeeuummeeeeeeieieoee sssonoolocciicoo for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
//nipss.cc/public c des CodeSubmn bub ss onPol icy for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Details of the experiment setting is provided in Section 5
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification:
We report the error bar for all the experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
23

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We mention in Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics htttts/s//aarrss..cc////ll/tttas
Answer: [Yes]
Justification:
Our research conform with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: no societal impacts
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: no such risks
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: Existing assets properly cited.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms of use the package should be provided. For popular datasets, paper thcode com dataaetteett has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
25

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
The documentation is provided.
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
Our research is not involved in such subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
Our research is not involved in such subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
26


Reviews:
[{"review_id": "5X2v71GAu8", "reviewer": "Reviewer_d4vx", "summary": "The authors describe a method to efficiently obtain aggregate information about forces acting on all particles in a system, in an effort to compute dynamics of macroscopic (aggregated) quantities. The key idea described in the paper is to sub-sample the particles to a small set, and only compute the forces on the small set (instead of computing all forces). The forces are then used to train a vector field of a latent space variable (through the chain rule). The latent space is obtained by training an auto-encoder on the microscopic states, independent of the force computations. The authors demonstrate the efficiency on multiple numerical systems, and include theoretical insights into the loss functions on micro- and macro (latent) states.", "questions": "1) What could be done with quantities that do not rely on individual molecule evaluations (e.g. force), but on integral quantities (e.g. \"density\")?\n2) I assume the authors require all particles for the force computation because even if a particle is not chosen for force computaiton, it still needs to be present because other particles require its position for \"their\" force computation? This would also answer question 1.\n3) What is the core difference between reference C2 and the approach by the authors?\n4) l159: \"we get rid of the matrix-vector product...\": why is this beneficial? The new loss also has a matrix vector product with phi' and g.\n5) Equation 2 is not precise enough (and contains two typos: the parenthesis and no period at the end). What is the \"almost equivalent\" here? What is assumed, precisely? How accurate does the approximate force need to be?\n6) The autoencoder described in section 4.1 seems to have a very simple architecture. How is it possible to train it for such high-dimensional microscopic input states, if the encoder is not permutation invariant? For example, if the model is trained using a fixed index for each molecule, and then the indices are shuffled at random, the input to the autoencoder completely changes while the conformation (independent of the index) does not change at all. Since the autoencoder is not permutation invariant, it would need to learn all conformations including permutations, which is an incredibly high-dimensional space (not possible to sample in reasonable time).", "limitations": "The authors do not discuss the limitations of the auto-encoder they use to map to a latent space, in particular regarding permutation invariance of the molecular indices (see my question).\nSocietal concerns are not addressed at all; e.g. that speeding up force computations can speed up insights into materials that are harmful (not just beneficial).", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "Obtaining dynamics of macroscopic variables from microscopic simulations is an important and challenging problem. Sub-sampling particles to reduce the burden on force computations and learning latent spaces of microscopic states automatically (instead of prescribing known quantities) are reasonable approaches to address this challenge. The chosen particle systems in the computational experiments are well-known and reasonable choices, and the reduction in the number of force computations (while keeping an accurate macroscopic state dynamics) are impressive.", "weaknesses": "1) The methods proposed in the paper are (a) sub-sampling particle indices to reduce number of force computations and (b) using a standard auto-encoder network to obtain a latent space. The former idea - especially when combined with the chain rule for the macroscopic dynamics - is noteworthy. Still, the general advancement of the state of the art is not large enough to warrant acceptance, in my opinion. In particular, the choice of auto-encoder is not suitable for the given problem (see question 6). \n\n2) Some parts of the literature are not cited.\n\n[A] l.65: reduced order modeling has not started in 2020 with deep learning methods. Appropriate literature from the decades before should be cited. A review can be found here:\n\n[A1] Schilders, Wilhelmus H.A., Joost Rommes, and Henk A. van der Vorst, eds. Model Order Reduction: Theory, Research Aspects and Applications. Mathematics in Industry. Springer Berlin Heidelberg, 2008. https://doi.org/10.1007/978-3-540-78841-6.\n\n[B] l.55: similarly, \"learning from partial observations\" is a decades (if not centuries) old problem. Key mathematical contributions are from Ruelle and Takens over 40 years ago, refined by Yorke:\n\n[B1] Takens, Floris. “Detecting Strange Attractors in Turbulence.” Lecture Notes in Mathematics, 1981, 366–81. https://doi.org/10.1007/bfb0091924.\n[B2] Ruelle, David, and Floris Takens. “On the Nature of Turbulence.” Commun. Math. Phys. 20, no. 3 (September 1971): 167–92. https://doi.org/10.1007/bf01646553.\n[B3] Sauer, Tim, James A. Yorke, and Martin Casdagli. “Embedology.” Journal of Statistical Physics 65, no. 3 (1991): 579–616. https://doi.org/10.1007/BF01053745.\n\n[C] In general, the idea of \"sparse sampling\" of the microscopic dynamics has also been published before. This must not just be cited but critically compared to, especially [C2].\n\n[C1] Samaey, Giovanni, Ioannis G. Kevrekidis, and Dirk Roose. “Patch Dynamics with Buffers for Homogenization Problems.” Journal of Computational Physics 213, no. 1 (March 2006): 264–87. https://doi.org/10.1016/j.jcp.2005.08.010.\n[C2] Liu, Ping, Giovanni Samaey, C. William Gear, and Ioannis G. Kevrekidis. “On the Acceleration of Spatially Distributed Agent-Based Computations: A Patch Dynamics Scheme.” Applied Numerical Mathematics 92 (June 2015): 54–69. https://doi.org/10.1016/j.apnum.2014.12.007.\n\n[D] The idea of creating a latent space with an auto-encoder is of course also not new. The authors cite a few papers in l.143, but should do so in section 4.1.\n\n\n3) l141: it is misleading to write \"$g_\\theta = \\phi'(x)f(x)$\". g depends on z, not x (correct?). It is not clear if the authors mean \"$g_\\theta(z) = \\phi'(\\phi^{-1}(z))f(\\phi^{-1}(z))$, i.e., as a combination of $\\phi'$ (encoder Jacobian), $\\phi^{-1}$ (decoder) and $f$. At this point in the paper, it could also be that training of g happens directly on dynamics of z and avoid $\\phi$. The latter is probably the case, looking at equation 9, but it is not clear from the text.\n\n\nMinor:\n1) l74: \"where\" instead of \"here\", and it is $f(x)\\in\\mathbb{R}^N$, not $f\\in\\mathbb{R}^N$ (the former is a vector, the latter a function).\n2) l157: \"we constrict the condition number...\" (I think \"constrain\" is meant?) is not ideal wording; because the loss in eq. 6 does not constrain the condition number at all, it just penalizes it.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed review and thoughtful feedback. We have corrected all the minor issues and typos that you pointed out. We have also cited the previously missing literature you mentioned. Below are our detailed responses to the questions: \n\n**Q: Equation 2 is not precise enough. What is the \"almost equivalent\" here? What is assumed, precisely? How accurate does the approximate force need to be?**\n\nA: Thanks for pointing this out. In Eq 2, we mean $f_i(x_1, \\cdots, x_n)$ can be calculated accurately through the microscopic coordinate of at most $M$ particles. The approximation is accurate enough such that the error is neglectable. For example, it is a common practice in molecular dynamics simulations to employ a cutoff distance for force fields. We have changed the '$\\approx$' in Eq 2 to '$=$'. \n\nIf we want to formalize Eq 2 more rigorously, we can require the error to be bounded by a tolerance $\\epsilon$:\n$$\n||f_i(x_1, \\cdots, x_n) -  \\tilde{f_i}|| < \\epsilon\n$$ where $f_i$ is the exact microscopic force and $\\tilde{f}_i$ is the approximation.$\\epsilon$ will play a role in Theorem 1 such that Eq 11 becomes:\n$$\nb_1(L_x(\\theta) + C) + \\mathcal{O}(\\epsilon) \\leq L_z \\leq b_2(L_x(\\theta) + C) + \\mathcal{O}(\\epsilon) \n$$\nThe performance of $L_x(\\theta)$ can still be guaranteed if $\\mathcal{O}(\\epsilon)$ is much smaller compared to $b_1 C$.  \n\n**Q: I assume the authors require all particles for the force computation because even if a particle is not chosen for force computations, it still needs to be present because other particles require its position for \"their\" force computation?**\n\nA: Not really. By the sparsity assumption mentioned in l 100, the microscopic force on each particle $i$ depends only on the microscopic coordinates of several particles that are in $J(x_i)$. If we want to calculate the forces on particles $x_1, \\cdots, x_m$, we only need the coordinates of the particles that belong to $J(x_1), \\cdots, J(x_m)$. The coordinates of particles in $J(x_i)$ are only used for the force calculation on particle $i$, not for any other particles. The computation cost increases linearly with the number of particles for which the microscopic forces are calculated.\n\nWe also want to clarify that, since the macroscopic observable $z^{\\ast}$ depends on the microscopic coordinate of all the particles, the calculation of $z^{\\ast}$ needs all the particles to be present. But the calculation of $z^{\\ast}$ is fast and cheap and the computation cost is almost neglectable. \n\n**Q: l159: \"we get rid of the matrix-vector product...\": why is this beneficial? The new loss also has a matrix-vector product with phi' and g.**\n\nA: The i-th entry of $\\varphi^{\\prime}(x)f(x)$ is $\\sum_{i}\\varphi_{ij}^{\\prime}(x)f_j(x)$. It is difficult to find an unbiased estimation of $\\sum_{i}\\varphi_{ij}^{\\prime}(x)f_j(x)$ using a subset of {$f_j(x)$}$_{j=1, \\cdots, n}$.\n\n\nHowever, in the loss $L_x$ shown in Eq 10, $||f(x) - (\\varPhi^{\\prime}(x))^{\\dagger}g_{\\theta}(z)||_2^2$  can be written as \n\n$$\\sum_{i=1}^n ||f_i(x) - ((\\varphi^{\\prime}(x))^{\\dagger} g_{\\theta}(z))_i||_2^2$$\n\nFor any subset {$f_j(x)$}$_{j \\in I(x)}, |I(x)| = n \\cdot p$, \n\n$$ \\frac{1}{p}\\sum_{i\\in I(x)}||f_i(x) - ((\\varPhi^{\\prime}(x))^{\\dagger}g_{\\theta}(z))_i||_2^2$$ \n\nwhich can be used as a stochastic and unbiased estimation of $||f(x) - (\\varPhi^{\\prime}(x))^{\\dagger}g_{\\theta}(z)||_2^2$. The new loss can learn the macroscopic dynamics from a subset of microscopic forces and requires less force computation compared to $L_z$. The computation cost reduction is manifested through less force computations required by the new loss. \n\n**Q: What could be done with quantities that do not rely on individual molecule evaluations (e.g. force), but on integral quantities (e.g. \"density\")?**\n\nA: Our method can learn the dynamics of any macroscopic observables that can be written as a differential function of the microscopic coordinate: $z^{\\ast} = \\varphi^{\\ast}(x)$. The density within a fixed region $\\Omega$ is defined as the ratio of the number of particles contained in $\\Omega$ to the area of $\\Omega$: \n$$\n\\frac{\\sum_{i} \\mathbb{1}(x_i\\in \\Omega)}{|\\Omega|}\n$$The above function is not differentiable, but we can use a differential function to approximate it. Then our method can be applied.\nCurrently, our method still can not address the case where $\\varphi^{\\ast}$ is not differentiable, or not deterministic.\n \n**Q: The autoencoder described in section 4.1 seems to have a very simple architecture. How is it possible to train it for such high-dimensional microscopic input states, if the encoder is not permutation invariant?**\n\nA: Thank you for the insightful question. Yes, we use MLP for both the encoder and the decoder, which indeed have a very simple architecture. In I 587, we mentioned each configuration has the same atom positions and velocity directions, and we only vary the temperature of the system. The order of the particles is fixed in the input to the autoencoder. In this way, no permutation needs to be considered and our simple MLP is enough to learn a latent space. We mainly want to use the experiment on the Lennard-Jones system to validate that our method can be applied to particle systems and can scale to very large systems. Note that in PDE experiments, the initial configuration is sampled from the initial distribution, and our method can handle such case. \n\nThank you for pointing this out. We have added the limitation of the autoencoder in our paper. We will also try more advanced autoencoders such as graph neural networks that are permutation invariant to further improve our paper. \n\n**Q: What is the core difference between reference C2 and the approach by the authors?**\n\nA:  We answer this question in the top-level Author Rebuttal [C].", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "j4OnyKGSuV", "reviewer": "Reviewer_ktyB", "summary": "The authors aims to use ML to compute macroscopic dynamics of a system from partially observed microscopic dynamics. The paper defines the macroscopic dynamics to be the lower dimensional latent space of an autoencoder that encodes the microscopic system. Given the autoencoder they train a macroscopic dynamics model using the partially observed forces of the microscopic dynamics that are projected to the macroscopic space via the encoder. Experiments are carried out on toy systems and lennard jones potential systems that are scaled to a large number of particles.", "questions": "1. What if the particles to observe are not chosen uniformly at random? Is that also a common scenario where we do not have a uniform subsampling but our observations are biased by the specific subregion that we observe.", "limitations": "The paper prominently discusses two limitations (sparsity assumption and sampled distribution of microscopic dynamics) which are significant and put the paper into perspective of the field of work. I do not see further meaningful limitations that should be discussed apart from the possible limitation in terms of realistic evaluation which I mention in the weaknesses section.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper tackles an impactful problem with important scientific applications \n2. The underlying idea is simple (training an autoencoder to obtain the mapping from fine to coarse dynamics and using it to project the microscopic forces). However, non-trivial technical problems arise which the authors solve with creative techniques that would be valuable to share and present at a conference.\n3. Rigorous evaluation: The authors have several creative insightful evaluations that go beyond the experiments that I would have imagined or expected (this is with the caveat that I do not work on reduced order modeling and am not familiar with what the standard evaluations are).", "weaknesses": "1.  Experiment system choices: The authors carry out experiments on small toy systems and large lennard jones systems of the same particles. It seems to me that the problem should become significantly harder with the \"homogeneity\" of the modeled system decreasing. It further seems to me that this would be the case in real scientific applications where we potentially have different atom types and multiple forces beyond lennard jones potential forces. Is this assessment correct? It would be great if you could put into perspective how close the experiments you carry out and the systems you choose are to actual scientific applications of interest. Thanks!\n2. Presentation: you briefly mention closure modeling in the related work. Then in the methods section you say you follow closure modeling and define quantities such that a closed system arises. At this point I ask myself, what does closed mean? I hoped it becomes clear later in the paper but find there to be no sufficient explanation. Could you please let me know how you train the autoencoder in such a manner that the latent space captures specific desired macroscopic dynamics?\nMinor:\n1. Presentation: Motivation: What are some compelling application examples in which learning from partially observed microscopic dynamics is valuable?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed review and thoughtful feedback. Below are our responses:\n\n**Q: What does closed mean?**\n\nA: Let the state of a system be $z$. When we say the system is closed, we mean the dynamics of the system can be written in the following form:\n$$\n\\dot{z} = f(z)\n$$where $f$ is a function that only depends on the system's state $z$, not on any external variables. Thank you for pointing this out. We will include an explanation of this in the paper. \n\n**Q: Could you please let me know how you train the autoencoder in such a manner that the latent space captures specific desired macroscopic dynamics?**\n\nA: The autoencoder is trained with Eq. 7 in the paper:\n$$\nL_{AE} = L_{rec} + \\lambda_{cond} L_{cond}\n$$ which is a reconstruction loss plus a condition number normalization loss. \n\nThe minimization of the reconstruction loss ensures the latent space captures almost all the key dynamics and structures in the high-dimensional system. Then the specific macroscopic observables along with the latent space can be viewed as a closed system since the latent space captures almost all the information. The autoencoder only finds the closure to the macroscopic observables. Next, We use a neural network to parametrize their dynamics and train the model with $L_{x, p}$ to capture the desired macroscopic dynamics. \n\n\n**Q: The problem should become significantly harder with the \"homogeneity\" of the modeled system decreasing, In real scientific applications where we potentially have different atom types and multiple forces beyond Lennard-Jones potential forces. Is this assessment correct? It would be great if you could put into perspective how close the experiments you carry out and the systems you choose are to actual scientific applications of interest**\n\nA: Thank you for the insightful question. The application of our method to real scientific problems is also a primary focus of our next step. We agree that compared to the Lennard-Jones system we choose, we potentially have 1. different atom types and 2. more complex forces in real scientific applications. \n\nIn the experiment of the Lennard-Jones system, we have validated that our method can be applied to particle systems and can scale to very large systems. We think the main challenge posed by different atom types and more complex forces lies in the closure modeling. More advanced autoencoders can be used for efficient closure modeling. Once the closure to the desired macroscopic observables is found, our framework can be readily applied to learn the macroscopic dynamics in real large-scale scientific problems. \n\n\n\n**Q: What are some compelling application examples in which learning from partially observed microscopic dynamics is valuable?**\n\nA: For example, in the design of Li-ion batteries, the viscosity and ionic diffusivity of liquid electrolytes are usually of key concern. But to obtain the macroscopic observables, existing method require long-term microscopic simulation where the calculation of the microscopic forces on all the atoms is extremely expensive [1]. In this situation, our method can learn the desired macroscopic dynamics from partial computation of microscopic forces, then we can significantly reduce the computational cost for force computations. \n\n**Q: What if the particles to observe are not chosen uniformly at random? Is that also a common scenario where we do not have a uniform subsampling but our observations are biased by the specific subregion that we observe**\n\nA:  When we do not have a uniform subsampling, we can use importance sampling to reweight the data such that the probability of each particle being chosen is the same. For example, assume a region $\\Omega$ is divided into four subregions $\\Omega_i, i=1,2,3,4$. The i-th region is chosen with probability $p(\\Omega_i) = p_i >0$, and $p_1 + p_2 + p_3 + p_4 = 1$ . We have $n$ samples drawn from the distribution. Then we can reweight each sample from region $\\Omega_i$ by $1/p_i$. In the reweighted data, each particle is chosen with the same probability. \n\n---\nReferences:\n\n[1] Jia, W., Wang, H., Chen, M., Lu, D., Lin, L., Car, R., ... & Zhang, L. (2020, November). Pushing the limit of molecular dynamics with ab initio accuracy to 100 million atoms with machine learning. In SC20: International conference for high performance computing, networking, storage and analysis (pp. 1-14). IEEE.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "R1xP8v9EQj", "reviewer": "Reviewer_RRQb", "summary": "This paper introduces an efficient framework for learning macroscopic dynamics of complex systems. The training data consists of microscopic configurations with only partial dynamics, i.e., time derivatives of only a small subset of the microscopic variables. The paper shows how this information nevertheless allows unbiased estimation of dynamics at the level of low-dimensional macroscopic dynamics. Experiments show the data-efficiency of the scheme compared to learning on full microscopic dynamics.", "questions": "See above", "limitations": "Yes", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The paper’s approach is novel, addresses an important bottleneck in the field, and has the potential to become an important contribution in the area of ML for dynamical systems\n* The paper is generally well written and the exposition is easy to follow.\n* The experiments cover a diversity of dynamical systems, including MD and PDE systems.", "weaknesses": "* Although the theoretical results are nice and the experimental results empirically support them, I am still skeptical of the efficacy of $L_x$ as a proxy loss for $L_z$. To bolster the claims, I strongly recommend that authors present some empirical analysis on the spectrum of the Jacobian $\\psi’$, or even better, plot loss curves over the course of a training run showing the behavior of $L_z$ versus $L_x$. This will give the reader a much better intuition for how these different loss functions behave.\n* Without an understanding of how significant the final test errors are, it is not particularly insightful to merely show test errors after an arbitrarily selected amount of training data. Instead it would be nice to see scaling plots like in Figure 2 for the other systems in the analysis.\n* Table 1 and Figure 2 are redundant and I would recommend eliminating the former.\n* Theorem 2 could be further clarified. What is $\\arg \\min L_{x,p}$ since $L_{x,p}$ is also a random variable?\n* Is it difficult or tricky to tune the weight of the conditioning number loss term? What happens if the weight on this term is not high enough? What if it is too high? Ablation analyses of these factors would improve the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed review and thoughtful feedback. Below are our responses:\n\n**Q: Plot loss curves over the course of a training run showing the behavior of $L_x$ versus $L_z$. This will give the reader a much better intuition for how these different loss functions behave.**\n\nA: Thanks for this suggestion. We have attached a pdf file in the top-level Author Rebuttal. In figure 1 of the pdf file, we plotted the training loss of $L_z$ and $L_x$, and their test metric. We observe that during the training of the model using $L_x$, the loss $L_x$ exhibits a sharp decline in the first several epochs. Then the loss $L_x$ exhibits a very slight decline in the following epochs, while the test error continues decreasing. \n\n\n**Q:Without an understanding of how significant the final test errors are, it is not particularly insightful to merely show test errors after an arbitrarily selected amount of training data. Instead it would be nice to see scaling plots like in Figure 2 for the other systems in the analysis.**\n\nA:  In Figure 3, each model is trained with the same number of force computations. For each latent structure, we compare the relative performance of $L_{x, p}$ versus $L_{z}$, and $L_{x, p}$ always outperforms $L_{z}$. Thus we show our model is agnostic to different latent structures. Due to the time constraint, we find it difficult to finish the scaling plots. We will continue to refine our paper based on your valuable suggestion. \n\n**Q: Table 1 and Figure 2 are redundant**\n\nA: Thanks for pointing this out. We have revised the paper to remove Table 1 from the main content. \n\n**Q: Theorem 2 could be further clarified. What is $\\arg \\min L_{x,p}$ since $L_{x,p}$ is also a random variable?**\n\nA: In statistical learning theory, the empirical risk is considered a random variable. Denote the data by $X$ and assume $X \\sim \\mathcal{D}$. Let $X^n=$ {$X_1 , \\cdots, X_n$} be a collection of n i.i.d. samples from $\\mathcal{D}$. Then the empirical risk is :\n$$\nL_n(\\theta) = \\frac{1}{n}\\sum_{i=1}^n l_{\\theta}(X_i)\n$$\n\nwhere $l_{\\theta}$ is a loss function that depend on $\\theta$. Note that the empirical risk is a random variable since $X^n$ is a random variable. We can also take the expectation of the empirical risk $L_n(\\theta)$ to get the expected risk:\n\n$$ E_{X^n} L_n(\\theta) = E_X l_{\\theta}(X)  $$\n\nWhen we write $\\arg \\min_{\\theta} L_n(\\theta)$, it is implicitly assumed that the n data samples are already drawn from its distribution, then $L_n(\\theta)$ can be calculated deterministically. Now in our case, $L_{x,p}$ is a random variable. When we write $\\arg \\min_{\\theta} L_{x,p}$, we implicitly assume the data samples \n\n{$ x^i, f_{I(x^i)}(x^i)$ } $_{i=1, \\cdots, K}$\n\nare already drawn from its distribution, thus $L_{x,p}$ can be calculated deterministically. \n\n\n\n\n**Q: Is it difficult or tricky to tune the weight of the conditioning number loss term? What happens if the weight on this term is not high enough? What if it is too high? Ablation analyses of these factors would improve the paper** \n\nA: In Eq 7, the loss $L_{AE} = L_{rec} + \\lambda_{cond} L_{cond}$.  \n- If $\\lambda_{cond}$ is too high, $L_{AE}$ will be dominated by $L_{cond}$.  Then the autoencoder may not reconstruct the microscopic dynamics well, hence may not capture the closure terms well. If the latent space is not closed enough, we can not learn the macroscopic dynamics well.\n  \n- If $\\lambda_{cond}$ is too low, $\\varphi^{\\prime}(x)$ may be very ill-conditioned. In Theorem 1, the eigenvalues of $\\varphi^{\\prime}(x)^T \\varphi^{\\prime}(x)$ is lower bounded by $b_1$ and upper bounded by $b_2$, and we have:\n$$\nb_1(L_x + C) \\leq L_z \\leq b_2 (L_x + C)\n$$This theoretically guarantees the effectiveness of $L_x$ when $b_1$ and $b_2$ are close. If $\\varphi^{\\prime}(x)$ is ill-conditioned, then $b_1$ is much smaller thant $b_2$. Theorem 1 can not guarantee the effectiveness of $L_x$ anymore. \n\nIn our experiments, the value of $\\lambda_{cond}$ is determined through a logarithmic grid search (more specifically, 1e-7, 1e-6, 1e-5, 1e-4, 1e-3). Thus in our experience the tuning $\\lambda_{cond}$ is not very tricky.  Based on our experience, tuning $\\lambda_{cond}$ is relatively straightforward and not tricky.  \n\nBelow we show the test error of the Predator-Prey system. We vary $\\lambda_{cond}$ and train the autoencoder, then learn the macroscopic dynamics with $L_{x,p}(p=1/5)$ and report the test error. \n|  | | | | | | | |\n|  ----                                | ----      |----         | ----             |----        |----            |----                 |----             |\n|      $\\lambda_{cond}$    | 0     | 1e-8  |    1e-7    |  1e-6 |   1e-5   |     1e-4     |    1e-2   |\n|  Test error of $L_{x,p} (p=1/5)$             | 6.42e-02 | 2.62e-03 | 2.84e-03 | *8.36e-04* | 3.66e-03 | 2.60e-03 | 4.24e-03| \n\nWe can observe that the performance of $L_{x,p}$ may deteriorate when the parameter $\\lambda_{cond}$ is either too high or too low.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We would like to thank all reviewers for providing detailed reviews and constructive feedback that have improved the paper. 

Reviewer d4vx mentioned some relevant literature. We would like to compare these studies with our research to provide deeper insights into our method. 

**[A]** A1 includes many model order reduction methods such as the Krylov Projection Framework and proper orthogonal decomposition. A1 also mentions the data-driven reduced order method but considers the dimension reduction mapping to be the linear combination of several chosen basis functions. In this work, we use the neural network to parametrize the dimension reduction mapping, which has more flexible form but harder optimization problem. We want to mention that our method can be generally combined with any dimension reduction method as long as they can learn the closure to the desired macroscopic observables well. 



**[B]** In B1, B2, and B3, the 'partial observation' refers to an observable of the microscopic state $x$ (e.g. the observable is a function of $x$ ). These studies aim to obtain information (e.g., attractor) about the microscopic system from these macroscopic observables.

In our work, 'partial observation' refers to the computation of microscopic forces on a subset of particles. Our work aims to learn the macroscopic dynamics from microscopic forces. 


**[C]** The approach in C1, C2 belongs to the equation-free framework in [1]. The core difference between these studies and our approach is:
- C1 and C2 only consider partial differential equation (PDE) systems. The macroscopic observables in C1 and C2 are the solution of the PDE at the coarse spatial grid.  Then the macroscopic observables depend locally on the microscopic coordinates. In our work, the macroscopic observables depend globally on the microscopic coordinates of all the particles. In the patch dynamics scheme mentioned in C1 and C2,  the lifting and restriction operator can be viewed as the encoder and decoder in our case. The lifting and restriction operator is explicitly defined, but we need to train the autoencoder, which may cause some optimization problems. 
- The approach in C1 and C2 does not learn the macroscopic dynamics. Our method explicitly learns the macroscopic dynamics parametrized by a neural network.  When the closure to the macroscopic observables is difficult to learn, our method may not handle this case. However, the equation-free framework can still work by bypassing the derivation of the macroscopic evolution equations. 
- In C1 and C2, during the simulation of macroscopic dynamics, microscopic simulation still needs to be performed in small spatial domains and for short times. In our method, once the neural network for parametrizing the macroscopic dynamics is trained, we can simulate the macroscopic dynamics directly in macroscopic space, without performing any microscopic simulation. 

---
References:

[1] Gear, C. W., Hyman, J. M., Kevrekidid, P. G., Kevrekidis, I. G., Runborg, O., & Theodoropoulos, C. (2003). Equation-free, coarse-grained multiscale computation: Enabling mocroscopic simulators to perform system-level analysis.

Meta-review: The reviewers find this work studying an important problem and encourage the authors to incorporate the suggestions and developments during the review process.
Decision: Accept (poster)