Momentum Conserving Lagrangian Neural Networks
Authors: Ravinder Bhattoo
Sayan Ranu
N M Anoop Krishnan
Venue: ICLR 2022 Submitted (2022)

Abstract: Realistic models of physical world rely on differentiable symmetries that, in turn, correspond to conservation laws. Recent works on Lagrangian and Hamiltonian neural networks show that the underlying symmetries of a system can be easily learned by a neural network when provided with an appropriate inductive bias. However, these models still suffer from issues such as inability to generalize to arbitrary system sizes, poor interpretability, and most importantly, inability to learn translational and rotational symmetries, which lead to the conservation laws of linear and angular momentum, respectively. Here, we present a momentum conserving Lagrangian neural network (MCLNN) that learns the Lagrangian of a system, while also preserving the translational and rotational symmetries. We test our approach on linear and non-linear spring systems, and a gravitational system, demonstrating the energy and momentum conservation. We also show that the model developed can generalize to systems of any arbitrary size. Finally, we discuss the interpretability of the MCLNN, which directly provides physical insights into the interactions of multi-particle systems.

Under review as a conference paper at ICLR 2022
MOMENTUM CONSERVING LAGRANGIAN NEURAL NETWORKS
Anonymous authors Paper under double-blind review
ABSTRACT
Realistic models of physical world rely on differentiable symmetries that, in turn, correspond conservation laws. Recent works on Lagrangian and Hamiltonian neural networks show that the underlying symmetries of a system can be easily learned by neural network when provided with an appropriate inductive bias. However, these models still suffer from issues such as inability to generalize to ar- bitrary system sizes, poor interpretability, and most importantly, inability to learn translational and rotational symmetries, which lead to the conservation laws of linear and angular momentum, respectively. Here, we present a momentum con- serving Lagrangian neural network (MCLNN) that learns the Lagrangian of a system, while also preserving the translational and rotational symmetries. We test approach linear and non- non-linear spring systems, and gravitational system, demonstrating the energy and momentum conservation. We also show that the model developed can generalize systems of any arbitrary size. Finally, we dis- cuss the interpretability of the MCLNN, which directly provides physical insights into the interactions of multi-particle systems.
INTRODUCTION AND RELATED WORK
Realistic modeling of the time-evolution of multi martti-e physical systems lie at the heart of several fields of science and engineering (Goldstein, 2011: Park al., 2021; Roehrl al., 2020; Lutter et al., 2019). Examples include multiple bodies connected by springs, bodies interacting under gravitational forces, or even systems the atomic and mesoscale such as proteins or colloidal gels, which range several orders (+= _020) of length and timescales from atomic systems to planctary systems (Park al., 2021; Cranmer al., 2020b). Traditionally, the trajectory of these systems are obtained by solving the associated differential equations numerically. These differential equations, in turn, are derived based on the invariant or conserved quantities of a system. Recent studies on Lagrangian (LNN) and Hamiltonian neural networks show that one of these invariant quantities, energy, can be learned directly from the data enabling realistic simulation of systems (Cranmer et al., 2020a; Finzi et al., 2020; Lutter al., 2019; Greydanus et al., 2019; Zhong al., 2021; 2019; Zhong & Leonard, 2020). However, these approaches fail to model multi-particn interactions effectively due the following limitations.
Conservation of momentum: Invoking Noether's theorem (Noether, 1971), "a differentiable symmetry in action results in conserved quantity". Conversely, "every conserved quantity can be derived from an underlying symmetry in action". It can be shown that an interacting multi-particle system, that is closed, respects three conservation laws, namely, energy, linear momentum and an- gular momentum (Noether, 1971; Goldstein, 2011). While the first is a consequence of symmetry of the system with respect to time, the second and third is a consequence of symmetry with respect to translation and rotation in space, respectively. Thus, the Lagrangian of such system remains invariant under the translation and rotation operations on the Cartesian coordinates. For example, three balls connected by spring will have the same Lagrangian (and hence the interaction forces) when the system as a whole subjected to rotation or translation. That is, as long as the relative positions of the balls are preserved, translation or rotation by any magnitude will not have effect on the system dynamics. While LNNS have been demonstrated to learn the symmetry in time by conserving the energy (Cranmer et al., 2020a), they have not been designed respect the other two conservation laws with respect to momentum. Learning these symmetries directly from the data is an extremely hard problem and may require large amounts of data. Furthermore, we will show

---

Under review as a conference paper at ICLR 2022
in Sec. 4, even when trained on large volumes of training data, the LNNS fail generalize well on unseen data. This questions the realistic nature of the dynamics simulated by LNNS
Generalizability to unseen system sizes: LNNS lack the ability generalize to system sizes beyond the training set. For instance, an LNN trained on three balls connected by a spring cannot model system of four balls connected by spring despite having the same conserved quantities and interactions. This limits the broader applicability of LNNS to realistic physical systems.
LNN (Cranmer et al., 2020a) directly learns (and predicts) the Lagrangian of a system as a function of the position and velocities of all the particles in the system. Owing to this design, it is not possible recover the inter-particle dynamics within multi-particle system in the form of pairwise potential energy or forces. This limits the interpretability of the LNN as the contribution of positions and velocities towards the total Lagrangian of the system is represented using a black-box deep neural network.
Interpretability:
In this paper, we address the above limitations. Specifically, our contributions are as follows:
Problem Formulation and Architecture Design: We reformulate LNN (Cranmer et al., 2020a) with relational inductive bias by applying a transformation on the Cartesian coordinates of the system and decoupling the terms of the Lagrangian. Armed with this reformulation, we develop a momentum conserving LNN called MCLNN. MCLNN introduces several key innovations. (1) First, MCLNN preserves the rotational and translation symmetries. (2) Second, MCLNN gener- alizes to unseen system sizes. (3) Third, MCLNN generates interpretable models with the output characterizing pair-wise interactions of the bodies in the system, and thereby providing physical insights into the system dynamics.
Theoretical Characterization: We rigorously establish that MCLNN conserves energy, linear and angular momentum. Furthermore, in contrast to LNN (Cranmer et al., 2020a), we show that MCLNN can directly learn trajectories (positions) of systems without requiring training on accel- eration. This is desirable property. In many cases including experimental systems, access to acceleration of each entity every time instant may not be possible. While position of particles is a direct observable, acceleration is a derived quantity.
Empirical Evaluation: We perform in-depth evaluation across three multi- cuutti-artticle systems namely, balls connected by linear and non-linear springs, and gravitational system. Our exper- iments establish that MCLNN performs significantly better than LNN with long-term stability, generalizes to arbitrary-sized multi-particle systems, and is capable of learning interaction dy- namics with 3-4 orders of magnitude lower volume of training data.
2 PRELIMINARIES AND PROBLEM FORMULATION
In this section, we introduce the preliminary concepts central to our work and formulate the problem. As notational convention, vectors represented with an overhead arrow (Ex. 7) and higher-order tensors, such as matrices, in bold. A summary notations used is provided in the appendix A.7. Definition 1 (Multi-particle System) An -ody multi-particle system contains N particles P {NI,... n. At any given time f, particle n; is characterized by its position oj and velocity qi' The positions of particles in an A-body system are not static. They change due various interaction forces at play (Ex: a gravitational system). These positional changes are captured in the form of trajectories.
Definition (Multi-particle Trajectory) The trajectory of an N-body multi-particle system P over a time horizon T = [1s: Le] is the sequence of position and velocity vectors (q ( [
where q- = In physics, the time-evolution or trajectory of interacting particles is obtained by solving the dif- ferential equations of motion. The numerical solution to these equations provide the acceleration of the particle, which can then be used to obtain the updated velocity and positions. Indeed, the acceleration of particles can be directly learned by neural networks as a function of its position and velocities. However, it fails to learn the underlying symmetries, which in turn leads to the laws of conservation of energy and momenta.
( (7. n, E P) and q' = {qp n& E P).
Recent approaches to predict trajectories by learning the Lagrangian through an LNN (Cranmer et al., 2020a; Finzi et al., 2020; Zhong et al., 2021; Zhong & Leonard, 2020) has been shown to be effective in learning the symmetry in time (Cranmer et al., 2020a; Zhong et al., 2021). Lagrangian, defined as [. 4) T(q)- V(g), , is scalar functional of kinetic energy (T())) and potential energy (V (V()),
2

---

Under review as a conference paper at ICLR 2022
Timestamp Position Informel.oo on
Lagrangian computation
Predicted Poter Ual Energy
Accelerztion
Timestamp
PP40 R2 (Fe
cime innegration
V
(q""1.a""))
body conj (guracion
(Updsted positior and velocilies)
Velcoity [ 4: ..u]
= V
Kinelic Energy
Lagrangiar
Iteration over t
Figure 1: MCLNN framework.
that depends on both the set of positions q. and velocities å of the particles in the system (Goldstein, 2011). 1). LNN bypasses the necessity to learn accelerations for each particle and directly predicts scalar Lagrangian for system. Additionally, the trajectory predicted using Ealer-Lagrange (EL) equation conserves energy resulting an overall better prediction the system dynamics.
Theoretically, although learning the Lagrangian of system is enough to predict its dynamics, the Lagrangian itself exhibits some symmetries that are not imposed by the EL equation. Lagrangian of system exhibits translational and rotational symmetry. Specifically, the Lagrangian of a system L(q. á) and L(q', 4). where q' = { + n4 E P} for any finite value of are equivalent. Similarly, L(q. q) and L(Qq, q). where Q is orthogonal tensor representing rotation, equivalent. These symmetries are not provided as an inductive bias in the LNN framework.
Further, LNNS trained by giving the coordinates input and minimizing the loss between the actual acceleration and acceleration predicted by LNN through EL equations. This requires a priori access to the actual acceleration, which may not be available in many cases, for example, experi- mental trajectory of colloidal gels visualized by video camera.
Here, aim address these open challenges by reformulating the LNN with a relational inductive bias, resulting in Momentum Conserving LNN (MCLNN). Specifically, we aim to learn the dynamics of multi-partice interacting systems purely from the trajectory of the constituent particles.
Problem 1 (Momentum Conserving LNN for Trajectory Prediction) Let be set of trajecto- ries of N-body systems. Furthermore, let there be a hidden joint distribution of physics-constrained configurational and temporal space from which T. has been sampled. Our goal is to learn this hidden distribution. Towards that end, we want learn:
A generative model TTT) that maximizes the likelihood of generating ii. The generative model must respect the laws of physics such conservation of energy and momenta.
Once learned, this generative model can be used to predict trajectories of unseen N N-body systems.
3
THEORY
The flowchart of MCLNN is shown in Fig. 1. In this approach, the positions and velocities any time t is used to predict the trajectory of the system, while respecting the conservation laws of energy and momenta. We analyze the performance of MCLNN on three systems, namely, linear spring, non-linear spring, and gravitational system to predict the accurate dynamics of these multi- particle interacting systems.
3.1 BULER-LAGRARE EQUATION
Consider system A -particless interacting with each other under a potential. Let define func- tional, namely, "action", S as:
S= L dt
(1)
where the Lagrangian of the system. Then trajectory taken by this system to move from the position qo q² in time to l is the one that makes the "action" S stationary. This leads the
3

---

Under review as a conference paper at ICLR 2022
equation of motion governing the dynamics of system, namely, the EL equation d aT 0T.
aq Ōq
The acceleration of particles the system, q = ( n, E P) can be computed directly from the
EL equation (see Appendix A.3) as:
q (V V (V q)
This acceleration can then be used to compute the updated positions and velocities of particles. Note that EL equation leads to the conservation of Hamiltonian, defined as: H(q. q) = T()) V(g), which also represents the total energy of the system.
3.2 MOMENTUM CONSERVING LAGRANGIAN NEURAL NETWORK
As mentioned earlier, Lagrangians exhibit translational and rotational symmetry. Learning this di- rectly from the positions of particles extremely challenging. To address this challenge, we refor- mulate the LNN. Consider five balls connected by linear springs. We will use this running example explain MCLNN. The input for the MCLNN the position and velocities at any time t. Further, we identify the set of all the pairs of particles in the system as (i,j). For given set, we compute the difference between the positions followed by computing the 12 norm to obtain the pairwise distances aij. which corresponds to the length of the spring. Thus,
9. v (q Vla
4 (g 9 -g)
(4)
The Qij is given as the input to a neural network which outputs a scalar Vij. Note that the Vi, may be considered be the pair-wise potential energy of a spring connecting two particles and j. Vig summed over all the pairs of particles gives the ((g), the total potential energy of the system. For multi-particle systems, the kinetic energy of particle n, is the function of its velocity d can defined as (Goldstein, 2011; Zhong et al., 2021):
:2 T ( m.
(5)
Here, we invoke this expression of kinetic energy, which when summed over all the particles provide the total kinetic energy. T'. Then, T and V are used to obtain the L as in Eq. The L when substituted into the EL equation (Eq. 2), provides the acceleration and the updated position and velocities of the system using the velocity-Verlet integration (Rapaport, 2004). The loss function of MCLNN is on the predicted and actual positions at time t + 1, t 2,..., É + back pockppropagated to train the neural network. Specifically, the loss function is as follows.
which is then
L f=
- T TTT
Σ
Σ (T -T<k=)((T'))))
T
N
Here, p(T p(T.q') is the predicted position for the 2th particle in T at time and T.q. is the true position. [7] denotes the last time step in trajectory T. The updated expression for Lagrangian for an --particle system can be written as:
L(q. ④)
22 m
Viy(as) i-j==++
It should be noted that the specific form of kinetic energy T does not limit the applicability of the present approach (Zhong et al., 2021). Since, the focus of the work to model multi multi-particle systems and it is well-known that the kinetic energy of the system generally follows Eq. 5, we choose this expression to simplify the problem. In cases where the expression for kinetic energy is not known or different, the function can be replaced by additional neural network with an approach similar to that for V.
3.3 TRANSLATIONAL AND ROTATIONAL SYMMETRY OF MCLNN
Theorem MCLNN exhibits translational and rotational symmetry.
PROOF. Consider, the Lagrangian of the system of N --particles as L(q. q) = T(4) V(q). First, focus the translational symmetry and next rotational symmetry.
4

---

Under review as a conference paper at ICLR 2022
Lemma MCLNN exhibits translational symmenry.
PROOF. When the particles are subjected to a translation E, the updated positions are given by q== ( + + n, E P). The updated Lagrangian of the system is given by L(q', q). To prove L(q'àq) = L(q. q), we need to prove V(q') = (())) as the other term in the Lagrangian remains unaffected. Applying the transformation the positions and computing the 1 norm,
h3 = V(0
[10 + + (0) +0)
1 + 2) (06 + +))
= -V (Th - t (g: - xz)
4i
(8)
which implies V(d')= (q). Therefore, L(q', à) L(q. d).
Lemma 2 MCLNN exhibits rotational symmetry.
PROOF. When the system is subjected to pure rotation, the positions are transformed as Qq, where Q is an orthogonal tensor representing rotation. Correspondingly. the Lagrangian is modified as L(Qq. Q). As in the case of translation, is worth recalling that, for an orthogonal tensor Q, QQ' = Q'Q = I Now, the proof can be
need only to prove that V(Qq) = V (q). To prove this,
obtained by computing 0/1 as:
(QQ - Q (QT Qń)
V Q(4 - ( Q - (
QQT [g) - ()) (
V (q: 43) (g: 4g)
This implics that (Qq)) = V(q) and hence, Liq. q) = L(qq, 4). Thus, we demonstrate that the Lagrangian in MCLNN exhibits both translational and rotational symmetry. Invoking Noether's the- orem, can be proven that the Lagrangian preserving these symmetries will consequently conserve of linear and angular momenta, respectively. (For proof, App. A.4 and A.5).
Combining Lemma and Lemma 2, we get that the MCLNN will respect the laws of conservation energy, linear momentum, and angular momentum.
3.4 GENERALIZABILITY AND INTERPRETABILITY
The MCLNN exhibits granular structure, which applies the computations individual or pair- wise entities that are aggregated to obtain the total Lagrangian of the system. This structure, by design, allows model generalizability unseen system sizes. Once trained on an n-body system, the MCLNN can be applied on an m-body system (m + n) directly by replacing with m in Eq. 7.
Finally, we discuss interpretability of MCLNN. The main function learned by MCLNN the Viy. which depends qij. Since, Vij adds to form the total potential energy V(g) of the system, it is reasonable assume that Vig represents the pair-wist wise potential energy. we demonstrate later in Sec. 4.4, we show that Vij indeed corresponds to the pair-wise potential energy of the ground truth.
EXPERIMENTS
In this section, we benchmark MCLNN and establish:
Accuracy: MCLNN is more accurate in modeling trajectories of N-body systems than LNN (Cranmer et al., 2020a).
Conservation of Physics: Consistent with the theoretical analysis in Sec. 3.3, MCLNN preserves
the laws of physics more comprehensively than LNN
Generalizability and Interpretability: MCLNN generalizes to unseen data better than LNN and enables higher interpretability due predicting the potential energy instead of the Lagrangian directly.
The codebase along with all baselines used and the datasets available at the anonymous reposi- http://wwwwppp.oooooo./////////a wwtpm//iiiiiee//ncmcco.N------
tory https://nonnymou..emm www.uusssoopnssso...m
5

---

Under review as a conference paper at ICLR 2022
4.1 EXPERIMENTAL SETUP
We use JAX (Bradbury et al., 2020), a high- sigh-performance numerical computing python package and JAX MD (Schoenholz & Cubuk, 2020), a JAX based molecular dynamics package for the n-body simulations.
4.1.1 TASKS
To benchmark the performance of MCLNN, we conduct experiments on three multi-body systems, namely, linear and non-linear springs, and gravitational systems.
Linear spring. We simulate the dynamics of three particles (e.g.: balls with volume tending to zero) connected with each other by linear springs. The initial conditions of the simulations include the positions and velocities of the three particles and the stiffness of the spring. The potential energy of two balls connected by spring is given by Vij } qu)² where qj is the instantaneous distance between the springs and qu is the equilibrium distance, and & is the stiffness of the spring.
The force between connected pair of particles is computed Fij
dV21 - k(g:; qo). Note
dms
that the value of k is kept constant for all the springs.
Non-linear spring. We simulate the dynamics of three particles connected by non- non-linear springs. The initial conditions of the simulations include the positions and velocities of the three particles. The potential energy of two particles connected by a non-linear spring is given by Vi, = k(g 20)² 1 where Qt is the instantaneous distance between the springs and go is the equilibrium distance, and l is the stiffness of the spring. The force between connected pair of partieles is computed as Fiy -2k 90) The value of k is kept constant for all the springs.
Gravitational system. We simulate the dynamics of four particles interacting with each other under the gravitational force. The initial conditions of the simulations include the positions and velocities of the four masses. The potential energy of a pair of particles with mass mi and my if Cmnymy where G is universal gravitational constant and (5 is the instantaneous 9:
given by ViJ = distance between the pair of particles. The force between the pair of particles is computed as Fi- Gmima, Note that the mass m of all the particles are kept constant.
We use forward simulation N particles to generate training data corresponding to each task. First, NV W-body configuration with given initial positions and velocities are consid- ered. The potential energy of the system is computed from the positions using analytical expressions associated with each task as described above. The force on each particle is then calculated using as the gradient of potential energy, which is used to compute the acceleration. Finally, the velocity- Verler algorithm is used for time integration obtain the updated positions of the particles. Please see the appendix (A.6) for configurations (including the number of particles, initial configuration. and time step) of each of the tasks.
Data Generation:
4.1.2 BASELINE
The original LNN takes both the positions and velocities of all the particles as input and gives the total Lagrangian as the output. In the present case, we make the assumption on the functional form of kinetic energy (see 5q.5). As such, to ensure fair comparison, a slightly modified version of the original LNN is considered here as the baseline. For this baseline, the positions are given as input to the neural network which gives the potential energy V(Ø) as output. The kinetic energy is computed using the functional form, from which the Lagrangian is computed. All the remaining training procedures are maintained same as in the original LNN. It should be noted that decoupling the T and V terms and providing the expression for T should make the learning easier for the LNN and hence should give improved performance in comparison to the original LNN. This model will be referred to as bascline or baseline LNN, henceforth.
4.1.3 METRICS
To analyze the performance of the MCLNN in comparison to the baseline, the choice of the ap- propriate metrics is crucial. Since the systems considered here are chaotic, slight differences in the initial conditions predictions will lead to differences in the trajectories. More importantly, given the current state system, multiple "correct" trajectories may exist as long as they represent the degenerate equivalent configurational states of the system. As such, purely computing the error in trajectory with respect to the ground truth may not be representative of the performance of the model. Learning and predicting the Lagrangian ensures that the state of the system is represented ac-

---

Under review as a conference paper at ICLR 2022
04:11
0.21
04.31
MORN Foscine (irerrsppog
MONN
MCTMR strin)
NCTM cotine
12
Rosclive Hireer spring
Baod ne 2o spring
Booe ee O. Spring
1.27
127
0.0
0e.00.
100
60
200
60
00.31
(3.21
stes
10.31
10.41
5700
1500
ScUr foicer spring
VOURM near spring Bosciin lirest sprigg
NCCUM 0 Lioaa string
Ze+60
NC dlecar soring
buocine tur sprigg
baad no lu Horr sprigg
5ood ns Suaoo spring
Laginggan
2200
20102-
0.
Lreet Brow
300
-1.0
0e+00
Time
Time
Figure 2: Results of the forward simulation of linear spring system from the same initial config- uration using ground truth (continuous line), baseline LNN (dashed and dotted line) and MCLNN (dashed lines). (a) The evolution of Lagrangian, Hamiltonian (total energy), linear and momentum for all the systems. (b) The error in Lagrangian, Hamiltonian (total energy), linear and momentum predicted by the baseline LNN and MCLNN with respect to the ground truth.
curately, resulting in the prediction of the same or equivalent trajectories of the system. Further, the trajectories should respect the spatial and temporal symmetries resulting in conservation of Hamil- tonian (total energy), linear, and angular momenta. Thus, we focus on the Mean Absolute Error (MAE) between predicted (1) Lagrangian, (2) Hamiltonian (total energy), (3) linear and (4) angular momenta, with respect to the ground truth.
4.1.4 TRAINING
For training MCLNN, 100 different trajectories, each having 20 points with fixed time intervals, used for all three tasks. Each trajectory has different initial conditions and hence represent different configurations that accessible to the system. The model performance is evaluated by comparing the predicted trajectory with respect to the ground truth for the same initial state.
For training LNN, we sample 10,000 data points (set of (q, q) as input) and q as output) from for- ward simulations. The training is performed using the mean squared error between the accelcration predicted by the EL equation and the ground truth. The detailed configurations and parameters associated with simulation and training of each task is provided in App. A.6.
4.1.5 INFERENCE
To asses the long term stability of models, we perform forward simulations using trained LNN and MCLNN models for a prolonged time interval for each task. The initial condition and other parameters are kept same for both LNN and MCLNN. Furthermore, the initial conditions of the trajectories during inference are diffferent from those encountered during the training. Consequently, all trajectories during evaluation are unseen. We record Lagrangian, Hamiltonian, linear momentum and angular momentum for the whole trajectory, The predicted forward simulation compared with the ground truth data as generated in Sec. 4.1.1.
4.2 ACCURACY
First, we focus on the linear spring system with three particles. Figure 2(a. (a.44) shows the La- grangian, Hamiltonian (total energy), linear, and angular momenta of the system. The error in these quantities predicted by the baseline LNN and MCLNN are shown in Figure 2(b. (bb.4). We observe that the Lagrangian predicted by the baseline LNN starts diverging after 30 time steps of the forward simulation. We notice that the divergence in the Lagrangian is accompanied by a divergence in the linear momentum as well. Up on further simulation, the angular momentum also starts diverging. In contrast, the Lagrangian predicted by the MCLNN follows the ground truth with very little error. Further, the Hamiltonian, and linear and angular momenta remain conserved in the MCLNN. This suggests that the trajectory predicted by the MCLNN is accurate and stable with no long term drift or divergence. It is worth noting that, for the baseline LNN, despite the poor predictions of the La- grangian at higher values of time steps, the total energy remain conserved. This points to the fact that the energy conservation is a constraint enforced by the EL equation and is not a reflection on the quality Lagrangian. Any value of Lagrangian can still yield a constant Hamiltonian (energy), provided the constraints as imposed by the EL equations on the Lagrangian are satisfied. As such, LNNS should be evaluated considering multiple metrics as demonstrated here.
7

---

Under review as a conference paper at ICLR 2022
04:11
52
Rose pootton Inoor spring
auet sping
Baselie (oo0 sprig)
Rasolie (oon recy Sprng)
Roccting 1oon inoor sprrig
0e+00
2000
3000
2000
2000
3030
1030
2000
3200
2000
2000
06:11
Tire Atep
10:20
16.31
stop
0.41
step
MCUUN (Wio liner spring
Busc tin (Man lanoe Jon
MCLU (oon (neeropring
Boadie Ntan spriy)
Droe nar nerreetuu
MCLIN (Non ircrrsptyyi
Scadin (Mar oour gling)
20+33
Buasting vo (ftcn insu sphr
MC. thi lncor opn
Cree 100
Div Rn
-1.0
1003
0n+30
Time
Time
Model (Crentationall tie (Gravitation))
Modil (Crectationall Eneeliie (Grniiaaional)
Mocel (Gredtarioat
3e+22
Model (arenlttionll
Baseline (Gravitation ora)
Basedina (Gra Instioal)
1125
22112
24:22
0++32
08:31
Time stop
14:20
Time 1000
(6.3)
stop
01:4)
Time step
300
MICUM tationali Booe live (Grnslatoond)
MCUUN (Gra Iational Roeeline (irsstation))
MCLIN (Graitattoee
Se+22
MCAAN (Grevizatinnll
Bsoelie (Sravisntera srab
Boreline (lri bntional)
Ligergah
Eroris
500
30.00
3500
300 1000
1500
350 1200
1300
300 3000
3300
Figure 3: Results of the forward simulation of non-linear spring, and gravitational system from the same initial configuration using ground truth (continuous line) and MCLNN (dashed lines). (a) The evolution of Lagrangian, Hamiltonian (total energy), linear and momentum for all the systems. (b) The error in Lagrangian, Hamiltonian (total energy), linear and momentum predicted by the baseline LNN and MCLNN with respect to the ground truth.
Now, we focus on the non-linear spring and the gravitational system. Figure 3(a) and (c) shows the Lagrangian, Hamiltonian (total energy), linear, and angular momenta of the non-linear spring and gravitational systems. The error in these quantities predicted by the baseline LNN and MCLNN are shown in Figure 2(b) and (d). As in the case of the linear spring system, we observe that there is long term drift in the Lagrangian predicted by the baseline LNN. Similarly, the linear and angular momenta of the baseline also diverge in few time steps. In contrast, the Lagrangian predicted by the MCLNN exhibits a good match with the ground truth. It should be noted that the multi- particle interacting systems are chaotic in nature and hence the exact trajectory of the ground truth and simulated system may vary. However, the low values of error in the Lagrangian along with the long-term stability of both non-linear spring and gravitational system suggests the realistic nature of the trajectory predicted by MCLNN. In addition, we observe that the linear and angular momenta of the system remain conserved in the MCLNN. Altogether, the results confirm that the MCLNN can successfully learn the generative model p(TT) that can sample the trajectory effectively from the physics-constraing configurational and temporal space.
4.3 GENERALIZABILITY TO UNSEEN SYSTEM SIZES
To demonstrate the generalizability of MCLNN, we use the generative model P(T)) for each of the tasks learned during the training. The models are then used to predict trajectories of systems
(a)
(b)
(c)
8F1
1.0
Linear spring (6 bodies)
Non-linear spring bodies)
Gravitational (8 bodies)
MCLNN
2.0
MCLNN
MCLNN
Lagrangian
0.8
Lagrand
Lagrand an
6-
0.6
0.5
0.4
0.2
0.0
100
150
200
50
100
150
200
0
50
100
150
200
Time step
Time step
Time step
Figure 4: Results of the forward simulation of (a) linear spring with 6 particles, (b) non-lineas spring with 6 particles, and (b) gravitational system with 8 particles from the same initial configuration using ground truth (continuous line) and MCLNN (dashed lines). The evolution of Lagrangian for all the systems are plotted.
8

---

Under review as a conference paper at ICLR 2022
(a)
(b)
(c)
0.0
Linear spring
Non-linear spring
1.0
1.25
MCLNN
MCLNN
-0.5
0.8
1.00
Vilga)
0.6
0.75
-1.0
0.4
0.50
-1.5
0.2
0.25
Gravitational
MCLNN
0.0
0.00
-2.0
1
2
2
0
10
15
qjj
Figure 5: Variation of Vi, with respect li learned by the MCLNN (dashed line) in comparison to the ground truth (continuous line) for (a) linear spring, (b) non-linear spring, and (c) gravitational potential. The shaded region represents the range of qi values in the training set. Non-shaded region between two shaded region corresponds to interpolation; otherwise, it represents extrapolation.
with different number of particles. Figure 4 shows the trajectories for linear and non non-linear springs with six particles (MCLNN trained on three particles). and gravitational system with eight particles (MCLNN trained on four particles), predicted by the MCLNN in comparison to the ground truth. We observe that the Lagrangian predicted by the MCLNN exhibits excellent match with the ground truth. In addition, all other quantities such as energy and momenta remain conserved (see Appendix Fig. 7). This suggests that once the generative model p(T) has been learned, MCLNN can generalize it to predict the trajectory of systems with any number of particles. Note that the baseline LNN cannot simulate systems of different sizes as the positions of all the particles are given simultaneously as the input to the neural network.
4.4 INTERPRETABILITY
Figure shows the Vi predicted by the MCLNN in comparison to the pair- wise potential energy obtained analytically from ground truth. The shaded region represents the bi values that were present the training set. observe that the MCLNN is able to learn the pairwise potential energy function accurately for the Øy values in the training set. Further, the MCLNN is able interpolate values excellently and extrapolate reasonably. It is worth noting that the MCLNN is trained only on the trajectories of the system. Hence, information regarding the potential energy, force, acceleration is given to the neural network during training. This suggests that MCLNN is indeed able to learn the physics of the problem by training purely the trajectory.
5 CONCLUSION
We introduced a new framework, namely, momentun conserving Lagrangian neural networks (MCLNN), for incorporating physics-based priors in neural networks for accurate simulations of multi-particle systems. We demonstrated that the MCLNN respects the symmetries in space and time, leading to conservation of energy, and linear and angular momentum. We showed that the incorporation of these additional conservation laws of momenta makes the Lagrangian of the system stable by avoiding any fong-term drift in it. This, in turn, results in realistic simulation of multi- body systems. Further, we showed that the MCLNN once trained, can generalize to systems of any size. Finally, we demonstrated that the MCLNN highly interpretable and provides direct insights into the interaction laws governing the dynamics of multi-particle systems. This, in turn, allows one to verify the realistic nature of the function learned by the MCLNN.
At this juncture, it is worth discussing some of the open questions and shortcomings of MCLNN that can be addressed as part of future works. (i) MCLNN assumes interaction between all the particles in the system. A graph-basee MCLNN with contrastive loss can potentially address this challenge by incorporating the topology of the particle system along with differential importance for nearby particles. (ii) MCLNN considers only pair-wise interactions for computing the Vij- However, there could be additional interactions involving three (VeR),, four (Vikt) or even higher particles. This could be addressed by feature engineering or graph-based MCLNN. (iii) Similarly, simulations in- volving different types of interacting particles is challenging in MCLNN as it may lead to non-unique solutions. (iv) In addition, present work can be extended address more challenging problems, for instance, to learn the Lagrangian of system with non- con-conservative forces, and to lear generalized kinetic energy functions, while maintaining the granularity and generalizability.

---

Under review as a conference paper at ICLR 2022
6 REPRODUCIBILITY STATEMENT
Please find all codes and experiments at the following anonymous link: https: www.en.ccciennee//nboyyl..eo.ee
In Appendix A.6, we have given addi- tional details to make the work completely reproducible. In particular, we have: (1) additional derivations and proofs in App. A.3.A.4.A.5,. (2) initial conditions, task configurations, and param- eters for the training of neural networks used for each task in App. A.6, (3) detailed notation in App. A.7. The dataset generation (using simulations) step is added to the code before training step and hence there are no additional dataset files are provided.
REFERENCES
James Bradbury. Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, and Skye Wanderman-Milne. Jax: composable transformations of python+ numpy programs, 2018. URL http://githubb. com/google/jax, 4:16, 2020.
Miles Cranmer, Sam Greydanus, Stephan Hoyer, Peter Battaglia, David Spergel, and Shirley Ho. Lagrangian neural networks. In ICLR 2020 Workshop Integration of Deep Neural Models and Differential Equations, 2020a. URL https://openreview net/forumm
Miles Cranmer, Alvaro Sanchez Gonzalez, Peter Battaglia, Rui Xu, Kyle Cranmer, David Spergel, and Shirley Ho. Discovering symbolic models from deep learning with inductive biases. Advances in Neural Information Processing Systems, 33, 2020b.
Mare Finzi, Ke Alexander Wang, and Andrew G Wilson. Simplifying hamiltonian and lagrangian neural networks via explicit constraints. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Bal- can, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 13880-13889. Curran Associates, Inc., 2020. URL
fle2996655e884444aaddd6888 5cco0lee Pcco00e-Paaer..dpd...
Herbert Goldstein. Classical mechanics. Pearson Education India, 2011.
Samuel Greydanus. Misko Dzamba, and Jason Yosinski. Hamiltonian neural networks. Advances in Neural Information Processing Systems, 12:15379-15389, 2019.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR (Poster), 2015. URL http://arxxiv.org/bbbb /ttpabss/11122 412.6980.
Michael Lutter, Christian Ritter, and Jan Peters. Deep lagrangian networks: Using physics model
prior for deep learning. In International Conference on Learning Representations, 2019. URL
Emmy Noether. Invariant variation problems. Transport theory and statistical physics, 1(3):186-
207, 1971.
Cheol Woo Park, Mordechai Kornbluth, Jonathan Vandermause, Chris Wolverton, Boris Kozinsky, and Jonathan P Mailoa. Accurate and scalable graph neural network force field and molecular dynamics with direct force architecture. npj Computational Materials, 7(1):1-9, 2021.
Dennis Rapaport. The art of molecular dynamics simulation. Cambridge university press, 2004.
Manuel A Roehrl, Thomas Runkler, Veronika Brandtstetter, Michel Tokic, and Stefan Obermayer. Modeling system dynamics with ohysics-informec neural networks based on lagrangian mechan- ics. PAAC-PaperSOnl.ine ine, 53(2):9195 9200, 2020.
Samuel Schoenholz and Ekin Dogus Cubuk. Jax md: a framework for differentiable physics. Ad- vances in Neural Information Processing Systems, 33, 2020.
Yaofeng Desmond Zhong and Naomi Leonard. Unsupervised learning of lagrangian dynamics from
images for prediction and control. Advances in Neural Information Processing Systems, 33, 2020.
Yaofeng Desmond Zhong, Biswadip Dey, and Amit Chakraborty. Symplectic ode-net: Learning hamiltonian dynamics with control. In International Conference on Learning Representations, 2019.
10

---

Under review as a conference paper at ICLR 2022
Yaofeng Desmond Zhong, Biswadip Dey, and Amit Chakraborty. Benchmarking energy mergy-connsevvngg neural networks for learning dynamics from data. Learning for Dynamics and Control, pp- 1218-1229 PMLR, 2021.

---

Under review as a conference paper at ICLR 2022
A APPENDIX
A.1
BASELINE LNN
Figure shows the predicted force with respect the actual force for the training data of baseline for the three tasks.
(a)
(b)
(c)
1.25
Linear spring
2.5
Non-linear spring
3.0
Gravitational
MSE = 1.05e-03
MSE = 5.72e-03
2.5
MSE
2.31e-03
Predicted force
1.00
0.98
Predicted force
2.0
R² 0.98
Predicted force
R²
0.99
0.75
0.50
1.0
0.25
0.0
0.0
0.0
0.5
1.0
0
0
Measured force
Measured force
Measured force
Figure 6: Predicted force for (a) linear spring, (b) non-linear spring, and (c) gravitational system with respect to the measured forces on the test dataset of baseline LNN.
A.2 GENERALIZABILITY TO UNSEEN SYSTEM SIZES
Figure 7 shows the generalizability of the MCLNN to unseen system sizes for the three tasks consid- ered. For the three tasks, the number of particles considered are twice that of the training system size. We observe that the Lagrangian for the unseen system is predicted in excellent agreement to the ground truth for all the three tasks. In addition, the Hamiltomian (total energy), and momenta are conserved in all the three tasks, confirming the realistic nature of the simulations MCLNN.
(a.1
1.d
Lineer spring 16 bodiesi MCLUM
Linwer spring bodies)
Liwer momamua
Lineer spriin (3 boskee)
2:00
Linwer spring 1o bodiest
Homitonaan
MCUNN
VCCUNN
125
MCLNN
1.50
1.25
1.00
-1.0
100
0.75
130
1.
Mon-limear spring bodkess
Non- spring bodies!
monertian
Nor-ineer spring 16 bodkes)
2.00
Mon-llnear spring bodiest
Hom Homttonan
emoy)
MCUNN
0.5
MCLNK
2./5
MCCUNN
0.0
1.50
c.0
0.75gg
61.11
Time Step
(c.2)
Time step
(C-3)
1.0210
1220:1
Time step
(c44
Time Step
-1.07
Liwar momeittaa
Gravitational
Anquua momorrum
Lggo-gas
HLIO Haniaa
0.2
0.0
0.0
100
1:30
400
20
1000
200
100
150
120
Time StOP
Time 3750
Time 2000
Time St0p
Figure 7: Results the forward simulation of (a) linear spring, (b) non-linear spring, and (b) gravita- tional system from the same initial configuration using ground truth (continuous line) and MCLNN (dashed lines). The evolution of Lagrangian, Hamiltonian (total energy), linear and angular mo- menta for all the systems with different number of particles than that present in the training are plotted.
12

---

Under review as a conference paper at ICLR 2022
A.3 COMPUTING ACCELERATION FROM BULER-LAGRANE EQUATIONS
The acceleration of particles can be obtained from the Euler-Lagrange equations as follows.
dL
aL
oq dt
dq
0 [T dg
dT.qq
OL
0q
0q dt
0q dt
pq
0 0L
10
+
aq 3q
bq
0 1L
0 0L
4
0q pq
09 0q
AL
AL,
q
04 0q q
0q 09
bq
AL
OL
OL
0q oq 4
0q
(Vo4L)
[V
A.4 TRANSLATIONAL SYMMETRY
The conservation of linear momentum can be derived from translational symmetry of Lagrangian follows.
L(q q)
L(q e,
= +c
dq
0q
ØL
IL
OL 0q
θL
Σ dq
Σ
Σ
0q'
dq 0q'
oq
8T. Σ 89
0q
QL
8L Σ
0q
ØL
78 Σ
dq
13

---

Under review as a conference paper at ICLR 2022
since,
5T. = 0
OL
Σ dq
Σ OL
0
Ōq
ÖL
Σ di 04
d ÔT.
Σ Qq
0
di
Σ aL.
constant
Σ P: constant (linear momentum)
where Pt
al Jq.
A.5
ROTATIONAL SYMMETRY
The conservation of angular momentum can be derived from rotational symmetry of Lagrangian as follows.
Under an infinitesimal rotation 80
5q, = 58 x q
and
5q. = 50 x q0
since, Lagrangian does not change with infinitesimal rotation
8L =
Σ
ØL
5qp +
Σ
aL 0q: = 0
Oqj
0q.
using generalized momentum
ØL
p.
0q.
14

---

Under review as a conference paper at ICLR 2022
then EL equation gives
ØL
dt Pi
=0
dq
AL
Pi 0q:
ST. = Σ
pr dqj + Σ
Di oq. 0
Σ Be (50 x ( + F1 (60 x (.) =0
Σ 80. (7) x (.) + 60 (7 (1)
Σ 50 (07 x (
( pb)]
Σ 50. d
(% x p.) -0
dt
because 50 is arbitrary, then
Σ dt
f7 x =0
x p.)
di
angular momentum
constant
A.6 INITIAL CONDITIONS AND TASK CONFIGURATION
The number of hidden units are chosen from hyperparamter search given Table 1. The activation function for all hidden units square plus fx) (x+v=*+4) which is similar to soft plus (Cran- mer et al., 2020a). We use ADAM (Kingma & Ba, 2015) optimiser for model training with learning rate given in respective configuration tables. We velocity-Verlet for time integration during trajectory evolution. Mass of all the particles are maintained as 1.0 units.
A.6.1 LINEAR SPRING SYSTEM
0.486657678894505 0.6817739994414464
0.755511188588319
0.0
Initial position
0,2986602331992100
0.0
0.02255663774686400
(0.61264560125533)
0.0
16667996664666166
0.363013287999000
0.0
Initial velocity
0.5790749225540872
0.748157481446087
0.0
0.761784787007641
0.385144193447218
0.0.
Table 1: Train and validation loss for each NN architecture for the linear spring task for MCLNN. Note that for hyperparamter search (i.e. number of hidden units), minimum train loss was set to 1.0e-8 as stopping criteria.
Hidden Layers
Linear Spring
2, 2
0.0012 0.0015
4, 4
7.98e-09, 2.36e-08
8, 8
9.86e-09, 3.80e-08
16, 16
9.76e-09, 3,02e-08
15

---

Under review as a conference paper at ICLR 2022
dt 0.01
mass = 1.0
stride 10
runs = 20 = points per trajectory
I 1.0c 33 3
layers = [10, 10]
epochs 100000
samples = 100 number of trajectories
seed 100
time step dl x stride = 0.1
A.6.2
NON-LINEAR SPRING SYSTEM
0.486657678894505
0.755011888563519
0.0
Initial position
0.6817379994414444
0.288660233197210
0.0
0.0225566337466660
-0.612645601253588
0.0
0.182709864466616
0.363013287999004
0.07
Initial velocity
0.579074922540872
-07741557881466087
0.0
0.761784787007641
0.385144193447218
0.0,
dt = 0.01
mass = 1.0
stride = 10
runs 20 points per trajectory
Ir 1.0c 3
layers = [10, 10]
epochs 100000
samples = 100 = number of trajectories
seed 100
time step dl x stride = 0.1
A.6.3
GRAVITATIONAL SYSTEM
1.0
0.0
0.07
9.0
0.0
0.0
Initial position
11.0
0.0
0.0
-1.0
0.0
0.0
TO.0
0.05
0.07
0.0
-0.55
0.0
Initial velocity
0.0
0.65
0.0
10.0
0065
0.0
dt = 0.01
mass 1.0
stride = 10
runs 20 points per trajectory
Ir 1.0c 3
layers = [10, 10]
epochs 100000
samples = 100 = number of trajectories
seed 100
time step = dt x stride = 0.1
16

---

Under review as a conference paper at ICLR 2022
A.7
DEFAULT NOTATION
A scalar (integer real)
vector
A matrix
I
Identity matrix with rows and n columns
Identity matrix with dimensionality implied by context
e
Standard basis vector [O.... 0, 1....... O] with I at po-
sition i
Vector corresponding to particle
Element j of matrix A
Ag
Row i of matrix A
A
Column of matrix A
A.j.k
Element (i.j, k) of 3-D tensor A
2-D slice of 3-D tensor
dy
Derivative of y with respect
da
By
Partial derivative of with respect to
dx
VZL
Gradient of L with respect to 7
Xxy
Matrix derivatives of y with respect X
0f
Jacobian matrix J E m of f R= R
dæ
f f(x)dx
Definite integral over the entire domain of Definite integral with respect to over the set S
[ f(x))xx
J8
P(a)
A probability distribution over discrete variable
p(a)
A probability distribution over a continuous variable, or over variable whose type has not been specified
17


Reviews:
[{"review_id": "hshLPSQ8i0c", "reviewer": "Reviewer_1UZJ", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Major concerns\n\n- It seems to me that the kinetic energy is not learned. Instead the ground truth is used during training. This model design is not in the spirit of HNN and LNN line of works, where masses are assumed unknown and need to be learned. If only potential energy is learned, I think a name like \"potential energy net\" is more appropriate than MCLNN. What if the authors assume that masses are not known and instead need to be learned? You can assume the form of kinetic energy (eqn. 5) is known. Then the question is under this setting, how does the model perform in terms of momentum conservation? You can even comment on the learned and true masses. I think this is more reasonable setting if you want to enforce momentum conservation into LNN.\n- The description of the generative model to \"generalize to unseen system sizes\" is missing. Since in all the systems, the pairwise interactions is homogeneous, I guess the neural network is learning the pairwise potential energy. In other words, since the spring constants and the masses (in the gravitational experiments) are the same (for each pair). The neural network can learn the pairwise potential energy and they are summed up to get the total potential energy. In this way, the learned model can be easily generalized to unseen system sizes. If this is true, this kind of generalization cannot be done with heterogeneous pairwise interactions, e.g., different spring constants in a system. This is a big limitation on the claim of \"generalization to unseen system sizes\" and should be pointed out clearly in the paper. \n\nMinor concerns\n- Why did you choose square plus to be the activation function? Did you encounter any problem using softplus?\n- In section 4.2, the authors says \n>Any value of Lagrangian can still yield a constant Hamiltonian (energy),provided the constraints as imposed by the EL equations on the Lagrangian are satisfied. As such, LNNs should be evaluated considering multiple metrics as demonstrated here\n\nI do not agree with this statement. If we compare an MLP, a LNN and a MCLNN. We know that an MLP has no physics incorporated, so none of the conservation quantities in the prediction trajectory should be expected. A LNN enforces energy conservation, so energy conservation is expected. A MCLNN enforces both energy conservation and momentum conservation, so these convervations are expected in the prediction trajectory. LNN and MCLNN are different models that target different problems. It is not fair to expect LNN to conserve momentum simply because it is not enforced. In other words, consider a hypothetical case: another work that extends MCLNN in the future and says \"MCLNN should be evaluated on metrics other than energy and momentum conservations and you see, MCLNN perform badly on these additional metrics.\" This won't be fair to MCLNN as well. MCLNN conserves energy and momentum simply because it is designed to do so. We should not expect MCLNN to perform well on another, if any, metric that it is not enforced to do. \n- Typo\n    - section 4.1.5 asses -> assess\n    - Figure 4 amd Figure 7 caption (a), (b), (b) -> (a), (b), (c)", "summary_of_the_paper": "This paper is motivated by LNN and aim to enforce momentum conservation in learning based on the Lagrangian framework. The goal is straightforward and simple, but I have not seen similar work in the literature. The kinetic energy is assumed to be known and pairwise potential energy is learned through a neural network. Since the distance between each pair of particles is constructed as the input to the potential energy network, the learned model enforces energy as well as momentum conservation. The proposed model is tested on three Interacting particle systems with homogeneous pairwise interaction. ", "summary_of_the_review": "The paper is well written and the idea is simple and straightforward. The novelty is limited. My main concern is about the design of the known kinetic energy (mass) and the claim of generalizability to unseen system sizes. The known kinetic energy design does not make sense. The generalizability to unseen system sizes is only valid for homogeneous pairwise potential energy. The whole framework seems to only work for a specific system - interacting particles. This limited applicability also weakens the contribution. I think this submission does not meet the ICLR standard, thus I recommend rejection. \n\nTo meet the standard of major AI conferences, the paper should at least learn masses simultaneously and explore more application scenarios other than interacting particles. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "erbTyVwI0wT", "reviewer": "Reviewer_N3Bq", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "## Pros:\n- The empirical evaluation demonstrates a real gain of taking into account the conservation of additional quantities. It is indeed a good inductive bias to put in the model when we can do it.\n- The simplicity of the idea is a plus and it is clearly introduced.\n\n## Cons\n- Although I found the manuscript overall easy to follow and clear, I think the presentation could be improved. As specific examples, the many bullet points in the introduction breaks the flow. Same with \"Problem 1\", actually I did not get what was the purpose and formulating the learning problem like that as it is not really in the rest of the manuscript except for the loss. I find the usage of the word \"generative model\" a bit confusing in the context of a physical model (although I agree those are valid generative models).\n-  Lemma 1 and 2 are quite straightforward and so their proofs could go in the appendix I think.\n- I am surprised nobody tried to use GNNs as Lagrangian neural networks as this seems quite natural for multi bodied systems. Actually, you also mentioned this in your conclusion and I wonder how the current paper is too incremental.\n- As said in the previous point I think the contribution is very incremental.\n- Could you clarify equation 6 and how you backprop through p(T q)?\n- It is not clear if (4) is expressive enough for all kinds of systems, I suppose some could require higher-level interaction scheme for evaluating the potential energy?!\n- \n## Additional remarks\n- 3.1 \"then trajectory\" missing determinant.\n- just before (5) missing \"be\"\n- The L when substituted... eq 2 => should be (3) right?\n- page 7 up on -> upon \n", "summary_of_the_paper": "This paper enhances Lagrangian neural networks by adding conservation of the angular and linear momenta. It does so by enforcing symmetry with respect to translation and rotation of the system in the Lagrangian. Experiments performed on a set of physical systems shows angular and linear momenta conservation reduce the divergence between the (implicitly) learned and exact dynamic of those systems. ", "summary_of_the_review": "Incremental but sound idea. Presentation could be improved.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "bI4w9_j89DB", "reviewer": "Reviewer_Qsuh", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Pros**\n\nThe paper is well-organized and self-explanatory. The theoretical backgrounds such as Noether's theorem and the momentum conservation law are introduced in detail and in a reader-friendly way.\n\nThe performance improvement is valid by reliable experiments and supported by theories.\n\n**Cons**\n\nMy main concern is about novelty and significance.\n\nFor multi-body problems, the potential energy is the center of attention, while the kinetic energy is easily modeled. Among the community of molecular dynamics, an empirical approximation of the potential energy has a long history. One of the most famous ones is the Lennard-Jones potential [1]. Many studies have attempted to approximate the potential function by using machine learning methods, and many recent studies have employed neural networks [2][3][4]. The improvement from these studies is unclear.\n\n- [1] J. E. Jones (1924). \"On the determination of molecular fields.—I. From the variation of the viscosity of a gas with temperature\". Proceedings of the Royal Society of London. Series A, Containing Papers of a Mathematical and Physical Character.\n- [2] E. D. Cubuk, B. D. Malone, B. Onat, A. Waterland, and E. Kaxiras (2017) \"Representations in neural network based empirical potentials\". Journal of Chemical Physics.\n- [3] M. Wen and E. B. Tadmor. (2019) \"Hybrid neural network potential for multilayer graphene\". Physical Review B.\n- [4] J. Han, L. Zhang, R. Car, and W. E (2020). \"Deep Potential: A General Representation of a Many-Body Potential Energy Surface\". Communications in Computational Physics.\n\nIn my opinion, if a theory well-known in another community contributes greatly to the neural network community, an introduction of the theory should be valued highly. However, the contribution of the present study is directed to the computational science community rather than the neural network community. Hence, the evaluation from the viewpoint of the computational science is important. In this sense, the present study does not demonstrate a sufficient novelty or significance.\n\n**Other Comment**\n\nThe proposed method models the potential function as the sum of pair-wise potential functions. The proposed method is a special case of DeLaN [5], which approximates the whole potential function using a neural network. DeLaN is also a special case of LNN [6], which approximates the Lagrangian. Related works should be introduced in more detail.\n\nThe proposed method is useful but sometimes insufficient. The potential function of a planatry system (a.k.a. gravitational system) is one of the targets of the proposed method, but such system is too simple to be modeled by neural networks. The potential function of a molecular dynamics is led by the nonlinear interaction among more than two particles described by the Schrödinger equation. The proposed method is applicable only to the cases where the nonlinearity is negligible.\n\n- [5] M. Lutter, C. Ritter, and J. Peters (2019). \"Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning\". ICLR.\n- [6] M. Cranmer, S. Greydanus, S. Hoyer, P. Battaglia, D. Spergel, and S. Ho. (2020) \"Lagrangian neural networks\". ICLR 2020 Workshop on Integration of Deep Neural Models and Differential Equations.\n", "summary_of_the_paper": "Many recent studies have attempted to learn Lagrangian and Hamiltonian systems by using neural networks, but most of them still suffer from instability to learn the translational and rotational symmetries. These symmetries are associated with the conservation laws of linear momentum and angular momentum, respectively. In contrast to Lagrangian neural network, which learns a Lagrangian as a black-box, the proposed method learns the Lagrangian as the kinetic energy predefined by (5) minus the sum of pair-wise potential energy $V_{ij}$ modeled by a neural network. Thanks to this explicit formulation, the proposed method conserves not only the system energy but also linear/angular momentum, as confirmed by the experiments.", "summary_of_the_review": "The manuscript is well-organized, and the experiments are reliable. The main concern is the novelty. Many similar approaches have been investigated in the community of molecular dynamics, and the improvement from them is unclear.\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "FzG7xmCbSTC", "reviewer": "Reviewer_Vgjt", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The proposed modification of adding in the kinetic energy and only learning a parametrized potential is principled, well explained and one can see why it should work better, both in terms of modelling accuracy and physical soundness.\n\nHowever, I find the scope of the paper to be too narrow. It is only a marginal transformation of an existing model, the theoretical contributions are quite elementary and the experiments do not really suggest wider applicability of the model in practice. I think this work is more suited to be a (very good) workshop submission.", "summary_of_the_paper": "By constraining the form of the lagrangian and predicting only the value of the potential of a system of interacting particles, this paper proposes an improved version of the Lagrangian Neural Networks model which is shown to conserve the momentum of the system.", "summary_of_the_review": "While technically sound, I don't advocate for this work to be accepted at ICLR as the contribution is too marginal.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper enhances Lagrangian neural networks by adding conservation of the angular and linear momenta. According to the reviewers, the technical contribution of the paper is marginal, it is a incremental change of an existing model, and it seems that there is some over claim on the generalization of the model to unseen systems. The theoretical contributions in the paper are not significant, and the experiments have not demonstrate the practical potential of the proposed model yet. After the reviewers provided their comments, the authors did not submit their rebuttals. Therefore, as a result, we do not think the paper is ready for publication at ICLR.
Decision: Reject