Original: Symplectic Recurrent Neural Networks
Revised: Symplectic Recurrent Neural Networks

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: Abstract
Revised: Abstract

Original: We propose Symplectic Recurrent Neural Networks (SRNNs) as learning algorithms that capture the dynamics of physical systems from observed trajectories.
Revised: We propose Symplectic Recurrent Neural Networks (SRNNs) as learning algorithms that capture the dynamics of physical systems from observed trajectories.

Original: An SRNN models the Hamiltonian function of the system by a neural network and furthermore leverages symplectic integration, multiplestep training and initial state optimization to address the challenging numerical issues associated with Hamiltonian systems.
Revised: An SRNN models the Hamiltonian function of the system by a neural network and furthermore leverages symplectic integration, multiplestep training and initial state optimization to address the challenging numerical issues associated with Hamiltonian systems.

Original: We show SRNNs succeed reliably on complex and noisy Hamiltonian systems.
Revised: We show SRNNs succeed reliably on complex and noisy Hamiltonian systems.

Original: We also show how to augment the SRNN integration scheme in order to handle stiﬀ dynamical systems such as bouncing billiards.
Revised: We also show how to augment the SRNN integration scheme in order to handle stiﬀ dynamical systems such as bouncing billiards.

Original:  Introduction
Revised:  Introduction

Original: Can machines learn physical laws from data?
Revised: Can machines learn physical laws from data?

Original: A recent paper (Greydanus et al., 2019),
Revised: A recent paper (Greydanus et al., 2019),

Original: Hamiltonian Neural Networks (HNN), proposes to do so representing the Hamiltonian function H ( q, p ) as a multilayer neural network.
Revised: Hamiltonian Neural Networks (HNN), proposes to do so representing the Hamiltonian function H ( q, p ) as a multilayer neural network.

Original: The partial derivatives of this network are then trained to match the time derivatives ˙
Revised: The partial derivatives of this network are then trained to match the time derivatives ˙

Original: p and ˙ q observed along the trajectories in state space.
Revised: p and ˙ q observed along the trajectories in state space.

Original: The ordinary diﬀerential equations (ODEs) that express Hamiltonian dynamics are famous for both their mathematical elegance and their challenges to numerical integration techniques.
Revised: The ordinary diﬀerential equations (ODEs) that express Hamiltonian dynamics are famous for both their mathematical elegance and their challenges to numerical integration techniques.

Original: Except maybe for the simplest Hamiltonian systems, discretization errors and measurement noise lead to quickly diverging trajectories.
Revised: Except maybe for the simplest Hamiltonian systems, discretization errors and measurement noise lead to quickly diverging trajectories.

Original: In other words, Hamiltonian systems can often be stiﬀ , a concept that usually refers to diﬀerential equations where we have to take very small time-steps of integration so that the numerical solution remain stable (Lambert, 1991).
Revised: In other words, Hamiltonian systems can often be stiﬀ , a concept that usually refers to diﬀerential equations where we have to take very small time-steps of integration so that the numerical solution remain stable (Lambert, 1991).

Original: A plethora of numerical integration methods, symplectic integrators , have been developed to respect the conserved quantities in Hamiltonian systems, thereby usually being more stable and structure-preserving than non-symplectic ones (Hairer et al., 2002).
Revised: A plethora of numerical integration methods, symplectic integrators , have been developed to respect the conserved quantities in Hamiltonian systems, thereby usually being more stable and structure-preserving than non-symplectic ones (Hairer et al., 2002).

Original: For example, the simplest symplectic integrator is the well-known leapfrog method, also known as the Stömer-Verlet integrator (Leimkuhler and Reich, 2005).
Revised: For example, the simplest symplectic integrator is the well-known leapfrog method, also known as the Stömer-Verlet integrator (Leimkuhler and Reich, 2005).

Original: However, even the best integrators remain severely challenged by phenomena as intuitive as a mechanical rebound or a slingshot eﬀect, which are more severe forms of stiﬀness.
Revised: However, even the best integrators remain severely challenged by phenomena as intuitive as a mechanical rebound or a slingshot eﬀect, which are more severe forms of stiﬀness.

Original: Such numerical issues are almost doomed to conﬂict with the inherently approximate nature of a learning algorithm.
Revised: Such numerical issues are almost doomed to conﬂict with the inherently approximate nature of a learning algorithm.

Original: In the ﬁrst part of this paper, we propose Symplectic Recurrent Neural Networks (SRNNs), where ( i ) the partial derivatives of the neural-network-parametrized Hamiltonian are integrated with the leapfrog integrator and where ( ii ) the loss is back-propagated through the ODE integration over multiple time steps.
Revised: In the ﬁrst part of this paper, we propose Symplectic Recurrent Neural Networks (SRNNs), where ( i ) the partial derivatives of the neural-network-parametrized Hamiltonian are integrated with the leapfrog integrator and where ( ii ) the loss is back-propagated through the ODE integration over multiple time steps.

Original: We ﬁnd that in the presence of observation noise, SRNN are far more usable than HNNs.
Revised: We ﬁnd that in the presence of observation noise, SRNN are far more usable than HNNs.

Original: Further improvements are achieved by simultaneously optimizing the initial state and the Hamiltionian network, presenting an interesting contrast to previous literature on the hardness of general initial state optimization (Peifer and Timmer, 2007).
Revised: Further improvements are achieved by simultaneously optimizing the initial state and the Hamiltionian network, presenting an interesting contrast to previous literature on the hardness of general initial state optimization (Peifer and Timmer, 2007).

Original: The optimization can be motivated from a maximum likelihood estimation perspective, and we provide heuristic arguments for why the initial state optimization is likely convex given the symplecticness of the system.
Revised: The optimization can be motivated from a maximum likelihood estimation perspective, and we provide heuristic arguments for why the initial state optimization is likely convex given the symplecticness of the system.

Original: Furthermore, experiments in the three-body problem show that the SRNN-trained Hamiltonian compensates for discretization errors and can even outperform numerically solving the ODE using the true Hamiltonian and the same time-step size.
Revised: Furthermore, experiments in the three-body problem show that the SRNN-trained Hamiltonian compensates for discretization errors and can even outperform numerically solving the ODE using the true Hamiltonian and the same time-step size.

Original: This could be of particular interest to people applying machine learning to numerically solve diﬀerential equations.
Revised: This could be of particular interest to people applying machine learning to numerically solve diﬀerential equations.

Original: The second part of this paper focuses on perfect rebound as an example of the more severe form of stiﬀness.
Revised: The second part of this paper focuses on perfect rebound as an example of the more severe form of stiﬀness.

Original: When a point mass rebounds without loss of energy on a perfectly rigid obstacle, the motion of the point mass is changed in ways that can be interpreted as an inﬁnite force applied during an inﬁnitesimal time.
Revised: When a point mass rebounds without loss of energy on a perfectly rigid obstacle, the motion of the point mass is changed in ways that can be interpreted as an inﬁnite force applied during an inﬁnitesimal time.

Original: The precise timing of this event aﬀects the trajectory of the point mass in ways that essentially make it impossible to merely simulate the Hamiltonian system on a predeﬁned grid of time points.
Revised: The precise timing of this event aﬀects the trajectory of the point mass in ways that essentially make it impossible to merely simulate the Hamiltonian system on a predeﬁned grid of time points.

Original: In order to address such events in learning, we augment the leapfrog integrator used in our SRNN with an additional trainable operator that models the rebound events and relates their occurrence to visual hints.
Revised: In order to address such events in learning, we augment the leapfrog integrator used in our SRNN with an additional trainable operator that models the rebound events and relates their occurrence to visual hints.

Original: Training such an augmented SRNN on observed trajectories not only learns the point mass dynamics but also learns the visual appearance of the obstacles.
Revised: Training such an augmented SRNN on observed trajectories not only learns the point mass dynamics but also learns the visual appearance of the obstacles.

Original:  Related work
Revised:  Related work

Original: Learning physics with neural networks A popular category of methods attempts to replicate the intuitive ways in which humans perceive simple physical interactions, identifying objects and learning how they relate to each other (Battaglia et al., 2016; Chang et al., 2016).
Revised: Learning physics with neural networks A popular category of methods attempts to replicate the intuitive ways in which humans perceive simple physical interactions, identifying objects and learning how they relate to each other (Battaglia et al., 2016; Chang et al., 2016).

Original: Since such methods cannot be used for more general physical systems, another category of methods seeks to learn which diﬀerential equations govern the evolution of a physical system on the basis of observed trajectories.
Revised: Since such methods cannot be used for more general physical systems, another category of methods seeks to learn which diﬀerential equations govern the evolution of a physical system on the basis of observed trajectories.

Original: Brunton et al.
Revised: Brunton et al.

Original: (2016) assemble a small number of predeﬁned primitives in order to ﬁnd an algebraically simple solution.
Revised: (2016) assemble a small number of predeﬁned primitives in order to ﬁnd an algebraically simple solution.

Original: Lutter et al.
Revised: Lutter et al.

Original: (2019) use a neural network to model the Lagrangian function of a robotic system.
Revised: (2019) use a neural network to model the Lagrangian function of a robotic system.

Original: Most closely related to ours, Greydanus et al.
Revised: Most closely related to ours, Greydanus et al.

Original: (2019) use a neural network to learn the Hamiltonian of the dynamical system in such a way that its partial derivatives match the time derivatives of the position and momentum variables, which are both assumed to be observed.
Revised: (2019) use a neural network to learn the Hamiltonian of the dynamical system in such a way that its partial derivatives match the time derivatives of the position and momentum variables, which are both assumed to be observed.

Original: Although the authors show success on a simple pendulum system, this approach does not perform well on a more complex system such as a three-body problem.
Revised: Although the authors show success on a simple pendulum system, this approach does not perform well on a more complex system such as a three-body problem.

Original: ODE-based learning and recurrent neural networks  To learn a neural-networkparametrized ODE that governs time series data, Chen et al.
Revised: ODE-based learning and recurrent neural networks (RNNs) To learn an ODE that underlies some observed time series data, Chen et al.

Original: (2018a) proposes to  minimize the distance betweenmultiple steps of an observed time series and the corresponding steps of the  time series generated by an ODE solver starting from the same initial values.
Revised: (2018a) proposes to solve a neuralnetwork-parameterized ODE numerically and minimize the distance between the generated time series with the observed data.

Original: In order to save memory, they propose to use the adjoint ODE instead of back-propagating through thenumerical steps of the ODE solver. 
Revised: To save memory, they propose to use the adjoint ODE instead of back-propagating through the ODE solver. Using stability analysis of

Original: Drawing an analogy between recurrent neural networks (RNNs) and ordinary diﬀerential equations (ODEs), Chang et al.
Revised: ODEs, Chang et al.

Original: (2019) propose the AntisymmetricRNN with better stability properties.
Revised: (2019) propose the AntisymmetricRNN with better trainability.

Original: Inspired by quantum mechanics considerations, Niu et al. (2019) propose a recurrent neural network architecture with both stability and universality properties.
Revised: Niu et al. (2019) establish a correspondence between RNNs and ODEs, and propose an RNN architecture inspired by a universal quantum computation scheme.

Original: Summary of our main contributions: In this paper, we propose SRNN, which
Revised: Summary of our main contributions: In this paper, we propose SRNN, which

Original: • learns Hamiltonian dynamics directly from position and momentum time series • performs well on noisy and complex systems such as a spring-chain system and a three-body system, and is compatible with initial state optimzation • is augmented to handle perfect rebound, an example of very stiﬀ Hamiltonian dynamics
Revised: • learns Hamiltonian dynamics directly from position and momentum time series • performs well on noisy and complex systems such as a spring-chain system and a three-body system, and is compatible with initial state optimzation • is augmented to handle perfect rebound, an example of very stiﬀ Hamiltonian dynamics

Original:  Framework 3. Hamiltonian systems
Revised:  Framework 3. Hamiltonian systems

Original: A Hamiltonian system of dimension d is described by two vectors p, q ∈ R d .
Revised: A Hamiltonian system of dimension d is described by two vectors p, q ∈ R d .

Original: Typically, they correspond to the momentum and position variables, respectively.
Revised: Typically, they correspond to the momentum and position variables, respectively.

Original: The evolution of the system is determined by the Hamiltonian function H : ( p, q, t ) ∈ R 2 d +

Revised: The evolution of the system is determined by the Hamiltonian function H : ( p, q, t ) ∈ R 2 d +

Original: (cid:55)→ H ( p, q, t ) ∈
Revised: (cid:55)→ H ( p, q, t ) ∈

Original: R through a system of ordinary diﬀerential equations called Hamilton’s equations ,
Revised: R through a system of ordinary diﬀerential equations called Hamilton’s equations ,

Original: [Equation]
Revised: [Equation]

Original: where we use the dot notation to compactly represent derivatives with respect to the time variable t .
Revised: where we use the dot notation to compactly represent derivatives with respect to the time variable t .

Original: We are focusing in this work on Hamiltonians that are conservative , 1 that is, they do not depend on the time variable t , and separable , 2 that is, they can be written as a sum H ( p, q ) =
Revised: We are focusing in this work on Hamiltonians that are conservative , 1 that is, they do not depend on the time variable t , and separable , 2 that is, they can be written as a sum H ( p, q ) =

Original: K ( p ) + V ( q ) .
Revised: K ( p ) + V ( q ) .

Original: In this case, (1) becomes
Revised: In this case, (1) becomes

Original: [Equation]
Revised: [Equation]

Original: With a proper choice of the p and q variables, the evolution of essentially all physical systems can be described with the Hamiltonian framework.
Revised: With a proper choice of the p and q variables, the evolution of essentially all physical systems can be described with the Hamiltonian framework.

Original: In other words, Hamilton’s equations restrict the vast space of dynamical systems to the considerably smaller space of dynamical systems that are physically plausible.
Revised: In other words, Hamilton’s equations restrict the vast space of dynamical systems to the considerably smaller space of dynamical systems that are physically plausible.

Original: Therefore, instead of modeling the dynamics of a physical system with a neural network f θ ( p, q ) whose outputs are interpreted as estimates of the time derivatives ˙ p and ˙ q , we can also use a neural network H θ ( p, q ) =
Revised: Therefore, instead of modeling the dynamics of a physical system with a neural network f θ ( p, q ) whose outputs are interpreted as estimates of the time derivatives ˙ p and ˙ q , we can also use a neural network H θ ( p, q ) =

Original: K θ 1 ( p ) + V θ 2 ( q ) with θ =
Revised: K θ 1 ( p ) + V θ 2 ( q ) with θ =

Original: [ θ 1 , θ 2 ] , whose partial derivatives − V (cid:48) θ 2 ( q ) and K (cid:48) θ 1 ( p ) are interpreted as the time derivatives ˙ p and ˙ q .
Revised: [ θ 1 , θ 2 ] , whose partial derivatives − V (cid:48) θ 2 ( q ) and K (cid:48) θ 1 ( p ) are interpreted as the time derivatives ˙ p and ˙ q .

Original: We refer to the former as ODE neural networks (O-NET) and the latter approach as Hamiltonian neural networks (H-NET).
Revised: We refer to the former as ODE neural networks (O-NET) and the latter approach as Hamiltonian neural networks (H-NET).

Original: In order to deﬁne a complete learning system, we need to explain how to determine the parameter θ of the neural networks on the basis of observed discrete trajectories.
Revised: In order to deﬁne a complete learning system, we need to explain how to determine the parameter θ of the neural networks on the basis of observed discrete trajectories.

Original: For instance, Greydanus et al.
Revised: For instance, Greydanus et al.

Original: (2019) trains H-NET in a fully supervised manner using the observed tuples ( p, q, ˙ p, ˙ q ) .
Revised: (2019) trains H-NET in a fully supervised manner using the observed tuples ( p, q, ˙ p, ˙ q ) .

Original: 3. From ODEs to discrete trajectories
Revised: 3. From ODEs to discrete trajectories

Original: A numerical integrator (or ODE solver) approximates the true solution of an ODE of the form ˙
Revised: A numerical integrator (or ODE solver) approximates the true solution of an ODE of the form ˙

Original: z = f ( z, t ) at discrete time steps t 0 , t 1 . .
Revised: z = f ( z, t ) at discrete time steps t 0 , t 1 . .

Original: .t
Revised: .t

Original: T .
Revised: T .

Original: For instance, the simplest integrator,
Revised: For instance, the simplest integrator,

Original: Euler’s integrator, starts from the initial state z 0 at time t 0 and estimates the function z ( t ) at uniformly spaced time points t n = t 0 + n ∆ t with the recursive expression
Revised: Euler’s integrator, starts from the initial state z 0 at time t 0 and estimates the function z ( t ) at uniformly spaced time points t n = t 0 + n ∆ t with the recursive expression

Original: [Equation]
Revised: [Equation]

Original: In stiﬀ ODE systems, however, using Euler’s method could easily lead to unstable solutions unless the time-step is chosen to be very small (Lambert, 1991).
Revised: In stiﬀ ODE systems, however, using Euler’s method could easily lead to unstable solutions unless the time-step is chosen to be very small (Lambert, 1991).

Original: The development of eﬃcient and accurate numerical integrators is the object of considerable research (Hairer et al., 2008; Hairer and Wanner, 2013).
Revised: The development of eﬃcient and accurate numerical integrators is the object of considerable research (Hairer et al., 2008; Hairer and Wanner, 2013).

Original: Symplectic integrators 3 are particularly attractive for the integration of Hamilton’s equations (Leimkuhler and Reich, 2005).
Revised: Symplectic integrators 3 are particularly attractive for the integration of Hamilton’s equations (Leimkuhler and Reich, 2005).

Original: They are able to preserve quadratic invariants, and therefore usually have desired stability properties as well as being structure-preserving (McLachlan et al., 2004), even for certain non-Hamiltonian systems (Chen et al., 2018b).
Revised: They are able to preserve quadratic invariants, and therefore usually have desired stability properties as well as being structure-preserving (McLachlan et al., 2004), even for certain non-Hamiltonian systems (Chen et al., 2018b).

Original: A simple and widely-used symplectic integrator is the leapfrog integrator.
Revised: A simple and widely-used symplectic integrator is the leapfrog integrator.

Original: When the Hamiltonian is conservative and separable (2), it computes successive estimates ( p n , q n ) with
Revised: When the Hamiltonian is conservative and separable (2), it computes successive estimates ( p n , q n ) with

Original: [Equation]
Revised: [Equation]

Original: Repeatedly executing update equations (4) is called the leapfrog algorithm , which is as computationally eﬃcient as Euler’s method yet considerably more accurate when the ODE belongs to a Hamiltonian system (Leimkuhler and Reich, 2005).
Revised: Repeatedly executing update equations (4) is called the leapfrog algorithm , which is as computationally eﬃcient as Euler’s method yet considerably more accurate when the ODE belongs to a Hamiltonian system (Leimkuhler and Reich, 2005).

Original: 3. Learning ODEs from discrete trajectories
Revised: 3. Learning ODEs from discrete trajectories

Original: Following Chen et al.
Revised: Following Chen et al.

Original: (2018a), let the right hand side of the ODE be a parametric function f θ ( z, t ) and let z 0 . . .
Revised: (2018a), let the right hand side of the ODE be a parametric function f θ ( z, t ) and let z 0 . . .

Original: z T be an observed trajectory measured at uniformly spaced time points t 0 . . .
Revised: z T be an observed trajectory measured at uniformly spaced time points t 0 . . .

Original: t T .
Revised: t T .

Original: We can estimate the parameter θ that best represents the dynamics of the observed trajectory by minimizing the mean squared error (cid:80)
Revised: We can estimate the parameter θ that best represents the dynamics of the observed trajectory by minimizing the mean squared error (cid:80)

Original: Ti =1 (cid:107) z i − ˆ
Revised: Ti =1 (cid:107) z i − ˆ

Original: z i ( θ ) (cid:107) 2 between the observed trajectory { z i } Ti =0 and the trajectory { ˆ z i ( θ ) } Ti =0 generated with our integrator of choice, { ˆ z i ( θ ) } Ti =
= Integrator ( z
Revised: z i ( θ ) (cid:107) 2 between the observed trajectory { z i } Ti =0 and the trajectory { ˆ z i ( θ ) } Ti =0 generated with our integrator of choice, { ˆ z i ( θ ) } Ti =

Original: [Equation] f θ [Equation] { t i } Ti =0 ) .
Revised: = Integrator ( z 0 , f θ , { t i } Ti =0 ) .

Original: For instance, this minimization can be achieved using stochastic gradient descent after back-propagating through the steps of our numerical integration algorithm of choice and then through each call to the functions f θ .
Revised: For instance, this minimization can be achieved using stochastic gradient descent after back-propagating through the steps of our numerical integration algorithm of choice and then through each call to the functions f θ .

Original: This can be done when f θ ( z ) is a neural network (O-NET), or is the concatenation [ − V (cid:48) θ 2 ( q ) , K (cid:48)
Revised: This can be done when f θ ( z ) is a neural network (O-NET), or is the concatenation [ − V (cid:48) θ 2 ( q ) , K (cid:48)

Original: θ 1 ( p )] of the partial derivatives of an H-NET
Revised: θ 1 ( p )] of the partial derivatives of an H-NET

Original: H θ ( p, q ) =
Revised: H θ ( p, q ) =

Original: K θ 1 ( p ) + V θ 2 ( q ) , where the partial derivatives can be expressed using the same parameters θ as the Hamiltonian H θ ( p, q ) , for instance using automatic diﬀerentiation.
Revised: K θ 1 ( p ) + V θ 2 ( q ) , where the partial derivatives can be expressed using the same parameters θ as the Hamiltonian H θ ( p, q ) , for instance using automatic diﬀerentiation.

Original: We can then predict trajectories at testing time using the trained f θ ∗ and initial state z test0 , 
Revised: We can then predict trajectories at testing time using the trained f θ ∗ and initial state z test 0 , { ˆ z test i [Equation] test i [Equation]

Revised: =

Revised: Integrator ( z test0 , f θ

Original: [Equation]
Revised: [Equation]

Original:  Note that neither the integrator, nor the number of steps, nor the step size, need to be the same at training and testing.
Revised: , { t [Equation] test i =0 ) . Note that neither the integrator, nor the number of steps, nor the step size, need to be the same at training and testing.

Original: 3. Symplectic Recurrent Neural Network
Revised: 3. Symplectic Recurrent Neural Network

Original: This framework provides a number of nearly orthogonal design options for the construction of algorithms that model dynamical systems using trajectories:
Revised: This framework provides a number of nearly orthogonal design options for the construction of algorithms that model dynamical systems using trajectories:

Original: • The time derivative model could be an O-NET or H-NET.
Revised: • The time derivative model could be an O-NET or H-NET.

Original: • The training integrator can be any explicit integrators.
Revised: • The training integrator can be any explicit integrators.

Original: In our experiments, we only focus on Euler’s integrator and the leapfrog integrator.
Revised: In our experiments, we only focus on Euler’s integrator and the leapfrog integrator.

Original: • The training trajectories can consist of a single step, T =1 , or multiple steps, T> 1 .
Revised: • The training trajectories can consist of a single step, T =1 , or multiple steps, T> 1 .

Original: We refer to the ﬁrst case as single-step and the second case as multi-step or recurrent training, because back-propagating through multiple steps of the training integrator is comparable to back-propagating through time in recurrent networks.
Revised: We refer to the ﬁrst case as single-step and the second case as multi-step or recurrent training, because back-propagating through multiple steps of the training integrator is comparable to back-propagating through time in recurrent networks.

Original: • The testing integrator can also be chosen freely and can use a diﬀerent time-step size as it does not involve back-propagation.
Revised: • The testing integrator can also be chosen freely and can use a diﬀerent time-step size as it does not involve back-propagation.

Original: In order to save space while describing the possibly diﬀerent integrators used for training and testing, we use the labels “E-E”, “E-L”, and “L-L”, where the ﬁrst letter tells which integrator was used for training —“E” for Euler and “L” for leapfrog— and the second letter indicates which integrator was used as testing time.
Revised: In order to save space while describing the possibly diﬀerent integrators used for training and testing, we use the labels “E-E”, “E-L”, and “L-L”, where the ﬁrst letter tells which integrator was used for training —“E” for Euler and “L” for leapfrog— and the second letter indicates which integrator was used as testing time.

Original: For instance, with our terminology, the
Revised: For instance, with our terminology, the

Original: HNN model of Greydanus et al.
Revised: HNN model of Greydanus et al.

Original: (2019) is a “single-step E-E H-NET” with the additional subtlety that they supervise the training with actual derivatives instead of relying on ﬁnite diﬀerences between successive steps of the observed trajectories.
Revised: (2019) is a “single-step E-E H-NET” with the additional subtlety that they supervise the training with actual derivatives instead of relying on ﬁnite diﬀerences between successive steps of the observed trajectories.

Original: A Symplectic Recurrent Neural Network (SRNN) is a recurrent H-NET that relies on a symplectic integrator for both training and testing, such as, for instance, a "recurrent L-L
Revised: A Symplectic Recurrent Neural Network (SRNN) is a recurrent H-NET that relies on a symplectic integrator for both training and testing, such as, for instance, a "recurrent L-L

Original: H-NET".
Revised: H-NET".

Original: As shown in the rest of this paper, SRNNs are far more usable and robust than the alternatives, especially when the Hamiltonian gets complex and the data gets noisy.
Revised: As shown in the rest of this paper, SRNNs are far more usable and robust than the alternatives, especially when the Hamiltonian gets complex and the data gets noisy.

Original: We believe that SRNNs may also have other potential beneﬁts: because leapfrog preserves volumes in the state space (Hairer et al., 2002), we conjecture that vanishing and exploding gradients’ issues in backpropagating through entire state sequences are ameliorated (Arjovsky et al., 2015).
Revised: We believe that SRNNs may also have other potential beneﬁts: because leapfrog preserves volumes in the state space (Hairer et al., 2002), we conjecture that vanishing and exploding gradients’ issues in backpropagating through entire state sequences are ameliorated (Arjovsky et al., 2015).

Original: Finally, because the leapfrog integrator is reversible in time, there is no need to store states during the forward pass as they can be recomputed exactly during the backward pass.
Revised: Finally, because the leapfrog integrator is reversible in time, there is no need to store states during the forward pass as they can be recomputed exactly during the backward pass.

Original: We leave studying these other computational and optimization beneﬁts as a topic of future work.
Revised: We leave studying these other computational and optimization beneﬁts as a topic of future work.

Original:  SRNN can learn complex and noisy Hamiltonian dynamics
Revised:  SRNN can learn complex and noisy Hamiltonian dynamics

Original: As an example of a complex Hamiltonian system, we ﬁrst present experiments performed on the spring-chain system: a chain of 20 masses with neighbors connected via springs.
Revised: As an example of a complex Hamiltonian system, we ﬁrst present experiments performed on the spring-chain system: a chain of 20 masses with neighbors connected via springs.

Original: Each of the two masses on the ends are connected to ﬁxed ground via another spring.
Revised: Each of the two masses on the ends are connected to ﬁxed ground via another spring.

Original: The chain can be assumed to lay horizontally and the masses move vertically but no gravity is assumed.
Revised: The chain can be assumed to lay horizontally and the masses move vertically but no gravity is assumed.

Original: The 20 masses and the 21 spring constants are chosen randomly and independently.
Revised: The 20 masses and the 21 spring constants are chosen randomly and independently.

Original: The training data consist of 1000 trajectories of the same chain, each of which starts from a random initial state of positions and momenta of the masses and is 10-time-step long (including the initial state).
Revised: The training data consist of 1000 trajectories of the same chain, each of which starts from a random initial state of positions and momenta of the masses and is 10-time-step long (including the initial state).

Original: We thus take T = 9 when performing recurrent training.
Revised: We thus take T = 9 when performing recurrent training.

Original: When performing single-step training, each training trajectory of length 10 is instead considered as 9 consecutive trajectories of length 2.
Revised: When performing single-step training, each training trajectory of length 10 is instead considered as 9 consecutive trajectories of length 2.

Original: In this way, 1000 sample trajectories of length 10 ( T =9) are turned into 9000 sample trajectories of length 2 ( T =1), allowing for a fair comparison between single-step training and recurrent training.
Revised: In this way, 1000 sample trajectories of length 10 ( T =9) are turned into 9000 sample trajectories of length 2 ( T =1), allowing for a fair comparison between single-step training and recurrent training.

Original: During testing, the trained model is given 32 random initial states in order to predict 32 trajectories of length 100.
Revised: During testing, the trained model is given 32 random initial states in order to predict 32 trajectories of length 100.

Original: Detailed experiment setups and model architectures are provided in Appendix A.1.
Revised: Detailed experiment setups and model architectures are provided in Appendix A.1.

Original: 4. Going symplectic - rescuing HNN with the leapfrog integrator
Revised: 4. Going symplectic - rescuing HNN with the leapfrog integrator

Original: First, we consider the noiseless case, where the training data consist of exact values of the positions ( q ) and momenta ( p ) of the masses on the chain at each discrete time point.
Revised: First, we consider the noiseless case, where the training data consist of exact values of the positions ( q ) and momenta ( p ) of the masses on the chain at each discrete time point.

Original: As shown in ﬁgure 1, the prediction of a single-step E-E H-NET deviates from the ground truth quickly and is unable to capture the periodic motion.
Revised: As shown in ﬁgure 1, the prediction of a single-step E-E H-NET deviates from the ground truth quickly and is unable to capture the periodic motion.

Original: By comparison, a single-step E-E ONET yields predictions that is qualitatively reasonable.
Revised: By comparison, a single-step E-E ONET yields predictions that is qualitatively reasonable.

Original: This shows that using Hamiltonian models without paying attention to the integration scheme may not be a good idea.[Figure] [Figure]
Revised: This shows that using Hamiltonian models without paying attention to the integration scheme may not be a good idea.

Original: We then replace Euler’s integrator used during testing by a leapfrog integrator, yielding a
Revised: We then replace Euler’s integrator used during testing by a leapfrog integrator, yielding a

Original: Single-step E-L H-NET.
Revised: Single-step E-L H-NET.

Original: Figure 1 shows that this helps the H-NET produce predictions that  remain stable and periodic over a longer period of time.
Revised: Figure 1 shows that this helps the H-NET produce predictions that [Figure] [Figure] remain stable and periodic over a longer period of time.

Original: Since the training process remains the same, this implies that part of the instability and degeneration of H-NET’s predictions comes from the nature of Euler’s integrator rather than the lack of proper training.
Revised: Since the training process remains the same, this implies that part of the instability and degeneration of H-NET’s predictions comes from the nature of Euler’s integrator rather than the lack of proper training.

Original: In contrast, using a leapfrog integrator for both training and testing substantially improve the performance, as also shown again in ﬁgure 1.
Revised: In contrast, using a leapfrog integrator for both training and testing substantially improve the performance, as also shown again in ﬁgure 1.

Original: This improvement shows the importance of consistency between the integrators used in training and predicting modes.
Revised: This improvement shows the importance of consistency between the integrators used in training and predicting modes.

Original: This can be understood with the concept of modiﬁed equations (Hairer, 1994): when we use a numerical integrator to solve an ODE, the numerical solution usually does not strictly follow the original equation due to discretization, but can be regarded as a solution to a modiﬁed version of the original equation that depends on the integrator and the time-step size.
Revised: This can be understood with the concept of modiﬁed equations (Hairer, 1994): when we use a numerical integrator to solve an ODE, the numerical solution usually does not strictly follow the original equation due to discretization, but can be regarded as a solution to a modiﬁed version of the original equation that depends on the integrator and the time-step size.

Original: Therefore, training and testing with the same numerical integrator and time-step size could allow the system to learn a modiﬁed Hamiltonian that corrects some of the errors caused by the discretization scheme.
Revised: Therefore, training and testing with the same numerical integrator and time-step size could allow the system to learn a modiﬁed Hamiltonian that corrects some of the errors caused by the discretization scheme.

Original: 4. Going recurrent - using multi-step training when noise is present
Revised: 4. Going recurrent - using multi-step training when noise is present

Original: Since noise is prevalent in real-world observations, we also test our models on noisy trajectories.
Revised: Since noise is prevalent in real-world observations, we also test our models on noisy trajectories.

Original: Independent and identically distributed Gaussian noise is added to both the position and the momentum variables at each time step.
Revised: Independent and identically distributed Gaussian noise is added to both the position and the momentum variables at each time step.

Original: Applying the single-step methods described above yield considerably worse predictions, as shown in ﬁgure 2 (left).
Revised: Applying the single-step methods described above yield considerably worse predictions, as shown in Figure 2 (left).

Original: This phenomenon can be controlled by training on multiple steps, eﬀectively arriving at a type of recurrent neural network: if noise is added independently at each time-step, then having data from multiple consecutive time steps may allow us to discern the actual noiseless trajectory, analogous to performing linear regression on multiple (more than 2) noisy data[Figure] points.
Revised: This phenomenon can be controlled by training on multiple steps, eﬀectively arriving at a type of recurrent neural network: if noise is added independently at each time-step, then having data from multiple consecutive time steps may allow us to discern the actual noiseless trajectory, analogous to performing linear regression on multiple (more than 2) noisy data points.

Original: As we see in ﬁgure 2 (left), recurrent training consistently improves the predictions except for E-E H-NET.
Revised: As we see in Figure 2 (left), recurrent training consistently improves the predictions except for E-E H-NET.

Original: The best performing model is the SRNN (recurrent L-L H-Net) which improves substantially over the single-step L-L H-NET.
Revised: The best performing model is the SRNN (recurrent L-L H-Net) which improves substantially over the single-step L-L H-NET.

Original: Interestingly, the recurrent
Revised: Interestingly, the recurrent

Revised: [Figure]

Original: E-E H-NET does not improve over the single-step E-E H-NET, which means that recurrent training does not help if one uses a naïve integrator.
Revised: E-E H-NET does not improve over the single-step E-E H-NET, which means that recurrent training does not help if one uses a naïve integrator.

Original: 4. Initial state optimization (ISO)
Revised: 4. Initial state optimization (ISO)

Original: However, one issue remains to be addressed: in the framework that we have adopted so far, the initial states p 0 and q 0 are treated as the actual initial states from which the system begins to evolve despite the added noise in observation.
Revised: However, one issue remains to be addressed: in the framework that we have adopted so far, the initial states p 0 and q 0 are treated as the actual initial states from which the system begins to evolve despite the added noise in observation.

Original: With noise added to the observation of p 0 and q 0 , our dynamical models will start from these noisy states and remain biased as we advance in time in both the training and the testing mode.
Revised: With noise added to the observation of p 0 and q 0 , our dynamical models will start from these noisy states and remain biased as we advance in time in both the training and the testing mode.

Original: To mitigate this issue, we propose to introduce two new parameter vectors for each sample, ˆ p 0 and ˆ q 0 , interpreted as our estimate of the actual initial states, and we let our dynamical models evolve starting from them instead of the observed p 0 and q 0 .
Revised: To mitigate this issue, we propose to introduce two new parameter vectors for each sample, ˆ p 0 and ˆ q 0 , interpreted as our estimate of the actual initial states, and we let our dynamical models evolve starting from them instead of the observed p 0 and q 0 .

Original: Treating ˆ p 0 and ˆ qas parameters, we can optimize them based on the loss function while ﬁxing the model’s parameters, a process that we call initial state optimization (ISO).
Revised: Treating ˆ p 0 and ˆ qas parameters, we can optimize them based on the loss function while ﬁxing the model’s parameters, a process that we call initial state optimization (ISO).

Original: When the model is good enough, we hope that this will guide us towards the true initial states without observation noise.
Revised: When the model is good enough, we hope that this will guide us towards the true initial states without observation noise.

Original: In Appendix B, we motivate the use of ISO from the perspective of maximum likelihood inference.
Revised: In Appendix B, we motivate the use of ISO from the perspective of maximum likelihood inference.

Original: In actual training, we ﬁrst train the neural network parameters for 100 epochs as usual, and starting from the 101st, after every epoch we perform ISO with the L-BFGS-B algorithm (Zhu et al., 1997) on the ˆ p 0 and ˆ q 0 parameters for every training trajectory.
Revised: In actual training, we ﬁrst train the neural network parameters for 100 epochs as usual, and starting from the 101st, after every epoch we perform ISO with the L-BFGS-B algorithm (Zhu et al., 1997) on the ˆ p 0 and ˆ q 0 parameters for every training trajectory.

Original: At testing time, the model is given the noisy values of p and q for the ﬁrsttime steps and must complete the trajectory for the next 200 steps.
Revised: At testing time, the model is given the noisy values of p and q for the ﬁrsttime steps and must complete the trajectory for the next 200 steps.

Original: These 10 initial time steps allow us to perform the same L-BFGS-B optimization to determine the initial state ˆ p 0 before advancing in time to predict the entire trajectory.
Revised: These 10 initial time steps allow us to perform the same L-BFGS-B optimization to determine the initial state ˆ p 0 before advancing in time to predict the entire trajectory.

Original: As seen in ﬁgure 2 (right), SRNN-ISO (i.e. SRNN equipped with ISO) clearly yields the best prediction among all the methods.
Revised: As seen in Figure 2 (right), SRNN-ISO (i.e. SRNN equipped with ISO) clearly yields the best prediction among all the methods.

Original: Figure 3 shows the predictions of HNN, SRNN and
Revised: Figure 3 shows the predictions of HNN, SRNN and

Original: SRNN-ISO on one test sample, and we clearly see the qualitative improvements thanks to recurrent training and ISO.
Revised: SRNN-ISO on one test sample, and we clearly see the qualitative improvements thanks to recurrent training and ISO.

Original: O-NET also beneﬁts from ISO while vanilla RNN and LSTM do not seem to, likely because the initial state optimization only works when we already have a reasonable model of the system.
Revised: O-NET also beneﬁts from ISO while vanilla RNN and LSTM do not seem to, likely because the initial state optimization only works when we already have a reasonable model of the system.

Original: In Appendix C, we give a heuristic argument for the convexity of ISO, which helps to explain the success of using L-BFGS-B for ISO, as will be demonstrated below.
Revised: In Appendix C, we give a heuristic argument for the convexity of ISO, which helps to explain the success of using L-BFGS-B for ISO.

Original: In summary, we have proposed three extensions to learning complex and noisy dynamics with H-NET and demonstrated the improvements they lead to: a) using the leapfrog integrator instead of Euler’s integrator; b) using recurrent instead of single-step training; and c) optimizing the initial states of each trajectory as parameters when data are noisy.
Revised: In summary, we have proposed three extensions to learning complex and noisy dynamics with H-NET and demonstrated the improvements they lead to: a) using the leapfrog integrator instead of Euler’s integrator; b) using recurrent instead of single-step training; and c) optimizing the initial states of each trajectory as parameters when data are noisy.

Original: Thorough comparisons of test errors are given in tables 1 and 2, where we highlight that the SRNN (recurrent L-L H-NET) models achieve the lowest errors.[Table] [Table]
Revised: Thorough comparisons of test errors are given in Tables 1 and 2, where we highlight that the SRNN (recurrent L-L H-NET) models achieve the lowest errors.

Original:  SRNN can learn the dynamics of a three-body system
Revised:  SRNN can learn the dynamics of a three-body system

Original: Next, we test SRNN with the three-body system, which is a well-known example of a chaotic system, meaning that a small diﬀerence in the initial condition could lead to drastically different evolution trajectories, even without noise added.
Revised: Next, we test SRNN with the three-body system, which is a well-known example of a chaotic system, meaning that a small diﬀerence in the initial condition could lead to drastically different evolution trajectories, even without noise added.

Original: As a result, even when the exact equations are known, simulating it with diﬀerent time-step sizes could also lead to qualitatively diﬀerent solutions.
Revised: As a result, even when the exact equations are known, simulating it with diﬀerent time-step sizes could also lead to qualita- [Table] [Table] tively diﬀerent solutions.

Original: Moreover, (Greydanus et al., 2019) mentions that HNN does not outperform a baseline method using O-NET in learning the three-body system’s evolution.
Revised: Moreover, (Greydanus et al., 2019) mentions that HNN does not outperform a baseline method using O-NET in learning the three-body system’s evolution.

Original: Here, we test our SRNN together with other baselines on the noiseless three-body system with the same conﬁgurations as (Greydanus et al., 2019).
Revised: Here, we test our SRNN together with other baselines on the noiseless three-body system with the same conﬁgurations as (Greydanus et al., 2019).

Original: The detailed experimental setup and model architectures are provided in Appendix A.2.
Revised: The detailed experimental setup and model architectures are provided in Appendix A.2.

Original: As we see in table 3, the best-performing model is SRNN and the second-best is the singlestep L-L H-NET.
Revised: As we see in Table 3, the best-performing model is SRNN and the second-best is the singlestep L-L H-NET.

Original: Interestingly, and perhaps counter-intuitively, they even outperform the baseline method of simulating the correct equation with the same time-step size.
Revised: Interestingly, and perhaps counter-intuitively, they even outperform the baseline method of simulating the correct equation with the same time-step size.

Original: How is this possible?
Revised: How is this possible?

Original: In short, our explanation is that the error introduced by numerical discretization could be learned and therefore compensated for by the models we learn.
Revised: In short, our explanation is that the error introduced by numerical discretization could be learned and therefore compensated for by the models we train.

Original: More concretely, once again using the concept of modiﬁed equations mentioned in section 4.1, we argue thatin theory, the ODE-based learning models, including our models based on H-NET and O-NET, could learn not the correct underlying equation, but rather the equation whose modiﬁed equation associated with our choice of numerical integrator and time-step size is the original equation.
Revised: More concretely, once again using the concept of modiﬁed equations mentioned in Section 4.1, we argue that the ODE-based learning models, including both H-NET and O-NET models, could learn not the correct underlying equation, but rather the equation whose modiﬁed equation associated with our choice of numerical integrator and time-step size is the original equation.

Original: Hence, when the time-step size is large and the error of numerical discretization[Table] is not negligible, it is not surprising that the learned equation could yield better predictions than the correct equation.
Revised: Hence, when the time-step size is large and the error of numerical discretization is not negligible, it is possible that the learned equation could yield better predictions than the correct one.

Original: In addition [Table] we also see that the recurrently trained models outperform the corresponding single-step-trained models.
Revised: In addition, we also see that the recurrently trained models outperform the corresponding single-step-trained models.

Original: Plots of the predicted trajectories are provided in Appendix E.
Revised: Plots of the predicted trajectories are provided in Appendix E.

Revised: [Table]

Original:  Learning perfect rebound with an augmented SRNN
Revised:  Learning perfect rebound with an augmented SRNN

Original: We focus in this section on the perfect rebound problem as a prototypical example of stiﬀ
Revised: We focus in this section on the perfect rebound problem as a prototypical example of stiﬀ

Original: ODE in a physical system.
Revised: ODE in a physical system.

Original: We consider a heavy billiard, subject to gravitational forces pointing downwards, and bouncing around a two-dimensional square domain delimited by impenetrable walls.
Revised: We consider a heavy billiard, subject to gravitational forces pointing downwards [Table] and bouncing around a two-dimensional square domain delimited by impenetrable walls.

Original: Whenever it hits a wall, the billiard rebounds with loss of energy by reversing the component of its momentum orthogonal to the wall surface.
Revised: Whenever it hits a wall, the billiard rebounds with loss of energy by reversing the component of its momentum orthogonal to the wall surface.

Original: Microscopically, when the billiard hits the wall, the atomic structure deformation produces strong electromagnetic forces that reverse the momentum during a very brief timescale.
Revised: Microscopically, when the billiard hits the wall, the atomic structure deformation produces strong electromagnetic forces that reverse the momentum during a very brief timescale.

Original: Simulating this microscopic phenomenon with a Hamiltonian ODE would not only be computationally expensive, but also require a detailed knowledge of the atomic structures of the billiard and the walls.
Revised: Simulating this microscopic phenomenon with a Hamiltonian ODE would not only be computationally expensive, but also require a detailed knowledge of the atomic structures of the billiard and the walls.

Original: The perfect rebound is a macroscopic approximation that treats the billiard as a point mass and the rebound as an event with zero duration inﬁnite forces.
Revised: The perfect rebound is a macroscopic approximation that treats the billiard as a point mass and the rebound as an event with zero duration inﬁnite forces.

Original: Although this approximation is convenient for high-school level derivations, the singularity makes it hard to simulate using Hamiltonian dynamics.
Revised: Although this approximation is convenient for high-school level derivations, the singularity makes it hard to simulate using Hamiltonian dynamics.

Original: We propose to approach this problem by augmenting each time step of a leapfrog-based
Revised: We propose to approach this problem by augmenting each time step of a leapfrog-based

Original: SRNN with an additional operation that models a possible rebound event,
Revised: SRNN with an additional operation that models a possible rebound event,

Original: [Equation]
Revised: [Equation]

Original: where p pret is the pre-rebound momentum vector and p postt is the post-rebound momentum vector.
Revised: where p pret is the pre-rebound momentum vector and p postt is the post-rebound momentum vector.

Original: When the vector n is zero, this operation does not change the momentum in any way.
Revised: When the vector n is zero, this operation does not change the momentum in any way.

Original: When n is a unit vector orthogonal to a wall, this operation computes the momentum reversal that is characteristic of a perfect rebound.
Revised: When n is a unit vector orthogonal to a wall, this operation computes the momentum reversal that is characteristic of a perfect rebound.

Original: Vectors n of smaller length could also be used to model energy dissipation in manner that is reminiscent of the famous LSTM forget gate (Hochreiter and Schmidhuber, 1997).
Revised: Vectors n of smaller length could also be used to model energy dissipation in manner that is reminiscent of the famous LSTM forget gate (Hochreiter and Schmidhuber, 1997).

Original: Because the billiard trajectory depends on the exact timing of the rebound event, we also need a scalar α ∈ [0 , 1] that precisely places the rebound event at time t + α ∆ t between the successive time steps t and t + ∆ t .
Revised: Because the billiard trajectory depends on the exact timing of the rebound event, we also need a scalar α ∈ [0 , 1] that precisely places the rebound event at time t + α ∆ t between the successive time steps t and t + ∆ t .

Original: The augmented leapfrog schema then becomes
Revised: The augmented leapfrog schema then becomes

Original: [Equation]
Revised: [Equation]

Original: [Equation] The predictions are obtained by an SRNN plus the rebound module trained in the way described in sectionwhere equations (6) and (8) represent ordinary leapfrog updates (4) for time steps of respective durations α ∆ t and (1 − α )∆ t .
Revised: where equations (6) and (8) represent ordinary leapfrog updates (4) for time steps of respective durations α ∆ t and (1 − α )∆ t .

Original: More precisely, we ﬁrst compute a tentative position ˜ q t +∆ t and momentum ˜
Revised: More precisely, we ﬁrst compute a tentative position ˜ q t +∆ t and momentum ˜

Original: p t +∆ t assuming no rebound,
Revised: p t +∆ t assuming no rebound,

Original: [Equation]
Revised: [Equation]

Revised: The predictions are obtained by an SRNN plus the rebound module described in section 6.

Original: then compute both n and α as parametric functions of the tentative position ˜ q t +∆ t as well as the current position q t , and ﬁnally apply the forward model (6–8).
Revised: then compute both n and α as parametric functions of the tentative position ˜ q t +∆ t as well as the current position q t , and ﬁnally apply the forward model (6–8).

Original: Note that the ﬁnal state is equal to the tentative state when no rebound occurs, that is, when n = 0 .
Revised: Note that the ﬁnal state is equal to the tentative state when no rebound occurs, that is, when n = 0 .

Original: Directly modeling n and α with a neural network taking ˜ q t +∆ t as the input would be very ineﬃcient because we would need to train with a lot of rebound events to precisely reveal the location of the walls.
Revised: Directly modeling n and α with a neural network taking ˜ q t +∆ t as the input would be very ineﬃcient because we would need to train with a lot of rebound events to precisely reveal the location of the walls.

Original: We chose instead to use visual cues in the form of a background image representing the walls.
Revised: We chose instead to use visual cues in the form of a background image representing the walls.

Original: We model n as the product of a direction vector ¯ n and a magnitude γ ∈ [0 , 1] , and we want the latter to take value close to 1 when perfect rebound actually occurs between t and t + ∆ t and close to 0 otherwise.
Revised: We model n as the product of a direction vector ¯ n and a magnitude γ ∈ [0 , 1] , and we want the latter to take value close to 1 when perfect rebound actually occurs between t and t + ∆ t and close to 0 otherwise.

Original: Both ¯ n and α are modeled as MLPs that take as input two 10x10 neighborhoods of the background image, centered at positions q t and ˜ q t +∆ t , respectively.
Revised: Both ¯ n and α are modeled as MLPs that take as input two 10x10 neighborhoods of the background image, centered at positions q t and ˜ q t +∆ t , respectively.

Original: In contrast, γ is modeled as an MLP that takes as input a smaller 2x2 neighborhood centered at ˜ q t +∆ t and is trained with an additional regularization term (cid:107) γ (cid:107) 1 in order to switch the rebound module oﬀ when it is not needed.
Revised: In contrast, γ is modeled as an MLP that takes as input a smaller 2x2 neighborhood centered at ˜ q t +∆ t and is trained with an additional regularization term (cid:107) γ (cid:107) 1 in order to switch the rebound module oﬀ when it is not needed.

Original: Training is achieved by back-propagating through the successive copies of the augmented leapfrog scheme, through the models of ¯ n , α , and γ , and also through the computation of the tentative ˜ p t +∆ t and ˜ q t +∆ t .
Revised: Training is achieved by back-propagating through the successive copies of the augmented leapfrog scheme, through the models of ¯ n , α , and γ , and also through the computation of the tentative ˜ p t +∆ t and ˜ q t +∆ t .

Original: We use 5000 training trajectories of length 10 starting from a randomly-sampled initial positions and velocities.
Revised: We use 5000 training trajectories of length 10 starting from a randomly-sampled initial positions and velocities.

Original: Similarly, we use 32 testing trajectories of length 60.
Revised: Similarly, we use 32 testing trajectories of length 60.

Original: Detailed exprimental setup is included in Appendix A.3.
Revised: Detailed exprimental setup is included in Appendix A.3.

Original: Figure 5 plots some predicted and actual testing trajectories.
Revised: Figure 5 plots some predicted and actual testing trajectories.

Original: Appendix F compares these results with the inferior results obtained with several baseline methods, including SRNN without the rebound module, and SRNN with a rebound module that does not learn α .
Revised: Appendix F compares these results with the inferior results obtained with several baseline methods, including SRNN without the rebound module, and SRNN with a rebound module that does not learn α .

Original: One limitation of our method, however, results from the assumption that there is at most one rebound event per time step.
Revised: One limitation of our method, however, results from the assumption that there is at most one rebound event per time step.

Original: Although this assumption fails when the billiard rebounds twice near a corner, as shown in the bottom right plot in Figure 5, our method still outperforms the baseline methods even in this case.  
Revised: Although this assumption fails when the billiard rebounds twice near a corner, as shown in the bottom right plot in Figure 5, our method still outperforms the baseline methods even in this case.  Conclusion

Revised: We propose the Symplectic Recurrent Neural Network, which learns the dynamics of Hamiltonian systems from data.

Revised: Thanks to symplectic integration, multi-step training and initial state optimization, it outperforms previous methods in predicting the evolution of complex and noisy Hamiltonian systems, such as the spring-chain and the three-body systems.

Revised: It can even outperform simulating with the exact equations, likely by learning to compensate for numerical discretization error.

Revised: We further augment it to learn perfect rebound from data, opening up the possibility to handle stiﬀ systems using ODE-based learning algorithms.

Original:  A Experiment setup A. The spring-chain experiment
Revised: [Figure] A Experiment setup A. The spring-chain experiment

Original: We set ∆ t = 0 .
Revised: We set ∆ t = 0 .

Original: The ground truth trajectories in both training and testing are simulated by the leapfrog integrator using ∆ t (cid:48)
Revised: The ground truth trajectories in both training and testing are simulated by the leapfrog integrator using ∆ t (cid:48)

Original: = 0 .
Revised: = 0 .

Original: 001 and coarsened into time-grids of 0 .
Revised: 001 and coarsened into time-grids of 0 .

Original: 1 with a factor of 100, since simulating with a much smaller time-step leads to much more accurate solution, which we will treat as the ground truth solution.
Revised: 1 with a factor of 100, since simulating with a much smaller time-step leads to much more accurate solution, which we will treat as the ground truth solution.

Original: The O-NET that represents f θ ( p, q ) is a one-hidden-layer MLP with 40 input units,hidden units and 40 output units.
Revised: The O-NET that represents f θ ( p, q ) is a one-hidden-layer MLP with 40 input units,hidden units and 40 output units.

Original: The H-NET that represents H θ ( p, q ) =
Revised: The H-NET that represents H θ ( p, q ) =

Original: K θ 1 ( p ) + V θ 2 ( q ) consists of two one-hidden-layer MLPs, one for K θ 1 and the other for V θ 2 .
Revised: K θ 1 ( p ) + V θ 2 ( q ) consists of two one-hidden-layer MLPs, one for K θ 1 and the other for V θ 2 .

Original: Each of the MLPs have 20 input units, 2048 hidden units and 1 output unit.
Revised: Each of the MLPs have 20 input units, 2048 hidden units and 1 output unit.

Original: The vanilla RNN and LSTM models also have hidden states of size 2048.
Revised: The vanilla RNN and LSTM models also have hidden states of size 2048.

Original: Implemented in PyTorch, the models are trained over 1000 epochs with the Adam optimizer (Kingma and Ba, 2014) with initial learning rate 0.001 and using the ReduceLROnPlateau scheduler 4 with patience 15 and factor 0.7.
Revised: Implemented in PyTorch, the models are trained over 1000 epochs with the Adam optimizer (Kingma and Ba, 2014) with initial learning rate 0.001 and using the ReduceLROnPlateau scheduler 4 with patience 15 and factor 0.7.

Original: A. The three-body experiment
Revised: A. The three-body experiment

Original: The ground truth trajectories are simulated by SciPy’s solve_ivp adaptive solver 5 with method RK45.
Revised: The ground truth trajectories are simulated by SciPy’s solve_ivp adaptive solver 5 with method RK45.

Original: We coarse-grain the simulated ground truth trajectories into time-steps of ∆ t = 1 , so that the models developed in section 4 are numerically integrated with time-step ∆ t = 1 in both training and testing.
Revised: We coarse-grain the simulated ground truth trajectories into time-steps of ∆ t = 1 , so that the models developed in section 4 are numerically integrated with time-step ∆ t = 1 in both training and testing.

Original: We intentionally set the time-step to be relatively large, so that it becomes interesting to compare these models with a baseline method of simulating the true equations with time-step ∆ t = 1 .
Revised: We intentionally set the time-step to be relatively large, so that it becomes interesting to compare these models with a baseline method of simulating the true equations with time-step ∆ t = 1 .

Original: In addition, the training data consist of 100 sample trajectories of length 10 · ∆ t = 10 , which are then turned into 900 trajectories of length 2 and 600 trajectories of length 5, respectively for single-step and recurrent training, in the same way as for the spring-chain experiments above.
Revised: In addition, the training data consist of 100 sample trajectories of length 10 · ∆ t = 10 , which are then turned into 900 trajectories of length 2 and 600 trajectories of length 5, respectively for single-step and recurrent training, in the same way as for the spring-chain experiments above.

Original: The O-NET that represents f θ ( p, q ) is a three-hidden-layer MLP with 12 input units, 512 hidden units in each hidden layer and 12 output units.
Revised: The O-NET that represents f θ ( p, q ) is a three-hidden-layer MLP with 12 input units, 512 hidden units in each hidden layer and 12 output units.

Original: The H-NET that represents
Revised: The H-NET that represents

Original: H θ ( p, q ) =
Revised: H θ ( p, q ) =

Original: K θ 1 ( p ) + V θ 2 ( q ) consists of two three-hidden-layer MLPs, one for K θ 1 and the other for V θ 2 .
Revised: K θ 1 ( p ) + V θ 2 ( q ) consists of two three-hidden-layer MLPs, one for K θ 1 and the other for V θ 2 .

Original: Each of the MLPs have 6 input units, 512 hidden units in each hidden layer and 1 output unit.
Revised: Each of the MLPs have 6 input units, 512 hidden units in each hidden layer and 1 output unit.

Original: The vanilla RNN and LSTM models also have hidden states of size 512.
Revised: The vanilla RNN and LSTM models also have hidden states of size 512.

Original: Implemented in PyTorch, the models are trained over 1000 epochs with the Adam optimizer with initial learning rate 0.0003 and using the ReduceLROnPlateau scheduler with patience 15 and factor 0.7.
Revised: Implemented in PyTorch, the models are trained over 1000 epochs with the Adam optimizer with initial learning rate 0.0003 and using the ReduceLROnPlateau scheduler with patience 15 and factor 0.7.

Original: A. The heavy billiard experiment
Revised: A. The heavy billiard experiment

Original: The full image has size 128x128 pixels.
Revised: The full image has size 128x128 pixels.

Original: The thickness of the wall is 12 pixels on each of the four sides, which leaves the free space of size 104x

Revised: The thickness of the wall is 12 pixels on each of the four sides, which leaves the free space of size 104x

Original: pixels in the middle for the billiard to move within.
Revised: pixels in the middle for the billiard to move within.

Original: The billiard has size 3x3 pixels.
Revised: The billiard has size 3x3 pixels.

Original: The O-NET that represents f θ ( p, q ) is a one-hidden-layer MLP with 4 input units, 32 hidden units and 4 output units.
Revised: The O-NET that represents f θ ( p, q ) is a one-hidden-layer MLP with 4 input units, 32 hidden units and 4 output units.

Original: The H-NET that represents H θ ( p, q ) =
Revised: The H-NET that represents H θ ( p, q ) =

Original: K θ 1 ( p ) + V θ 2 ( q ) consists of two one-hidden-layer MLPs, one for K θ 1 and the other for V θ 2 .
Revised: K θ 1 ( p ) + V θ 2 ( q ) consists of two one-hidden-layer MLPs, one for K θ 1 and the other for V θ 2 .

Original: Each of the MLPs have 2 input units, 32 hidden units and 1 output unit.
Revised: Each of the MLPs have 2 input units, 32 hidden units and 1 output unit.

Original: The vanilla RNN model also has hidden states of size 32.
Revised: The vanilla RNN model also has hidden states of size 32.

Original: For the rebound module, ¯ n is computed as the normalized output of a two-hidden-layer MLP, with 200 input units, 128 units in the ﬁrst hidden layer, 32 units in the second hidden layer and 2 output units.
Revised: For the rebound module, ¯ n is computed as the normalized output of a two-hidden-layer MLP, with 200 input units, 128 units in the ﬁrst hidden layer, 32 units in the second hidden layer and 2 output units.

Original: α is also computed using a two-hiddenlayer MLP, sharing the ﬁrst hidden layer units with the MLP for ¯ n , and having 32 units in the second hidden layer and 1 output unit.
Revised: α is also computed using a two-hiddenlayer MLP, sharing the ﬁrst hidden layer units with the MLP for ¯ n , and having 32 units in the second hidden layer and 1 output unit.

Original: γ is computed by passing through sigmoid the output of a two-hidden-layer MLP, with 4 input units, 16 units in each hidden layer and 1 output unit.
Revised: γ is computed by passing through sigmoid the output of a two-hidden-layer MLP, with 4 input units, 16 units in each hidden layer and 1 output unit.

Original: All of the activation functions are tanh except for the hidden-to-output activation in the MLP for α , where ReLU is used.
Revised: All of the activation functions are tanh except for the hidden-to-output activation in the MLP for α , where ReLU is used.

Original: Implemented in PyTorch, the models are trained over 1500 epochs with the Adam optimizer with initial learning rate 0.005 and using the ExponentialLR scheduler 6 with decay factor 0.99 until the learning rate reaches 0.0001.
Revised: Implemented in PyTorch, the models are trained over 1500 epochs with the Adam optimizer with initial learning rate 0.005 and using the ExponentialLR scheduler 6 with decay factor 0.99 until the learning rate reaches 0.0001.

Original: We set ∆ t = 0 . 1 , and use 5000 trajectories of length 10 · ∆ t = 1 as training data, and 32 trajectories of length 60 · ∆ t = 6 as testing data.
Revised: We set ∆ t = 0 . 1 , and use 5000 trajectories of length 10 · ∆ t = 1 as training data, and 32 trajectories of length 60 · ∆ t = 6 as testing data.

Original: B The Maximum Likelihood Estimation perspective
Revised: B The Maximum Likelihood Estimation perspective

Original: In the presence of noise, we can interpret the learning problem described in section 3.3 above from the perspective of maximum likelihood inference, which also provides justiﬁcation for treating the initial states as trainable parameters.
Revised: In the presence of noise, we can interpret the learning problem described in section 3.3 above from the perspective of maximum likelihood inference, which also provides justiﬁcation for treating the initial states as trainable parameters.

Original: We deﬁne models as follow:
Revised: We deﬁne models as follow:

Original: [Equation]
Revised: [Equation]

Original: and L ( θ |{ z i } ni =1 ) =
Revised: and L ( θ |{ z i } ni =1 ) =

Original: P ( { z i } ni =1 | θ ) is the likelihood function given the time-series data { z i } ni =1 .
Revised: P ( { z i } ni =1 | θ ) is the likelihood function given the time-series data { z i } ni =1 .

Original: Note that this model assumes independence between z i and z j for i (cid:54) =
Revised: Note that this model assumes independence between z i and z j for i (cid:54) =

Original: j once θ is ﬁxed.
Revised: j once θ is ﬁxed.

Original: If we are to perform maximum likelihood inference, we arrive at the following:
Revised: If we are to perform maximum likelihood inference, we arrive at the following:

Original: [Equation]
Revised: [Equation]

Original: which is equivalent to
Revised: which is equivalent to

Original: [Equation]
Revised: [Equation]

Original: This provides a motivation for using the L 2 loss, as we did in the experiments.
Revised: This provides a motivation for using the L 2 loss, as we did in the experiments.

Original: So far, we consider θ as the only parameter of the model deﬁned by equations 10, and therefore the only argument of the likelihood function, while ˆ z 0 is ﬁxed to be the observed initial state z 0 .
Revised: So far, we consider θ as the only parameter of the model deﬁned by equations 10, and therefore the only argument of the likelihood function, while ˆ z 0 is ﬁxed to be the observed initial state z 0 .

Original: As a generalization, we can consider a strictly larger family of models by allowing z 0 to vary as well.
Revised: As a generalization, we can consider a strictly larger family of models by allowing z 0 to vary as well.

Original: In this way, we treat both θ and z 0 as the parameters in the model and therefore arguments of the likelihood function that we optimize on.
Revised: In this way, we treat both θ and z 0 as the parameters in the model and therefore arguments of the likelihood function that we optimize on.

Original: In other words, the model becomes
Revised: In other words, the model becomes

Original: [Equation]
Revised: [Equation]

Original: and the optimization problem becomes
Revised: and the optimization problem becomes

Original: [Equation]
Revised: [Equation]

Original: which justiﬁes optimizing over the initial states p 0 , q 0 in addition to the neural network parameters θ as described in the previous section.
Revised: which justiﬁes optimizing over the initial states p 0 , q 0 in addition to the neural network parameters θ as described in the previous section.

Original: Such an interpretation is similar to approaches for parameter estimation in the literature of inverse problems and systems biology, though in those cases the parameters of interest appear directly in ODEs instead of via neural networks (Peifer and Timmer, 2007;
Revised: Such an interpretation is similar to approaches for parameter estimation in the literature of inverse problems and systems biology, though in those cases the parameters of interest appear directly in ODEs instead of via neural networks (Peifer and Timmer, 2007;

Original: Stapor et al., 2018).
Revised: Stapor et al., 2018).

Original: In particular, jointly optimizing the parameters in the model as well as the initial value is called the initial value approach.
Revised: In particular, jointly optimizing the parameters in the model as well as the initial value is called the initial value approach.

Original: However, despite the success we demonstrate in section 4.2, two diﬃculties of this approach have been pointed out: 1)
Revised: However, despite the success we demonstrate in section 4.2, two diﬃculties of this approach have been pointed out: 1)

Original: The optimization could converge to local minima; 2)
Revised: The optimization could converge to local minima; 2)

Original: The numerical solution of the ODE can be unstable (Peifer and Timmer, 2007).
Revised: The numerical solution of the ODE can be unstable (Peifer and Timmer, 2007).

Original: As explained in section 4.1, using HNN together with the leapfrog integrator mitigates the second issue.
Revised: As explained in section 4.1, using HNN together with the leapfrog integrator mitigates the second issue.

Original: But what about the ﬁrst issue?
Revised: But what about the ﬁrst issue?

Original: In particular, even if we assume that the optimization of the neural network can work “magically” well and do not suﬀer from bag local minima, what about optimizing the initial value ˆ z 0 ?
Revised: In particular, even if we assume that the optimization of the neural network can work “magically” well and do not suﬀer from bag local minima, what about optimizing the initial value ˆ z 0 ?

Original: C Symplecticness and initial-state-optimization convexity
Revised: C Symplecticness and initial-state-optimization convexity

Original: The success of optimizing on the initial state of the system in addition to the recurrent
Revised: The success of optimizing on the initial state of the system in addition to the recurrent

Original: H-NET and O-NET models as described in section 4.2 raises the following question: If we already have a relatively well-trained H-NET or O-NET, is the optimization on the initial values convex?
Revised: H-NET and O-NET models as described in section 4.2 raises the following question: If we already have a relatively well-trained H-NET or O-NET, is the optimization on the initial values convex?

Original: We formalize the question below and provide a heuristic answer.
Revised: We formalize the question below and provide a heuristic answer.

Original: For simplicity, we restrict our attention to autonomous ODEs, which means that the function f in dzdt =
Revised: For simplicity, we restrict our attention to autonomous ODEs, which means that the function f in dzdt =

Original: f ( z ) does not depend on t .
Revised: f ( z ) does not depend on t .

Original: Assuming existence and uniqueness of solutions, there exists a function φ t that maps each initial state ˆ z 0 to the state of the system after evolving from ˆ z 0 for time t , φ t (ˆ z 0 ) .
Revised: Assuming existence and uniqueness of solutions, there exists a function φ t that maps each initial state ˆ z 0 to the state of the system after evolving from ˆ z 0 for time t , φ t (ˆ z 0 ) .

Original: This function is usually called the ﬂow map.
Revised: This function is usually called the ﬂow map.

Original: Flow maps have also been deﬁned for numerical solutions of ODEs, by letting φ t (ˆ z 0 )
Revised: Flow maps have also been deﬁned for numerical solutions of ODEs, by letting φ t (ˆ z 0 )

Original: =
Revised: =

Original: Integrator ( z 0 , f, { t i } Ti =0 ) with t 0 = 0 .
Revised: Integrator ( z 0 , f, { t i } Ti =0 ) with t 0 = 0 .

Original: We can extend this deﬁnition to all the trainable models we have considered, including the models based on O-NET and H-NET by deﬁning φ t (ˆ z 0 ) to be the state of the system after letting the system evolve from initial state ˆ z 0 for time t , for suitable choices of t .
Revised: We can extend this deﬁnition to all the trainable models we have considered, including the models based on O-NET and H-NET by deﬁning φ t (ˆ z 0 ) to be the state of the system after letting the system evolve from initial state ˆ z 0 for time t , for suitable choices of t .

Original: For example, for O-NET, we have φ t (ˆ z 0 ) =
Revised: For example, for O-NET, we have φ t (ˆ z 0 ) =

Original: Integrator ( z 0 , f θ , { t i } Ti =0 ) .
Revised: Integrator ( z 0 , f θ , { t i } Ti =0 ) .

Original: Suppose we impose an L2 loss on φ t (ˆ z 0 ) , e t (ˆ z 0 )
Revised: Suppose we impose an L2 loss on φ t (ˆ z 0 ) , e t (ˆ z 0 )

Original: = (cid:107) φ t
Revised: = (cid:107) φ t

Original: (ˆ z 0 )
Revised: (ˆ z 0 )

Original: − z t (cid:107) 22 , where z t corresponds to the observed data at time t .
Revised: − z t (cid:107) 22 , where z t corresponds to the observed data at time t .

Original: The question is, is e t ( z ) a (perhaps locally) convex function of z , for what functions and numerical integrators?
Revised: The question is, is e t ( z ) a (perhaps locally) convex function of z , for what functions and numerical integrators?

Original: To understand convexity, we compute the gradient and the Hessian as follow.
Revised: To understand convexity, we compute the gradient and the Hessian as follow.

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: (16) where F t ( z ) is the Jacobian matrix of the ﬂow map, deﬁned as F t ( z ) ij =
Revised: (16) where F t ( z ) is the Jacobian matrix of the ﬂow map, deﬁned as F t ( z ) ij =

Original: ∂∂z j ( φ t ( z ) i ) , and
Revised: ∂∂z j ( φ t ( z ) i ) , and

Original: G t ( z ) is a third-order tensor contains the second order derivatives of the ﬂow map, deﬁned as G t ( z ) ijk = ∂ 2 ∂z i
Revised: G t ( z ) is a third-order tensor contains the second order derivatives of the ﬂow map, deﬁned as G t ( z ) ijk = ∂ 2 ∂z i

Original: z j ( φ t ( z ) k ) .
Revised: z j ( φ t ( z ) k ) .

Original: We use · (3) to denote the dot product in the third dimension.
Revised: We use · (3) to denote the dot product in the third dimension.

Original: F t ( z ) (cid:124) · F t ( z ) is symmetric positive semideﬁnite for any matrix F t ( z ) .
Revised: F t ( z ) (cid:124) · F t ( z ) is symmetric positive semideﬁnite for any matrix F t ( z ) .

Original: If φ t corresponds to either the exact ﬂow map of a Hamiltonian system or the ﬂow of a symplectic integrator, such as the leapfrog integrator, applied to a Hamiltonian system, then F t ( z ) is a symplectic matrix, implying that det( F t ( z ))
Revised: If φ t corresponds to either the exact ﬂow map of a Hamiltonian system or the ﬂow of a symplectic integrator, such as the leapfrog integrator, applied to a Hamiltonian system, then F t ( z ) is a symplectic matrix, implying that det( F t ( z ))

Original: = 1 .
Revised: = 1 .

Original: Hence, det( F t ( z ) (cid:124) · F t ( z ))
Revised: Hence, det( F t ( z ) (cid:124) · F t ( z ))

Original: = 1 , which further implies that F t ( z ) (cid:124) ·
Revised: = 1 , which further implies that F t ( z ) (cid:124) ·

Original: F t ( z ) is a positive deﬁnite matrix.
Revised: F t ( z ) is a positive deﬁnite matrix.

Original: Therefore, non-rigorously, when (cid:107) φ t ( z ) − z t (cid:107)is small and so the ﬁrst term on the right hand side of equation 16 is negligible compared to the least eigenvalue of F t ( z ) (cid:124) · F t ( z ) , the entire Hessian matrix ∂ 2 ∂z 2 e t ( z ) is also positive deﬁnite, implying strong convexity of the optimization problem.
Revised: Therefore, non-rigorously, when (cid:107) φ t ( z ) − z t (cid:107)is small and so the ﬁrst term on the right hand side of equation 16 is negligible compared to the least eigenvalue of F t ( z ) (cid:124) · F t ( z ) , the entire Hessian matrix ∂ 2 ∂z 2 e t ( z ) is also positive deﬁnite, implying strong convexity of the optimization problem.

Original: If φ t is the exact ﬂow map, then (cid:107) φ t ( z ) − z t (cid:107) being small means that noise in the data is small.
Revised: If φ t is the exact ﬂow map, then (cid:107) φ t ( z ) − z t (cid:107) being small means that noise in the data is small.

Original: If φ t is the ﬂow map of a learned model, then it means that we have a model close to the true underlying system in addition to not having too much noise in the data.
Revised: If φ t is the ﬂow map of a learned model, then it means that we have a model close to the true underlying system in addition to not having too much noise in the data.

Original: Translating back to the learning problem, we see that, heuristically, when the model we use is close to symplectic, which is likely if the underlying system is a Hamiltonian system, and trained to be close enough to the true underlying system, and the noise in the data is small enough, then the optimization problem on the initial state is strongly convex.
Revised: Translating back to the learning problem, we see that, heuristically, when the model we use is close to symplectic, which is likely if the underlying system is a Hamiltonian system, and trained to be close enough to the true underlying system, and the noise in the data is small enough, then the optimization problem on the initial state is strongly convex.

Original: D Additional plots of the spring-chain experiments D. Noiseless data
Revised: D Additional plots of the spring-chain experiments D. Noiseless data

Original: (section 4.1)
Revised: (section 4.1)

Original: [Figure]
Revised: [Figure]

Original: D. Noisy data (section 4.2)
Revised: D. Noisy data (section 4.2)

Original: [Figure]
Revised: [Figure]

Original: [Figure] 
Revised: [Figure] [Figure] [Figure] [Figure]

Revised: [Figure]

Original: [Figure] [Figure] 
Revised: [Figure] [Figure] [Figure]

Original: [Figure][Figure] [Figure] [Figure]
Revised: [Figure]

Original: [Figure] [Figure][Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: Vanilla RNN
Revised: Vanilla RNN

Original: Noiseless dynamic
Revised: Noiseless dynamic

Original: Observations
Revised: Observations

Original: [Figure]
Revised: [Figure]

Original: LSTM
Revised: LSTM

Original: Noiseless dynamic
Revised: Noiseless dynamic

Original: Observations
Revised: Observations

Original: E Additional plots of the three-body experiments
Revised: E Additional plots of the three-body experiments

Original: In each of the plots below, the three dashdot curves represent the ground truth trajectories of the three masses, and the three sequences of dots are the predictions made by each method.
Revised: In each of the plots below, the three dashdot curves represent the ground truth trajectories of the three masses, and the three sequences of dots are the predictions made by each method.

Original: [Figure]
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: F Additional plots of the heavy billiard experiment
Revised: F Additional plots of the heavy billiard experiment