Title: UEPI: Universal  Energy-Behavior-Preserving Integrators for Energy Conservative/Dissipative  Differential Equations
Venue: NeurIPS 2025
Accepted: yes
Keywords: energy-preserving integrator

energy-dissipating integrator

Hamiltonian system

numerical integrator

discrete gradient method
Abstract: Physical phenomena in the real world are often described by energy-based modeling theories, such as Hamiltonian mechanics or the Landau theory. It is known that physical phenomena based on these theories have an energy conservation law or a dissipation law. Therefore, in the simulations of such physical phenomena, numerical methods that preserve the energy-conservation or dissipation laws are desirable. However, because various energy-behavior-preserving numerical methods have been proposed, it is difficult to discover the best one. In this study, we propose a method for learning highly accurate energy-behavior-preserving integrators from data. Numerical results show that our approach certainly learns energy-behavior-preserving numerical methods that are more accurate than existing numerical methods for various differential equations, including chaotic Hamiltonian systems, dissipative systems, and a nonlinear partial differential equation. We also provide universal approximation theorems for the proposed approach.

UEPI: Universal Energy-Behavior-Preserving Integrators for Energy Conservative/Dissipative Differential Equations

Abstract

Physical phenomena in the real world are often described by energy-based modeling theories, such as Hamiltonian mechanics or the Landau theory. It is known that physical phenomena based on these theories have an energy conservation law or a dissipation law. Therefore, in the simulations of such physical phenomena, numerical methods that preserve the energy-conservation or dissipation laws are desirable. However, because various energy-behavior-preserving numerical methods have been proposed, it is difficult to discover the best one. In this study, we propose a method for learning highly accurate energy-behavior-preserving integrators from data. Numerical results show that our approach certainly learns energy-behavior-preserving numerical methods that are more accurate than existing numerical methods for various differential equations, including chaotic Hamiltonian systems, dissipative systems, and a nonlinear partial differential equation. We also provide universal approximation theorems for the proposed approach.

1 Introduction

Differential equations serve as fundamental tools for modeling the dynamics of various physical systems. However, most real-world differential equations cannot be solved analytically, necessitating the use of numerical methods to obtain approximate solutions. A critical challenge in this context is ensuring both accuracy and stability over long simulation times, especially for systems governed by complex or nonlinear dynamics.

In recent years, advances in machine learning have inspired a new class of data-driven numerical methods [1, 12, 21], where neural networks are used to optimize the coefficients or design high-order schemes for classical integrators such as Runge-Kutta methods. These methods offer remarkable adaptability and high short-term accuracy when tailored to specific problems. However, a significant limitation is their lack of preservation of key invariants, such as energy conservation or symplecticity, which are essential for long-term simulations. In contrast, geometric numerical integrators [24] are specifically designed to preserve such intrinsic structures, thereby ensuring better performance over long time intervals. Among them, energy-preserving methods such as the discrete gradient methods[19, 47] have gained attention for their ability to conserve the system's energy evolution precisely. To the best of our knowledge, existing literature has not yet explored the potential of neural networks in optimizing energy-preserving geometric integrators.

In this study, we propose a method for learning numerical integrators that preserve the energy behaviors of target differential equations. More specifically, the focus of this paper is on dynamical systems characterized by their energy-conserving or dissipating properties. We present a general form of the discrete gradient method and propose a method for learning highly accurate energy-behaviorpreserving integrators from data by employing neural networks. In addition, we show universal approximation theorems for the proposed approach.

The main contributions of this paper are as follows.

We characterize and then parameterize the energy-behavior-preserving numerical integrators.

We propose neural network models with universal approximation properties for the energybehavior-preserving numerical integrators. Numerical experiments have confirmed that the method can be applied to a wide range of phenomena, including chaotic conservative systems, dissipative systems, and nonlinear partial differential equations.

Numerical experiments also show that the proposed method, in fact, finds highly accurate computational methods, which are difficult to discover theoretically. In particular, it is numerically shown that numerical schemes derived by the proposed method are highly accurate and stable even for chaotic systems.

2 Related Work

Neural networks for improving numerical methods represent a hybrid paradigm. One line of research focuses on optimizing classical numerical methods through data-driven parameterization rather than approximating dynamics directly. Anastassi [1] developed artificial neural networks(ANNs) to generate the coefficients of two-stage Runge-Kutta methods specifically optimized for solving the two-body problem. Dehghanpour et al. [12] utilized ANNs to automatically compute the optimal coefficients of third-order Runge-Kutta methods. Similarly, Guo et al. [21] explored machine learning methods based on the Runge-Kutta-integrator architecture, capable of automatically generating high-order integrators for specific families of ODEs. Universal Numerical Integrators (UNIs) [60, 61] proposed a framework, which combines neural networks as universal approximators with numerical integrators. A second distinct line of work investigates learning discretization schemes from data. Bar-Sinai et al. [2] introduced a data-driven discretization approach that employs neural networks to optimize the computation of spatial derivatives. Maddu et al. [44] developed STENCIL-NET, which can adaptively learn local discretization operators based on solution data, addressing long-term prediction challenges for unknown nonlinear PDEs. Ranade et al. [55] introduced DiscretizationNet, a generative CNN-based approach to model discretization schemes. Our approach follows this hybrid

paradigm but shifts the focus from general integrator optimization to preserving the energy behavior of dynamical systems. We achieve this by combining neural networks with the structure-preserving discrete gradient method, rather than modifying traditional integration integrators.

Geometric numerical integration focuses on designing numerical methods that preserve certain qualitative characteristics of dynamical systems. Unlike traditional methods such as the explicit Euler method or the classical Runge-Kutta method, which may lead to long-term errors, geometric integrators maintain properties that make them particularly suitable for long-time simulations of physical systems.

Geometric numerical integrators are generally divided into symplectic integrators and energypreserving integrators, as Zhong [67] and Chartier et al. [6] have shown that it is impossible for a numerical integrator to preserve both properties. Symplectic integrators ([16, 56]) are integrators that preserve the symplectic structure of Hamiltonian systems. Energy-preserving methods aim to conserve the total energy of the system. Among energy-preserving methods, the discrete gradient method([19, 35-37, 47]) is particularly successful and has also been extended to partial differential equations ([5, 46]). Eidnes [15] develop order theory for discrete gradient methods. Projection methods([10, 14, 22, 23]) can also be employed to preserve energy by projecting the numerical solution back onto the manifold that satisfies a certain energy. Norton et al. [50] show that linear projection methods are a subset of discrete gradient methods. Miyatake and Butcher [48] provided a condition to be energy-preserving for continuous stage Runge-Kutta methods. For other geometric numerical integrators, see ([4, 24, 57]) for details. In our study, we place particular emphasis on discrete gradient methods, as they inherently preserve the energy behaviors, making them highly relevant for simulations involving Hamiltonian dynamics and dissipative systems.

Neural network-Based Physical Modelings have been widely applied to the study of differential equations by combining data-driven learning with physical principles. Chen et al. [7] introduced Neural ODEs, which model continuous-time dynamics via parameterized vector fields. This idea has been extended in various directions [3, 13, 65]. Another influential framework is that of PhysicsInformed Neural Networks (PINNs) [54], which incorporates differential equation residuals into the loss function to enforce physical consistency during training. Numerous extensions have been developed to enhance their accuracy, robustness, efficiency, and scalability [30, 32, 49, 54, 58]. Operator learning [38-42, 64] aims to learn mappings between infinite-dimensional function spaces. Dynamic Mode Decomposition (DMD) and the Koopman operator theory have emerged as powerful data-driven techniques for approximating nonlinear dynamical systems with linear representations [34, 43, 59, 63]. Beyond generic data-driven models, structure-preserving neural networks have been proposed to better reflect the physical properties of dynamical systems. Hamiltonian Neural Networks (HNNs) [20] model the Hamiltonian to enforce energy conservation. Extensions of HNNs [11, 25, 45] have incorporated flexibility and accuracy improvements. Lagrangian Neural Networks (LNNs) [9] take a similar approach via Lagrangian mechanics. Many studies also focus on the symplecticity [8, 31, 51, 62]. For energy-dissipative systems, some studies have introduced neural networks grounded in thermodynamic principles such as the GENERIC framework to ensure stability, interpretability, and physical consistency in modeling irreversible dynamics [26, 27, 66]. Despite their effectiveness in capturing qualitative physical properties, these methods are not numerical integrators. They focus on learning dynamics or flow maps, and their approximation quality can be difficult to control or quantify in a classical numerical analysis sense. This limits their applicability in contexts where rigorous accuracy guarantees or long-term stability are required.

In summary, our method bridges geometric numerical integration and neural networks by learning energy-behavior-preserving integrators. This offers improved long-term stability and accuracy in simulating a wide range of systems.

3 Target Energy-Based Dynamical Systems

We focus on differential equations written in the form

Here H ∈ C 1 ( R d , R ) and S ∈ C ( R d , R d × d ) . Differential equations in this form describe various physical systems modeled by ordinary differential equations (ODEs) and discretized partial differential equations (PDEs) [17, 52]. Two classes of problems are usually considered:

Conservative case: S ( u ) T = -S ( u ) (skew-symmetric,)

Dissipative case: S ( u ) ≤ O (negative semi-definite.)

Note that S ≤ O means that the matrix S is negative semi-definite, i.e., u ⊤ Su ≤ 0 for any vector u . Also, if a matrix S is skew-symmetric, then u ⊤ Su = 0 for any vector u .

The systems described by the above equations have the following properties.

Theorem 3.1. The system has the energy dissipation law if S ≤ O and the energy conservation law if S is skew-symmetric.

See Appendix A for the proof. Such differential equations include various equations that appear in physics. For example, if S is skew-symmetric and invertible and also satisfies a certain condition called the Jacobi identity, the equation describes Hamiltonian systems. As the Hamilton equation is a generalization of Newton's equation of motion, mechanical systems that can be described by classical mechanics, such as the motion of a pendulum. As an example outside of physics, the Lotka-Volterra predator-prey model is also a Hamiltonian system. On the other hand, this equation describes dissipative systems when S is negative-definite. A typical example of dissipative systems is the equation of motion for mechanical systems with friction. Thermodynamic systems can often be expressed as dissipative systems as well. Also, this equation has a geometric background; see Appendix B for details.

The system (1) also describes semi-discretized Hamiltonian or dissipative PDEs in space. For example, the PDEs of the form

are an example of Hamiltonian PDEs, where H ( u, u x ) is the energy density and δH/δu denotes the variational derivative of H , which is defined by δH δu := ∂H ∂u x -∂ ∂x ∂H ∂u . This equation admits the energy conservation law: d d t ∫ H d x = 0 under, for example, the periodic boundary condition. Similarly, the PDEs

admits the energy dissipation property: d d t ∫ H d x ≤ 0 under the periodic boundary condition. See Appendix C for details on the semi-discretization.

Hamiltonian PDEs include the Maxwell equation and the shallow water equations such as the KdV equation, the advection equation, and the Burgers equation. Dissipative PDEs express physical systems derived from the Landau free-energy minimization including the Cahn-Hilliard equation and the phase-field models for phase transitions and pattern formulations. Other target equations include the equations with complex state variables, such as the Schrödinger equation and the GinzburgLandau equation. See, e.g., [17] for details.

4 Methods

4.1 Discrete Gradient Methods as Energy-Behavior-Preserving Integrators

A popular way of discretizing (1) is by a class of numerical schemes called discrete gradient methods defined as

where h is the step size. In this discrete counterpart to (1) the skew-symmetrix matrix S ( u ) has been replaced by the matrix ¯ S ( u, v ) depending on two arguments, but still required to be skew-symmetric. ¯ ∇ H : R d × R d → R d is the discrete gradient defined as follows.

Definition 4.1. A discrete gradient for a smooth function H : R d → R is a continuous mapping ¯ ∇ H : R d × R d → R d that satisfies the following properties:

The first condition corresponds to the discrete chain-rule d H ( δu ; u ) = ∇ H ( u ) · δu for the Fréchet derivative d H ( · ; u ) of H at u , where δu is an infinitesimal change of u . The second condition ensures that a discrete gradient ¯ ∇ H is certainly an approximation of the gradient ∇ H . The inner product is typically the standard Hermitian inner product for ODEs and the discrete L 2 inner product for discretized PDEs. For details on discrete gradient methods, see, e.g., [15].

The discrete gradient schemes preserve the energy behavior of the target differential equations in the following sense.

Theorem 4.2. The discrete gradient scheme (4) admits the discrete energy conservation law

if ¯ S is skew-symmetric and the discrete energy dissipation law

if ¯ S ≤ O . In particular, if the system is dissipative, the amount of energy dissipation is an approximation of that of the original differential equation.

Proof. The proof is exactly the same as that for Theorem 3.1. See Appendix A for details.

There are several popular choices for discrete gradients. The average vector field (AVF) discrete gradient [53] is defined as

The Gonzales discrete gradient [19] is defined by

Another example is the Itoh-Abe discrete gradient [29]

4.2 Universal Energy-Behavior-Preserving Integrators

The above three discrete gradients are just examples from the family of maps ¯ ∇ H satisfying (5) and (6) above. In this paper, we fully characterize the set of all possible discrete gradients as follows .

Let ¯ ∇ 0 H be a particular choice of discrete gradient, henceforth denoted the reference gradient . Then, from (5) it follows that any other discrete gradient ¯ ∇ H ( u, v ) satisfies

Thus, the set of all discrete gradients can be described, and hence, parameterized for learning as follows.

Theorem 4.3 (Characterization of discrete gradients) . For a fixed function H , the set of all discrete gradient operators ¯ ∇ H : R d × R d → R d has the following parametrization

where A ( u, v ) is continuous and skew symmetric for all u, v ∈ R d , and where ¯ ∇ 0 H ( u, v ) is a fixed reference gradient satisfying (5) and (6).

Proof.

In fact, we can prove a stronger statement: all energy-conserving numerical methods for the Hamilton equations can be written in this form .

Theorem 4.4. For a fixed function H , any energy-preserving integrators for the Hamilton equation

can be written as a discrete gradient scheme with a discrete gradient of the following form:

where A ( u, v ) is skew symmetric for all u, v ∈ R d and ¯ ∇ 0 H ( u, v ) is a fixed reference gradient satisfying (5) and (6).

Proof.

Note that we cannot obtain a similar theorem for dissipative systems in a straightforward way because, in the proof of the above theorem, we use the energy-conservation law of the conservative systems and also the non-degeneracy of S .

Proposed Method Although the discrete gradient method preserves the energy behaviors, it still introduces numerical errors that accumulate and grow over time. In this study, we propose a method for learning highly accurate energy-behavior-preserving integrators. The method can be summarized as Figure 1. This paper aims to optimize the discrete gradient based on the above characterization of discrete gradients, with the aim of not only preserving the energy but also yielding numerical solutions closer to the true solution.

The proposed method is formulated as follows:

where M NN ( u ( n ) , u ( n +1) ) is a skew-symmetric matrix learned by a neural network that takes u ( n ) and u ( n +1) as

u(n),u(n+1)

(u)n

MNN

u(n+1)

NnH(u(n), u(n+1))

(u)n - (I+u)n)NNW + (I+u)n(u)n)H△

u(n+1) -u(n)

= SVNnH(u(n),u(n+1)

h

the input. The universal approximation properties are the direct consequences of the above theorems.

Theorem 4.5 (Universal approximation theorem for energy-preserving integrators for Hamiltonian systems) . Suppose that the phase space is compact and M NN is modeled by using a multilayer perceptron (MLP) with a sufficiently smooth activation function so that the MLPs admit the universal approximation property for the continuous skew symmetric matrices M ( u, v ) 's. Then the proposed method admits the universal approximation property for energy-preserving integrators for Hamiltonian systems.

Theorem 4.6 (Universal approximation theorem for energy-dissiptive discrete-gradient integrators for dissipative systems) . Suppose that the phase space is compact and M NN is modeled by using a MLP with a sufficiently smooth activation function so that the MLPs admit the universal approximation property for the continuous skew symmetric matrices M ( u, v ) 's. Then the proposed method admits the universal approximation property for energy-dissiptive discrete-gradient integrators for dissipative systems.

Given a time series data u ( n ) 's at time t ( n ) 's, the proposed model is trained to minimize the squared error between the left- and right-hand sides of (9):

where h ( n ) = t ( n +1) -t ( n ) .

If the matrix M NN satisfies M NN ( u, v ) = M NN ( v, u ) , then the proposed integrator (9) at least has the second-order accuracy. Otherwise, the proposed integrator (9) is the first-order method [24].

5 Experiment

We used standard multilayer perceptrons with the hyperbolic tangent (tanh) as the activation function to learn the matrix M NN . The hyperparameters for training are provided in Appendix D.1. The optimizer is set to Adam [33]. As for the loss function, we chose MSELoss (Mean Squared Error Loss). Final loss values are provided in Appendix D.2. All experiments were performed on an Intel i5-13500H CPU, and computations were done in double precision.

In our experiments, we employ the AVF and Itoh-Abe methods as reference discrete gradient. As demonstrated in the experiments provided in the Appendix D.2, for the choice of the reference discrete gradient, we prefer to use the AVF method for its accuracy when it is computationally feasible. Otherwise, the Itoh-Abe method offers a simpler approach, making it more suitable for general equations. Moreover, to improve the performance of the Itoh-Abe method, we employ its time-symmetric version as the reference method in all subsequent experiments as follows:

Although the proposed method can theoretically guarantee at most second-order accuracy, we conduct comparisons with reference numerical methods using half step size and double steps, as well as with fourth-order AVF and Itoh-Abe methods constructed by the composition method [24].

The implementation code is available in supplementary material on OpenReview: https:// openreview.net/forum?id=G2uILEbcLF .

5.1 AVF Discrete-Gradient-Based Method for Energy-Conservative Nonlinear System

q2

q1

Hénon-Heiles System The Hénon-Heiles model (Figure 2) describes the stellar motion restricted to a plane. It shows chaotic motion and has a separable and polynomial Hamiltonian given by:

Since this is a polynomial Hamiltonian equation, the A VF method can be easily applied. Due to the chaotic characteristics of the system, even small errors can evolve into large deviations over long iterations. Although the A VF method conserves energy, it can still lead to significant errors for this system.

We use two sets of data: the flow data and the irregular data for the Hénon-Heiles equation. The training flow data { ( u ( n -1) , u ( n ) ) | n = 1 , ..., N } consists of N = 100 pairs of points obtained from the trajectory calculated using a high-order integrator sampled with a time step h = 0 . 3 , starting from u (0) = ( q (0) 1 , q (0) 2 , p (0) 1 , p (0) 2 ) = (0 . 3 , 0 . 3 , -0 . 2 , -0 . 2) . After training, we use the trained integrator to compute the flow starting at u ( N ) for 300 steps.

0.75

Ground truth

0.75

Proposed method

0.75

AVF solution

0.75

AVF half stepsize

0.75

0.75

Sympnets

Global error

1.25

0.50

0.50

0.50

0.50

0.50

0.50

1.00

0.25

0.25

0.25

0.25

0.25

0.25

error

0.75

0.00

0.00

0.00

0.00

0.00

0.00

0.50

0.25

0.25

0.25

0.25

0.25

0.25

0.25

0.50

0.5

0.0

q1

0.5

0.50

0.5

0.0

q1

0.5

0.50

0.5

0.0

q1

0.5

0.50

0.5

0.0

q1

0.5

0.50

0.5

0.0

q1

0.5

0.50

0.5

0.0

q1

0.5

0.00

0

25

50

t

75

We also consider the computational time because the proposed numerical scheme is computationally more expensive than the AVF method due to the use of the neural network. When using a 10 -layer, 100 -width network for the Hénon-Heiles experiment, the computation time for the trajectory of 300 points falls between the AVF method and the AVF (half step size), which uses half the time step size and doubles the number of steps. For example, The computational times were: Proposed: 5.522 [sec], AVF: 3.608 [sec], AVF (half step size): 6.616 [sec]. In fact, the proposed method outperforms the fourth-order method, which offer higher accuracy while require more computation time than AVF(half step size). We also compared the trajectory computed using a 10 -layer, 100 -width G-sympnets trained for 100 , 000 iterations with learning rate 0 . 001 and the same training dataset by modifying the officially released code [31]. The predicted trajectory and the corresponding global

error ( ∥ u ( n ) -u ( nh ) ∥ 2 ) comparison are illustrated in Figure 3, from which we can deduce that the proposed method yields trajectory closer to the ground truth, and for details see Appendix D.4.

We further conducted experiments using irregular data. The training dataset { (( u ( i -1) , h ( i ) ) , u ( i ) ) | i = 1 , . . . , I } consists of I = 100 one-step input-output pairs, where the elements of u ( i -1) are randomly sampled from the domain [ -1 , 1] , and the corresponding step sizes h ( i ) are randomly chosen from [0 , 0 . 5] . Each u ( i ) represents the one-step solution computed from u ( i -1) using step size h ( i ) . After training, we simulate the flow starting from the initial state u (0) = ( q (0) 1 , q (0) 2 , p (0) 1 , p (0) 2 ) = (0 . 3 , 0 . 3 , -0 . 2 , -0 . 2) using a fixed step size h = 0 . 3 for 100 time steps. The numerical results indicate that the proposed method yields significantly improved accuracy over baseline methods. For detailed comparisons, please refer to Appendix D.4.

We also conducted experiments on the pendulum system to validate the generality of our proposed method. Detailed experimental settings and results are provided in Appendix D.3.

5.2 Itoh-Abe Discrete-Gradient-Based Method for Energy-Conservative Nonlinear System

q2

q1

Two-body Problem In the two-body problem, the motion of two bodies that attract each other is considered. We assume that one of the bodies is fixed at the center of the coordinate system and the position of the other one is given as q = ( q 1 , q 2 ) (Figure 4.) The Hamiltonian is given as follows:

The training flow data { ( u ( n -1) , u ( n ) ) | n = 1 , ..., N } consists of N = 30 pairs with a fixed time step h = 0 . 1 , starting from u (0) = ( q (0) 1 , q (0) 2 , p (0) 1 , p (0) 2 ) = ( -0 . 8 , 0 , 0 , 1) . After training, we use the trained integrator to compute the flow starting at u ( N ) for 300 steps.

Ground truth

Proposed method

Itoh-Abesolution

Itoh-Abe half stepsize

Itoh-Abeorder4

Globalerror

0.5

0.5

0.5

0.5

0.5

1.5

error

1.0

0.0

F0'0

0.0

F0°0

F0'0

0.5

0.5

-0.5

0.5

0.5

0.5

0.0

0.5

0.0

0.5

0.5

0.0

0.5

0.5

0.0

0.5

0.5

0.0

0.5

0.5

0.0

0.5

0

10

20

30

q1

q1

q1

q1

q1

t

Figure 5 compares the trajectories predicted by the proposed method with Itoh-Abe methods. While the Itoh-Abe methods yield solutions that generally follow the elliptical shape of the ground-truth, they exhibit noticeable deviations from the true trajectory. For details, see Appendix D.4.

Double Pendulum The double pendulum is known to exhibit complex behaviors with a strong sensitivity to initial conditions, which makes the computation very challenging.

q1

q2

This system has a non-polynomial Hamiltonian as follows:

Figure 6: 2pendulum The training flow data { ( u ( n -1) , u ( n ) ) | n = 1 , ..., N } consists of N = 100 pairs with a fixed time step h = 0 . 3 , starting from u (0) = ( q (0) 1 , q (0) 2 , p (0) 1 , p (0) 2 ) = ( 1 6 π, 1 8 π, 0 , 0) . After training, we use the trained integrator to compute the flow starting at u ( N ) for 300 steps.

Although the chaotic nature of the double pendulum makes learning difficult, the proposed method provides a solution that is close to the true trajectory (Figure 7). For more details, see Appendix D.4.

5.3 AVF Discrete-Gradient-Based Method for Dissipative Nonlinear System

The discrete gradient methods can also be applied to dissipative systems

0.75

Ground truth

0.75

Proposed method

0.75

Itoh-Abesolution

0.75

Itoh-Abe half stepsize

0.75

Itoh-Abe order4

Globalerror

0.50

0.50

0.50

0.50

0.50

0.4

0.25

0.25

0.00

0.25

0.25

0.00

0.25

0.25

0.00

0.25

0.25

0.00

0.25

0.25

0.00

0.2

0.3

0.50

0.50

0.50

0.50

0.50

0.1

0.75

0.5

0.0

0.5

0.75

0.5

0.0

0.5

0.75

0.5

0.0

0.5

0.75

0.5

0.0

0.5

0.75

0.5

0.0

0.5

0.0

0

25

50

75

q1

q1

q1

q1

q1

t

to construct energy-dissipating integrators, where δ denotes the dissipative coefficient. Therefore, our proposed approach can be employed to optimize them.

Dissipative Pendulum The Hamiltonian for the dissipative pendulum is the same as that for the pendulum H ( q, p ) = 1 2 p 2 -cos( q ) .

The dissipation coefficient was set to δ = 0 . 01 . We train the network with the flow data { ( u ( n -1) , u ( n ) ) | n = 1 , ..., N } consists of N = 100 pairs with a fixed time step h = 0 . 3 ,

Globalerror

Energy

0.15

AVF

Proposed method

0.05

Proposed method

Ground truth

0.10

AVF half stepsize

0.00

AVF

0.05

0.05

0.10

0.15

0.00

0

5

10

15

20

25

30

0

5

10

15

20

25

30

starting from u (0) = ( q (0) , p (0) ) = (2 , 0) . After training, the trained integrator was used to compute the flow starting at u ( N ) for 100 steps. As seen in Figure 8, the accuracy of the integrator was greatly improved again by the proposed method while preserving the dissipation property.

Duffing Oscillator The Duffing equation is a non-linear differential equation that models certain damped oscillators with the Hamiltonian:

where we set the linear stiffness coefficient to α = 1 , the non-linearity coefficient to β = -5 and the damping coefficient to δ = 0 . 05 . We train the flow data { ( u ( n -1) , u ( n ) ) | n = 1 , ..., N } consists of N = 200 pairs with a fixed time step h = 0 . 1 , starting from ( q (0) , p (0) ) = (4 , 0) . After training, we computed the flow starting at u ( N ) for 300 steps.

Figure 9 shows that the proposed method again improves the accuracy; however, compared to energy-conserving systems, the numerical error is slightly larger. This is because the discrete gradient method only guarantees the energy dissipation property but does not ensure the accuracy of energy for energy-dissipative systems.

Globalerror

Energy

Proposed method

2

Ground truth

1.5

AVF

Proposed method

AVF half stepsize

0

AVF

1.0

AVF 4order

0.5

0.0

0

5

10

15

20

25

30

0

5

10

15

20

25

30

5.4 AVF Discrete-Gradient-Based Method for Energy-Conservative Nonlinear PDEs

As the discrete gradient method is applicable to semi-discretized PDEs, our approach is also applicable. We applied the proposed method to a semi-discretized Hamiltonian PDE, the Korteweg-de Vries (KdV) equation:

The semi-discrete Hamiltonian is defined as H ( u ) = ∑ j ( 1 2(∆ x ) 2 ( u j +1 -u j ) 2 -u 3 j ) , where the index j denotes spatial discretization points at a fixed time level with the spatial step size ∆ x .

This equation is a partial differential equation that describes shallow water waves. We consider this equation on the interval [0 , 20] under the periodic boundary condition. We used the training data with the time length of T = 30 and the time step size of ∆ t = 0 . 01 . The spatial step size was ∆ x = 0 . 5 . For testing,

Globalerror

0.006

0.004

Proposed method

AVF

0.002

0.000

0

10

20

30

t

the computation was carried out from the time length T = 30 , with the starting being at the final state of the training dataset.

The numerical errors shown in Figure 10 confirm that the proposed method yields more accurate solutions compared to the AVF method. For more details, see Appendix D.4.

5.5 Supplementary Experiments

Impact of Training Data We also conducted comparative experiments to investigate how varying the size of irregularly training data influences the performance of the learned integrator. The experimental results demonstrate that increasing the amount of training data is beneficial for improving the training performance of the learned integrator. See Appendix D.2 for details

Comparison with Other Methods For completeness of exposition, we compare our approach with other methods. Our method focuses on learning structure-preserving numerical integrators, whereas many existing approaches (e.g., PINNs) aim to directly approximate continuous solutions. Because these objectives differ, such comparisons are inherently difficult and should be interpreted with care. We therefore report supplementary results with PINNs and TRS-ODEN on standard dynamical systems. The protocol and complete results are given in Appendix D.5. Overall, our method shows smaller errors and more stable long-horizon trajectories.

Computational Time One limitation of the method is the increased computational time associated with neural networks during training and inference. In particular, the proposed method requires time for data preparation and network training. To see the actual increase in total computation time, we conducted additional numerical experiments. Specifically, we discuss the computation time in detail, especially for the case of the KdV equation, which is the largest system in the experiments. For other experiments, see Appendix D.6. Also, GPUs can be used for training to accelerate calculations. In the experiment, we used an Intel Xeon 6900P CPU and an AMD Instinct MI300A GPU for computation.

Wefirst set the number of nodes for spatial discretization of the KdV equation to 100. The computation time for generating the training data was 28 . 709 sec using a CPU and the scipy odeint function. The training time was 6 . 407 sec using a MI300A GPU. The integration time during testing was 52 . 738 (proposed), 22 . 785 (2nd order AVF) and 127 . 280 (4th order A VF). Thus the total computational time of the proposed method was 28 . 709 + 6 . 407 + 52 . 738 = 87 . 854 , which is less than the 127 . 280 sec required by the 4th order AVF method. Similarly, if we set the number of the nodes to 120, the computational time for each method was 50 . 957 (data generation) + 8 . 311 (training) + 76 . 460 (integration) = 135 . 728 (proposed), 33 . 034 (2nd order AVF), 200 . 003 (4th order AVF). The total computational time of the proposed method was again shorter than that of the 4th-order A VF method. Because our proposed approach typically outperforms the 4th-order AVF method, the proposed approach can compute more accurate numerical solutions in a shorter amount of time, even when the training time and the time for generating data are included.

A closer look at the execution time of the above experiments shows that the computation of the neural network itself is actually not so large. Rather, the matrix-vector multiplication of the skew-symmetric matrix affects the computational complexity. However, the effect of this matrix multiplication does not seem to be so significant, since the multiplication with the skew-symmetric or negative-definite matrix placed in front of the gradient of the energy takes about the same amount of computation. It is also possible to reduce computational cost by low-rank approximation of the learned matrices. In addition, our method does not necessarily need to be used at all time steps. By using it only when higher computational accuracy is desired, computational time can be further reduced.

6 Conclusion

In this work, we proposed a method for learning numerical integrators that preserve the energy conservation or dissipation laws with universal approximation properties. We validated the proposed approach across a diverse set of systems, including chaotic nonlinear Hamiltonian systems, dissipative systems, and a nonlinear partial differential equation. In all scenarios, the method exhibited advantages, particularly in complex or chaotic regimes. With further optimization, we believe this method will play an important role in application areas in the future.

Acknowledgments and Disclosure of Funding

Funding in direct support of this work: JST CREST Grant Number JPMJCR1914 and JPMJCR24Q5, JSPS KAKENHI Grant Number 25K15148, JST ASPIRE JPMJAP2329 and NIFS Collaborative Research NIFS25KISC015. This work is also supported by the Horizon Europe, MSCA-SE project 101131557 (REMODEL).

References

Angelos A Anastassi. Constructing Runge-Kutta methods with the use of artificial neural networks. Neural Computing and Applications , 25:229-236, 2014.

Yohai Bar-Sinai, Stephan Hoyer, Jason Hickey, and Michael P Brenner. Learning data-driven discretizations for partial differential equations. Proc. Natl. Acad. Sci. U. S. A. , 116(31): 15344-15349, July 2019.

Marin Biloš, Johanna Sommer, Syama Sundar Rangapuram, Tim Januschowski, and Stephan Günnemann. Neural flows: Efficient alternative to neural ODEs. Advances in neural information processing systems , 34:21325-21337, 2021.

C. J. Budd and M. D. Piggott. Geometric integration and its applications, 2003. ISSN 15708659.

Elena Celledoni, Volker Grimm, Robert I McLachlan, David I McLaren, Dion O'Neale, Brynjulf Owren, and GRW Quispel. Preserving energy resp. dissipation in numerical PDEs using the 'Average Vector Field' method. Journal of Computational Physics , 231(20):6770-6789, 2012.

Philippe Chartier, Erwan Faou, and Ander Murua. An algebraic approach to invariant preserving integators: the case of quadratic and Hamiltonian invariants. Numerische Mathematik , 103: 575-590, 2006.

Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. Advances in neural information processing systems , 31, 2018.

Zhengdao Chen, Jianyu Zhang, Martin Arjovsky, and Léon Bottou. Symplectic recurrent neural networks, 2020. URL https://arxiv.org/abs/1909.13334 .

Miles Cranmer, Sam Greydanus, Stephan Hoyer, Peter Battaglia, David Spergel, and Shirley Ho. Lagrangian neural networks. arXiv preprint arXiv:2003.04630 , 2020.

Morten Dahlby, Brynjulf Owren, and Takaharu Yaguchi. Preserving multiple first integrals by discrete gradients. Journal of Physics A: Mathematical and Theoretical , 44(30):305205, 2011.

Marco David and Florian Méhats. Symplectic learning for Hamiltonian neural networks. Journal of Computational Physics , 494:112495, 2023.

MDehghanpour, A Rahati, and E Dehghanian. ANN-based modeling of third order Runge Kutta method. J. Adv. Comput. Sci. Technol. , 4(1):180, April 2015.

Emilien Dupont, Arnaud Doucet, and Yee Whye Teh. Augmented neural ODEs. Advances in neural information processing systems , 32, 2019.

Edda Eich. Convergence results for a coordinate projection method applied to mechanical systems with algebraic constraints. SIAM Journal on Numerical Analysis , 30(5):1467-1482, 1993.

Sølve Eidnes. Order theory for discrete gradient methods. BIT , 62(4):1207-1255, December 2022.

Kang Feng and Mengzhao Qin. Symplectic Geometric Algorithms for Hamiltonian Systems . Springer-Verlag, Berlin, 2010. doi: 10.1007/978-3-642-01777-3.

Daisuke Furihata. Finite Difference Schemes for ∂u/∂t = ( ∂/∂x ) α δG/δu That Inherit Energy Conservation or Dissipation Property. Journal of Computational Physics , 156(1):181-205, nov 1999. ISSN 00219991. doi: 10.1006/jcph.1999.6377.

Daisuke Furihata and Takayasu Matsuo. Discrete Variational Derivative Method: A StructurePreserving Numerical Method for Partial Differential Equations . Chapman and Hall/CRC, dec 2010. ISBN 9780429148460. doi: 10.1201/b10387.

O. Gonzalez. Time integration and discrete Hamiltonian systems. Journal of Nonlinear Science , 6, 1996. ISSN 09388974. doi: 10.1007/BF02440162.

Samuel Greydanus, Misko Dzamba, and Jason Yosinski. Hamiltonian neural networks. Advances in neural information processing systems , 32, 2019.

Yue Guo, Felix Dietrich, Tom Bertalan, Danimir T Doncevic, Manuel Dahmen, Ioannis G Kevrekidis, and Qianxiao Li. Personalized algorithm generation: A case study in learning ODE integrators. arXiv [math.NA] , May 2021.

Ernst Hairer. Symmetric projection methods for differential equations on manifolds. BIT Numerical Mathematics , 40:726-734, 2000.

Ernst Hairer. Geometric integration of ordinary differential equations on manifolds. BIT Numerical Mathematics , 41:996-1007, 2001.

Ernst Hairer, Christian Lubich, and Gerhard Wanner. Geometric numerical integration , volume 31 of Springer Series in Computational Mathematics . Springer-Verlag, Berlin, second edition, 2006. ISBN 3-540-30663-3; 978-3-540-30663-4. Structure-preserving algorithms for ordinary differential equations.

Chen-Di Han, Bryan Glaz, Mulugeta Haile, and Ying-Cheng Lai. Adaptable Hamiltonian neural networks. Physical Review Research , 3(2):023156, 2021.

Quercus Hernández, Alberto Badías, David González, Francisco Chinesta, and Elías Cueto. Structure-preserving neural networks. Journal of Computational Physics , 426:109950, 2021.

Quercus Hernández, Alberto Badías, Francisco Chinesta, and Elías Cueto. Thermodynamicsinformed graph neural networks. IEEE Transactions on Artificial Intelligence , 5(3):967-976, 2022.

In Huh, Eunho Yang, Sung Ju Hwang, and Jinwoo Shin. Time-reversal symmetric ODE network. Advances in Neural Information Processing Systems , 33:19016-19027, 2020.

Toshiaki Itoh and Kanji Abe. Hamiltonian-conserving discrete canonical equations based on variational difference quotients. Journal of Computational Physics , 76, 1988. ISSN 10902716. doi: 10.1016/0021-9991(88)90132-5.

Ameya D Jagtap and George Em Karniadakis. Extended physics-informed neural networks (XPINNs): A generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations. Communications in Computational Physics , 28(5), 2020.

Pengzhan Jin, Zhen Zhang, Aiqing Zhu, Yifa Tang, and George Em Karniadakis. SympNets: Intrinsic structure-preserving symplectic networks for identifying Hamiltonian systems. Neural Networks , 132, 2020. ISSN 18792782. doi: 10.1016/j.neunet.2020.08.017.

Ehsan Kharazmi, Zhongqiang Zhang, and George Em Karniadakis. Variational physics-informed neural networks for solving partial differential equations. arXiv preprint arXiv:1912.00873 , 2019.

Diederik P. Kingma and Jimmy Lei Ba. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings , 2015.

J. Nathan Kutz, Steven L. Brunton, Bingni W. Brunton, and Joshua L. Proctor. Dynamic Mode Decomposition . Society for Industrial and Applied Mathematics, Philadelphia, PA, 2016. doi: 10.1137/1.9781611974508. URL https://epubs.siam.org/doi/abs/10.1137/1. 9781611974508 .

Robert A LaBudde and Donald Greenspan. Discrete mechanics-a general treatment. Journal of Computational Physics , 15(2):134-167, 1974.

Robert A LaBudde and Donald Greenspan. Energy and momentum conserving methods of arbitrary order for the numerical integration of equations of motion: I. motion of a single particle. Numerische Mathematik , 25:323-346, 1975.

Robert A LaBudde and Donald Greenspan. Energy and momentum conserving methods of arbitrary order for the numerical integration of equations of motion: II. motion of a system of particles. Numerische Mathematik , 26:1-16, 1976.

Haolin Li, Yuyang Miao, Zahra Sharif Khodaei, and MH Aliabadi. An architectural analysis of DeepOnet and a general extension of the physics-informed DeepOnet model on solving nonlinear parametric partial differential equations. Neurocomputing , 611:128675, 2025.

Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. In ICLR 2021 - 9th International Conference on Learning Representations , 2021.

Zongyi Li, Daniel Zhengyu Huang, Burigede Liu, and Anima Anandkumar. Fourier neural operator with learned deformations for PDEs on general geometries. Journal of Machine Learning Research , 24(388):1-26, 2023.

Zongyi Li, Hongkai Zheng, Nikola Kovachki, David Jin, Haoxuan Chen, Burigede Liu, Kamyar Azizzadenesheli, and Anima Anandkumar. Physics-informed neural operator for learning partial differential equations. ACM/JMS Journal of Data Science , 1(3):1-27, 2024.

Lu Lu, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, and George Em Karniadakis. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nature Machine Intelligence , 3, 2021. ISSN 25225839. doi: 10.1038/s42256-021-00302-5.

Bethany Lusch, J Nathan Kutz, and Steven L Brunton. Deep learning for universal linear embeddings of nonlinear dynamics. Nature communications , 9(1):4950, 2018.

Suryanarayana Maddu, Dominik Sturm, Bevan L Cheeseman, Christian L Müller, and Ivo F Sbalzarini. STENCIL-NET: Data-driven solution-adaptive discretization of partial differential equations. arXiv [math.NA] , January 2021.

Marios Mattheakis, David Sondak, Akshunna S Dogra, and Pavlos Protopapas. Hamiltonian neural networks for solving equations of motion. Physical Review E , 105(6):065305, 2022.

Robert I. McLachlan and G. R.W. Quispel. Discrete gradient methods have an energy conservation law. Discrete and Continuous Dynamical Systems- Series A , 34, 2014. ISSN 10780947. doi: 10.3934/dcds.2014.34.1099.

Robert I McLachlan, G R W Quispel, and Nicolas Robidoux. Geometric integration using discrete gradients. Philos. Trans. A Math. Phys. Eng. Sci. , 357(1754):1021-1045, April 1999.

Yuto Miyatake and John C Butcher. A characterization of energy-preserving methods and the construction of parallel integrators for Hamiltonian systems. SIAM Journal on Numerical Analysis , 54(3):1993-2013, 2016.

Ben Moseley, Andrew Markham, and Tarje Nissen-Meyer. Finite basis physics-informed neural networks (FBPINNs): a scalable domain decomposition approach for solving differential equations. Advances in Computational Mathematics , 49(4):62, 2023.

Richard A. Norton, David I. McLaren, G. R.W. Quispel, Ari Stern, and Antonella Zanna. Projection methods and discrete gradient methods for preserving first integrals of ODEs. Discrete and Continuous Dynamical Systems- Series A , 35, 2015. ISSN 15535231. doi: 10.3934/dcds. 2015.35.2079.

Christian Offen and Sina Ober-Blöbaum. Symplectic integration of learned Hamiltonian systems. Chaos: An Interdisciplinary Journal of Nonlinear Science , 32(1), 2022.

G. R.W. Quispel and H. W. Capel. Solving ODEs numerically while preserving a first integral. Physics Letters, Section A: General, Atomic and Solid State Physics , 218(3-6):223-228, 1996. ISSN 03759601. doi: 10.1016/0375-9601(96)00403-3.

G. R.W. Quispel and D. I. McLaren. A new class of energy-preserving numerical integration methods. Journal of Physics A: Mathematical and Theoretical , 41, 2008. ISSN 17518113. doi: 10.1088/1751-8113/41/4/045206.

M. Raissi, P. Perdikaris, and G. E. Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics , 378, 2019. ISSN 10902716. doi: 10.1016/j.jcp.2018.10.045.

Rishikesh Ranade, Chris Hill, and Jay Pathak. DiscretizationNet: A machine-learning based solver for Navier-Stokes equations using finite volume discretization. Comput. Methods Appl. Mech. Eng. , 378(113722):113722, May 2021.

Jesus M Sanz-Serna. Symplectic integrators for Hamiltonian problems: an overview. Acta numerica , 1:243-286, 1992.

Jesus M Sanz-Serna and Mari-Paz Calvo. Numerical Hamiltonian problems , volume 7. Courier Dover Publications, 2018.

Khemraj Shukla, Ameya D Jagtap, and George Em Karniadakis. Parallel physics-informed neural networks via domain decomposition. Journal of Computational Physics , 447:110683, 2021.

Naoya Takeishi, Yoshinobu Kawahara, and Takehisa Yairi. Learning Koopman invariant subspaces for dynamic mode decomposition. Advances in neural information processing systems , 30, 2017.

Paulo M Tasinaffo, Luiz A V Dias, and Adilson M da Cunha. A survey about universal numerical integrators (UNIs): Part II or quantitative approach. Hum-Cent Intell Syst , November 2024.

Paulo M Tasinaffo, Luiz A V Dias, and Adilson M da Cunha. A qualitative approach to universal numerical integrators (UNIs) with computational application. Hum-Cent Intell Syst , 4 (4):571-598, November 2024.

Yunjin Tong, Shiying Xiong, Xingzhe He, Guanghan Pan, and Bo Zhu. Symplectic neural networks in Taylor series form for Hamiltonian systems. Journal of Computational Physics , 437:110325, 2021.

Enoch Yeung, Soumya Kundu, and Nathan Hodas. Learning deep neural network representations for Koopman operators of nonlinear dynamical systems. In 2019 American Control Conference (ACC) , pages 4832-4839. IEEE, 2019.

Huaiqian You, Quinn Zhang, Colton J Ross, Chung-Hao Lee, and Yue Yu. Learning deep implicit Fourier neural operators (IFNOs) with applications to heterogeneous material modeling. Computer Methods in Applied Mechanics and Engineering , 398:115296, 2022.

Chengxi Zang and Fei Wang. Neural dynamics on complex networks. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining , pages 892-902, 2020.

Zhen Zhang, Yeonjong Shin, and George Em Karniadakis. GFINNs: GENERIC formalism informed neural networks for deterministic and stochastic dynamical systems. Philosophical Transactions of the Royal Society A , 380(2229):20210207, 2022.

G Zhong. Lie-Poisson integrators and Lie-Poisson Hamilton-Jacobi theory. Phys. Lett. A , 133: 134-139, 1988.

A Proofs

Proof of Theorem 3.1. By using the chain rule, it holds that

{

H

Substitution of the equation yields

d

H

d

t

=

∇

H

S

∇

= 0

(

if

S

is skew symmetric

0

(

if

S

O

.

)

Proof of Theorem 4.2. The proof is an discrete analogue of the proof of Theorem 3.1. By using the discrete chain rule, it holds that

Substitution of the discrete gradient scheme yields

Proof of Theorem 4.3. First, note that if a vector w is perpendicular to another vector v -u , then there exists a skew symmetric matrix A such that w = A ( v -u ) . Hence, the proof is completed if we show that all discrete gradient operators have the following representation:

Let ¯ ∇ H ( u, v ) be of the form (8) for some w ( u, v ) , and we check that it satisfies (5) and (6).

By definition, ¯ ∇ H ( u, u ) = ¯ ∇ 0 H ( u, u ) + w ( u, u ) = ∇ H ( u ) . Conversely, if ¯ ∇ H ( u, v ) is a discrete gradient, then define w ( u, v ) = ¯ ∇ H ( u, v ) -¯ ∇ 0 H ( u, v ) which is continuous. By (7), w ( u, v ) ⊥ v -u , and w ( u, u ) = ¯ ∇ H ( u, u ) -¯ ∇ 0 H ( u, u ) = 0 by (6) since both of the discrete gradients are consistent.

Proof of Theorem 4.4. Suppose that a numerical scheme

Let u ( n +1) and u ( n ) be solutions to (11). It is sufficient to show that f can be written as

with a reference discrete gradient ¯ ∇ 0 H ( u, v ) of H .

From the property of the discrete gradient and the energy-conservation property, we get

Hence f must be perpendicular to the discrete gradient ¯ ∇ 0 H . Because we assume that the target system is a Hamiltonian equation, the matrix S is invertible. Therefore, we have

preserves the energy

⊤

)

Thus, S -1 f is in the orthogonal complement M of S ¯ ∇ 0 H . Because S is skew-symmetric, ¯ ∇ 0 H is perpendicular to S ¯ ∇ 0 H , which means that ¯ ∇ 0 H is in the orthogonal complement M . Therefore, we can obtain an orthogonal basis { e 1 , e 2 , . . . , e N } of M , where e 1 = ¯ ∇ 0 H and N = dim M . By using this basis, S -1 f can be written as

where c 1 ∈ R and e ∈ span { e 2 , ..., e N } . We want to show that e is perpendicular to u ( n +1) -u ( n ) = hf ( u ( n +1) , u ( n ) ) . The inner product of e and f is

because S -1 is skew-symmetric, ⟨ f, f ⟩ = 0 . Hence ⟨ f, ¯ ∇ 0 H ⟩ = 0 . Thus e is perpendicular to u ( n +1) -u ( n ) , so there exists a skew-symmetric matrix A ( u ( n +1) , u ( n ) ) such that e = A ( u ( n +1) , u ( n ) )( u ( n +1) -u ( n ) ) .

Finally, we show that c 1 = 1 . To this end, we take the limit h → 0 in

Since the scheme is consistent, S -1 f →∇ H as h → 0 . On the other hand, the right-hand side tends to c 1 ∇ H because of the consistency condition (6) of discrete gradients. Hence we get ∇ H = c 1 ∇ H and hence c 1 = 1 .

B Geometric Description of Target Differential Equations

The target equations in this study are characterized as gradient flows on symplectic or Riemannian manifolds. Let ( M , ω ) be a symplectic manifold and ( M , g ) be a Riemannian manifold. For symplectic manifolds, ω is a symplectic 2-form. In Riemannian manifolds, g denotes the inner product. These define a bilinear function ω u : T u M× T u M→ R or g u : T u M× T u M→ R for each u ∈ M . It is assumed that ω and g are nondegenerate in the sense that the matrix M u representing these bilinear functions is nondegenerate for any u ∈ M .

For a given energy function H : M→ R , we can define differential equations

on Riemannian manifolds, where d H : T M→ R is the differential of H . For symplectic manifolds, this equation is the Hamilton equation, and for Riemannian manifolds, it is the gradient flow.

These equations have energy conservation and the energy dissipation laws, respectively. In fact, in Hamiltonian systems, it holds that

because ω u is skew-symmetric due to the property of symplectic forms. Also, for Riemannian manifolds,

By using the relation between the gradient and the derivative

these equations can be rewritten as vector-matrix representations:

for Riemannian manifolds, respectively. These equations are the target equations of this study.

on symplectic manifolds and for symplectic manifolds and

C Target Partial Differential Equations and Its Semi-Discretization

The proposed method in this paper can also be applied to semi-discretized partial differential equations in the spatial direction. Typical examples of the target partial differential equations are equations of the following form

where H is the energy density, which is a function of u , u x , u xx , and so on. u x denotes ∂u ∂x , and u xx denotes ∂ 2 u ∂x 2 . δH/δu is called the variational derivative of H . For example, if H is a function of u and u x , then this is defined by

Equations of the form (12) include many partial differential equations, such as the advection equation, the KdV equation, the Burgers equation, the Allen-Cahn equation, and the Cahn-Hilliard equation. Equations that are slightly different but can be expressed in a similar form include the Maxwell equation, the wave equation, the Klein-Gordon equation, and the nonlinear Schrodinger equation. Although the above equations are defined in one-dimensional space, they can be extended to multiple dimensions in a straightforward way. See, e.g., Furihata and Matsuo [18] for details.

It is known that (12) has the global energy conservation law or the energy dissipation or increasing law under certain boundary conditions, such as the periodic boundary condition. Suppose that the equation is defined on the interval [0 , L ] . For simplicity, we assume the periodic boundary condition. In this case, the time differentiation of the total energy yields

Because we assume the periodic boundary condition, the boundary term must vanish. Therefore, we have

Substituting the definition of the variational derivatives and also the equation into the above equality, we obtain

If α is odd and expressed as α = 2 n +1 with an integer n , then by repeating the integration by parts, we get

Therefore, we have

and hence the energy conservation law follows:

Similarly, when α = 2 n , we have

Thus, the energy decreasing or increasing law follows. Thus, it can be seen that these equations have the same energy behavior as the target ordinary differential equations of this study.

Next, we semi-discretize these partial differential equations only in the spatial direction using the finite difference method while preserving the energy behaviors. We devide the interval [0 , L ] into M equal parts with the spatial step size ∆ x . The approximation of u ( t, j ∆ x ) is denoted by u j ( t ) , and we donote the vector ( u 0 ( t ) , ..., u M ( t )) as ⃗ u . Let D 1 , D + , and D -be difference matrices representing the central, forward, and backward differences, respectively:

First, the energy function can be approximated by approximating u x in the energy function by using the finite difference

We also discretize the differential operator

in the equation. For α = 2 n , we can use n copies of D + and D -to form a matrix

to approximate (13). This matrix is positive or negative semi-definite, since D ⊤ + = -D -. Thus, the semi-discretized equation

is an ordinary differential equation with the energy dissipating or increasing property. Similarly, if α is 2 n +1 , then the equation is semi-discretized into an ordinary differential equation with an energy conservation law:

Note that D + · · · D + D 1 D -· · · D -is skew-symmetric because D 1 is skew-symmetric. The proposed method yields numerical integrators that preserve the energy behavior for each of these semidiscretized equations.

D Supplementary experiments

D.1 hyperparameters

Table 2 shows the hyperparameters for training.

D.2 Experiment loss

Loss under Different Reference Methods with Varying Network Depths and Widths We use the Hénon-Heiles model:

The training flow data { ( u ( n -1) , u ( n ) ) | n = 1 , ..., N } consists of N = 100 pairs of points obtained from the trajectory calculated using a high-order integrator sampled with a time step h = 0 . 3 , starting from u (0) = ( q (0) 1 , q (0) 2 , p (0) 1 , p (0) 2 ) = (0 . 3 , 0 . 3 , -0 . 2 , -0 . 2) . After training, we use the flow starting at u ( N ) for 300 steps as test.

Loss

width

0.000001

10

50

100

—AVF2layer

..AVF2-test

AVF5layer

..AVF5-test

0.0000001

—AVF10layer

..AVF10-test

ITOH2layer

ITOH2-test

SSO

1E-08

ITOH5layer

..ITOH5-test

—ITOH10layer

....ITOH10-test

—SYMITOH2layer

.SYMITOH2-test

1E-09

-SYMITOH5layer

...SYMITOH5-test

—SYMITOH10layer..SYMITOH10-test

1E-10

We conducted experiments using both AVF, Itoh-Abe method and time-symmetric Itoh-Abe method as baseline schemes. The learning rate was set to 0.00001, and the number of epochs was 30,000. The network architecture was varied with 2, 5, and 10 layers, and width of 10, 50, and 100 per layer. For each configuration, we performed five independent training runs and reported the final loss as results, as shown in the Figure 11 .

As expected, the AVF-based method, being a second-order scheme, generally outperformed the Itoh-Abe method.

Based on the above experimental results and observations, we conclude that in the initial selection of the baseline method, the AVF scheme should be prioritized whenever it is computationally feasible, due to its higher accuracy and time symmetry. In cases where the original Hamiltonian system is too complex for the AVF discrete gradient to be computed, the Itoh-Abe method can serve as a practical alternative. Furthermore, it is worth exploring the possibility of constructing a time-symmetric version of the Itoh-Abe method to enhance its accuracy, potentially achieving a second-order scheme.

For this reason, we adopt a time-symmetric construction of Itoh-Abe method to achieve second-order. And the results in the Figure 11 indicate that the time-symmetric Itoh-Abe method outperforms the standard one.

Experiment loss The training loss was computed by the mean-squared-error between u ( n ) train -u ( n -1) train h ( n )

and S ∇ NN H ( u train , u train ) , and the test loss was computed by the mean-squared-error between u ( n ) test -u ( n -1) test h ( n ) test and ¯ S ¯ ∇ NN H ( u ( n ) test , u ( n -1) test ) . Table 3 shows the losses in the form of mean ± standard deviation measured from 5 independent experiments. Note that all computations were performed in double precision. It can be seen that the models were trained very accurately.

train ¯ ¯ ( n ) ( n -1)

Experiment loss with different data and initial conditions We conducted additional experiments on both the Hénon-Heiles and the double pendulum. The aim was to evaluate the impact of training data distribution, data size, and initial conditions on the performance of the learned integrator.

Hénon-Heiles system: We used irregular training data { (( u ( i -1) , h ( i ) ) , u ( i ) ) | i = 1 , ..., I } , where the elements of u ( i -1) are randomly sampled from the domain [ -1 , 1] and each step size h ( i ) is drawn independently from [0 , 0 . 5] . Each u ( i ) represents the one-step solution computed from u ( i -1) using step size h ( i ) with a high-order solver.

We performed experiments using 100 , 1000 , 5000 training samples, respectively. The neural network consists of 10 layers, each with 100 width. We set the learning rate to 0 . 00001 and trained the model for 30000 iterations. For testing, we selected random initial conditions from [ -0 . 4 , 0 . 4] 4 . For each test trajectory, a fixed step size h was randomly chosen from [0 , 0 . 5] and applied over 100 time steps.

Double pendulum: We generated training data as follows. First, we randomly sampled initial angles ( q 1 , q 2) from the interval [ -0 . 8 , 0 . 8] , and set the initial momenta ( p 1 , p 2) to zero. For each initial condition, we randomly chose a fixed step size h from the interval [0 , 0 . 5] , and integrated the system forward for 100 time steps using a high-order solver. This process was repeated to generate 500 trajectories, each of 100 steps. From the resulting collection of one-step pairs, we randomly shuffled and selected 100 , 1000 , 5000 samples to construct training datasets. The neural network consists of 10 layers, each with 100 width. We set the learning rate to 0 . 0001 and trained the model for 30000 iterations.

During testing, we followed the same protocol as in training: each test trajectory starts from a randomly chosen initial condition, with angles in [ -0 . 8 , 0 . 8] and momenta set to zero. A fixed step size h ∈ [0 , 0 . 5] was randomly selected per test and kept constant over 100 steps.

Table 4 shows the average training and test loss with standard deviations for different training dataset sizes. Each result is computed based on 5 independent training runs with different data. For both systems, the training loss consistently converges to a similar small value and the test loss improves as the number of training samples increases.

D.3 Pendulum Experiment

We conduct experiments on the pendulum (Figure 12). The mathematical pendulum (mass m = 1 , string length l = 1 , gravity g = 1 ) is a system with the Hamiltonian H ( q, p ) = 1 2 p 2 -cos( q ) .

The position coordinate q denotes the angle from the vertical suspension point and p = ˙ q denotes the momentum. Although the trigonometric function exists in the Hamiltonian, the integral in the A VF method is analytically computable, and hence the A VF method is available.

In this experiment, we used two sets of data: flow dataset and irregular dataset like the example shown in the Figure 13. The training flow data consists of N = 15 points sampled from a trajectory for each fixed time step h = 0 . 5 , calculated using a high-order integrator starting from u (0) =

( q (0) , p (0) ) = (2 , 0) . The training flow data are pairs of numerical solutions at a certain time step n along with the one at the next time step: { ( u ( n -1) , u ( n ) ) | n = 1 , ..., N } . After training, we use the trained gradient to compute the flow starting at u ( N ) for 200 steps.

The training irregular data { (( u ( i -1) , h ( i ) ) , u ( i ) ) | i = 1 , ..., I } consists of I = 100 pairs of points. u ( i -1) are randomly sampled from [ -3 , 3] × [ -3 , 3] and the time steps h ( i ) are randomly chosen from [0 , 0 . 8] . u ( i ) are numerical solutions at the next step of u ( i -1) with the step size h ( i ) . After training, we use the trained gradient to compute the flow starting at u (0) = (2 , 0) with h = 0 . 5 for 200 steps.

In Figure 14 , we show the additional for the pendulum experiment. The plot in the top-left corner shows the irregular training data. The top-right plot displays the test trajectories of the trained network and numerical integrators during the test. The two middle plots show the values of q and p over time. The bottom-left plot shows the error evolutions over time, while the bottom-right plot illustrates the energy conservation.

In Figure 15 , we show the additional results for the pendulum experiment with the irregular data. The plot in the top-left corner shows the irregular training data. The top-right plot displays the test trajectories of the trained network and numerical integrators during the test. The two middle plots show the values of q and p over time. The bottom-left plot shows the error evolutions over time, while the bottom-right plot illustrates the energy conservation.

D.4 Supplementary figures of the experimental results

Results for the Hénon-Heiles system In Figure 16 , we show the additional results for the HénonHeiles experiment. The plot in the top-left corner shows the positions of the training flow data, while the top-middle displays the values of test q 1 . The plot in the top-right shows the change in error over time, and the center-right shows energy conservation. The remaining plots display the test trajectories of the ground truth, the solution by the A VF method, that by the A VF method with a half step size, 4 order, and the solution by the proposed method.

In Figure 17 , we show the additional results for the Hénon-Heiles experiment with the irregular data. The plot in the top-left corner shows the positions of the randomly selected irregular training data, while the top-middle displays the values of test q 1 . The plot in the top-right shows the change in error over time, and the center-right shows energy conservation. The remaining plots display the test trajectories of the ground truth, the solution by the A VF method, that by the A VF method with a half step size, 4 order, and the solution by the proposed method.

2

1

0

1

2

q

1

0

1

p

Traindata

2

1

0

1

2

q

1

0

1

p

Pendlum

Ground truth

Proposed method

AVF

0

20

40

60

80

100

2

1

0

1

2

q

Ground truth

Proposed method

AVF

AVF half stepsize

0

20

40

60

80

100

1

0

1

p

Ground truth

Proposed method

AVF

AVF half stepsize

0

20

40

60

80

100

0.0

0.2

0.4

0.6

Global error

Proposed method

AVF

AVF half stepsize

0

20

40

60

80

100

0.4158

0.4160

0.4162

0.4164

0.4166

Energy

Ground truth

Proposed method

AVF

3

2

1

0

1

2

3

q

2

0

2

p

Traindata

2

1

0

1

2

q

1

0

1

p

Pendlum irregular

Ground truth

Proposed method

AVF

0

20

40

60

80

100

2

1

0

1

2

q

Ground truth

Proposed method

AVF

AVF half stepsize

0

20

40

60

80

100

1

0

1

p

Ground truth

Proposed method

AVF

AVF half stepsize

0

20

40

60

80

100

0.0

0.2

0.4

0.6

Global error

Proposed method

AVF

AVF half stepsize

0

20

40

60

80

100

0.4158

0.4160

0.4162

0.4164

0.4166

Energy

Ground truth

Proposed method

AVF

Results for 2body In Figure 18 , in addition to the numerical errors and the evolutions of the energy mentioned in the main text, the top-left corner shows the training flow data, while the top-middle display the values of test q 1 . The plot in the center-right shows energy conservation.

Results for Double pendulum In Figure 19 , the plot in the top-left corner shows the positions of the training flow data, while the two plots at the center-right show the evolutions of the errors and the energy over time. The bottom plots display the trajectories of the ground truth, the solution by the AVF method, the solution by the AVF method with a half step size, and the solution by the proposed method.

Results for the dissipative pendulum In Figure 20 , in addition to the numerical errors and the evolutions of the energy mentioned in the main text, the top-left corner shows the training flow data, the top-right corner displays the test trajectory after training, the middle display the values of test q, p , and the two plots at the bottom correspond to the values of the coordinates over time.

Results for the Duffing oscillator In Figure 21 , we show the results for the Duffing oscillator. In addition to the numerical errors and the evolutions of the energy mentioned in the main text, the top-left corner shows the training flow data, the top-right corner displays the test trajectory after training, the middle display the values of test q, p , and the two plots at the bottom correspond to the values of the coordinates over time.

Results for KDV We consider this equation on the interval [0 , L ] , L = 20 under the periodic boundary condition. We used the training data with the time length of T = 30 and the time step size of ∆ t = 0 . 01 . The spatial step size was ∆ x = 0 . 5 . The initial condition of the training data is constructed by superimposing two solitary wave solutions of the KdV equation: u 0 ( x ) = u sol ( x -x 1 , c 1 ) + u sol ( x -x 2 , c 2 ) , where u sol ( x, c ) = 1 2 c · sech 2 ( 1 2 √ c · x ) with c 1 = 0 . 75 , x 1 = 0 . 33 L, c 2 = 0 . 4 , x 1 = 0 . 65 L . The time evolution is computed by solving the KdV equation using the solver with high accuracy, where spatial derivatives were approximated using finite difference methods. For testing, the computation was carried out from the time length T = 30 , with the start point being at the final state of the training dataset.

In Figure 22 , we show the results for the KdV. The plot in the top-left corner shows the positions of the training flow data. The plot in the bottom-left shows the change in error over time, and the bottom-middle shows energy conservation. The remaining plots display the test trajectories of the ground truth, the solution by the A VF method, and the solution by the proposed method.

0.5

0.0

0.5

0.4

0.2

0.0

0.2

0.4

Traindata

0

25

50

75

1.0

0.5

0.0

0.5

1.0

q1

0

25

50

75

0.00

0.25

0.50

0.75

1.00

1.25

Global error

0

25

50

75

0.14790

0.14795

0.14800

0.14805

0.14810

Energy

Ground truth

Proposed method

AVF

0.5

0.0

0.5

q1

0.50

0.25

0.00

0.25

0.50

0.75

q2

Ground truth

0.5

0.0

0.5

q1

0.50

0.25

0.00

0.25

0.50

0.75

q2

Proposed method

0.5

0.0

0.5

q1

0.50

0.25

0.00

0.25

0.50

0.75

q2

AVF solution

0.5

0.0

0.5

q1

0.50

0.25

0.00

0.25

0.50

0.75

q2

AVF half stepsize

0.5

0.0

0.5

q1

0.50

0.25

0.00

0.25

0.50

0.75

q2

.

1

0

1

1.0

0.5

0.0

0.5

1.0

Traindata

0

10

20

30

1.0

0.5

0.0

0.5

1.0

q1

0

10

20

30

0.0

0.1

0.2

0.3

Global error

0

10

20

30

0.14790

0.14795

0.14800

0.14805

0.14810

Energy

Ground truth

Proposed method

AVF

0.5

0.0

0.5

q1

0.50

0.25

0.00

0.25

0.50

0.75

q2

Ground truth

0.5

0.0

0.5

q1

0.50

0.25

0.00

0.25

0.50

0.75

q2

Predicted solution

0.5

0.0

0.5

q1

0.50

0.25

0.00

0.25

0.50

0.75

q2

AVF solution

0.5

0.0

0.5

q1

0.50

0.25

0.00

0.25

0.50

0.75

q2

AVF half stepsize

0.5

0.0

0.5

q1

0.50

0.25

0.00

0.25

0.50

0.75

q2

0.5

0.0

0.5

0.50

0.25

0.00

0.25

0.50

Traindata

0

10

20

30

1.0

0.5

0.0

0.5

1.0

q1

0

10

20

30

0.0

0.5

1.0

1.5

Global error

0

10

20

30

0.75005

0.75000

0.74995

0.74990

Energy

Ground truth

Proposed method

Itoh-Abe

0.5

0.0

0.5

q1

0.5

0.0

0.5

q2

Ground truth

0.5

0.0

0.5

q1

0.5

0.0

0.5

q2

Proposed method

0.5

0.0

0.5

q1

0.5

0.0

0.5

q2

Itoh-Abe solution

0.5

0.0

0.5

q1

0.5

0.0

0.5

q2

Itoh-Abe half stepsize

0.5

0.0

0.5

q1

0.5

0.0

0.5

q2

Itoh-Abe order4

0.5

0.0

0.5

0.5

0.0

0.5

Traindata

0

25

50

75

1.0

0.5

0.0

0.5

1.0

q1

0

25

50

75

0.0

0.1

0.2

0.3

0.4

Global error

0

25

50

75

4.0

3.5

3.0

2.5

1e

5

2.6559

Energy

Ground truth

Proposed method

Itoh-Abe

0.5

0.0

0.5

q1

0.75

0.50

0.25

0.00

0.25

0.50

0.75

q2

Ground truth

0.5

0.0

0.5

q1

0.75

0.50

0.25

0.00

0.25

0.50

0.75

q2

Proposed method

0.5

0.0

0.5

q1

0.75

0.50

0.25

0.00

0.25

0.50

0.75

q2

Itoh-Abe solution

0.5

0.0

0.5

q1

0.75

0.50

0.25

0.00

0.25

0.50

0.75

q2

Itoh-Abe half stepsize

0.5

0.0

0.5

q1

0.75

0.50

0.25

0.00

0.25

0.50

0.75

q2

Itoh-Abe order4

2

1

0

1

2

q

1

0

1

p

Traindata

1

0

1

q

1

0

1

p

Dissipative Pendlum

Ground truth

Proposed method

AVF

0

5

10

15

20

25

30

1

0

1

q

Ground truth

Proposed method

AVF

AVF half stepsize

0

5

10

15

20

25

30

1

0

1

p

Ground truth

Proposed method

AVF

AVF half stepsize

0

5

10

15

20

25

30

0.00

0.05

0.10

0.15

Global error

Proposed method

AVF

AVF half stepsize

0

5

10

15

20

25

30

0.15

0.10

0.05

0.00

0.05

Energy

Ground truth

Proposed method

AVF

4

2

0

2

4

q

5

0

5

p

Traindata

2

0

2

q

4

2

0

2

4

p

Test

Ground truth

Proposed method

AVF

0

5

10

15

20

25

30

2

0

2

q

Ground truth

Proposed method

AVF

AVF_half

AVF_4order

0

5

10

15

20

25

30

4

2

0

2

4

p

Ground truth

Proposed method

AVF

AVF_half

AVF 4order

0

5

10

15

20

25

30

0.0

0.5

1.0

1.5

Global error

Proposed method

AVF

AVF half stepsize

AVF 4order

0

5

10

15

20

25

30

4

2

0

2

Energy

Ground truth

Proposed method

AVF

0

10

20

x

0

5

10

15

20

25

30

t

Traindata

0.0

0.1

0.2

0.3

0.4

0

10

20

x

0

5

10

15

20

25

30

t

Ground truth

0.0

0.1

0.2

0.3

0.4

0

10

20

x

0

5

10

15

20

25

30

t

Predicted solution

0.0

0.1

0.2

0.3

0.4

0

10

20

x

0

5

10

15

20

25

30

t

AVF solution

0.0

0.1

0.2

0.3

0.4

0

10

20

30

0.000

0.002

0.004

0.006

Global error

Proposed method

AVF

0

10

20

30

0.3175

0.3170

0.3165

0.3160

Energy

Proposed method

AVF

D.5 Comparison with other methods

We implemented the PINNs as in [54] with energy conservation/dissipation regularization terms, using the DeepXDE library, and applied it to three benchmark systems: Hénon-Heiles, double pendulum, and Duffing oscillator. Both our method and the PINNs use the same network architecture ( 10 layers, width 100 ), optimizer (Adam, learning rate 0 . 0001 ), and 30000 training iterations. We sampled 2000 random collocation points over the spatiotemporal domain.

We also implemented the Time-Reversal Symmetric ODE Network (TRS-ODEN) [28]. For the time-reverse symmetry loss term, we conducted experiments with three different weight values: λ = 10 , 1 , 0 . 5 , which are employed in the reference paper. The network architecture ( 10 layers, width 100 ), optimizer (Adam, learning rate 0 . 0001 ), and 30000 training iterations are kept the same.

To evaluate the performance, we generated a ground-truth reference trajectory for each system using a high-order solver with 100 time steps, starting from the same initial conditions. We then simulated trajectories using both the trained PINNs, TRS-ODEN and our proposed method under the same initial condition. We report the maximum error over the time domain [0 , T ] , defined as the maximum deviation from the ground-truth trajectory. Furthermore, since PINNs tends to degrade over long time horizons, we computed the maximum error accumulated up to each checkpoint ( T = 5 , 10 , 50 , 100 ), i.e., the largest deviation from the reference trajectory from step 0 up to the given time step T .

Hénon-Heiles System We set the initial condition as u (0) = ( q (0) 1 , q (0) 2 , p (0) 1 , p (0) 2 ) = (0 . 3 , 0 . 3 , -0 . 2 , -0 . 2) , and performed simulations over 100 steps with a fixed time step size h = 0 . 3 .

Double Pendulum We set the initial condition as u (0) = ( q (0) 1 , q (0) 2 , p (0) 1 , p (0) 2 ) = ( π/ 6 , π/, 0 , 0) , and performed simulations over 100 steps with a fixed time step size h = 0 . 3 .

Duffing Oscillator We set the initial condition as u (0) = ( q (0) , p (0) ) = (2 . 95 , -3 . 08) and performed simulations over 100 steps with a fixed time step size h = 0 . 1 .

The maximum errors of both models, measured against the reference solution, are summarized in table 5.

These results in all three systems suggest that our structure-preserving method offers improved accuracy and stability over the other two methods, especially in long-term simulations. This improvement is attributed to its formulation as a numerical integrator designed to respect the system's physical structure.

D.6 Computational Time

We performed additional experiments to re-evaluate the computation time across different systems. In the experiments, we used an Intel Xeon 6900P CPU.

The results, the size of the neural networks used and the number of time steps in the numerical calculations are shown in Table 6. The computation time is generally similar to that of the HénonHeiles problem reported in the main body. In particular, the proposed method achieves better accuracy with less computational time than the fourth-order methods. For the KdV equation experiment, the network was made as small as possible without sacrificing accuracy. The network used is quite small, yet the test loss was 1 . 706 × 10 -6 , which is not much different from the test loss of 1 . 695 × 10 -6 when a larger network with 5 layers with width of 200 is used. For other experiments, the computation time can be further reduced by making the network smaller.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The main claims in the abstract and introduction clearly reflect the key contributions and findings of the paper, and are well supported by both the theoretical analysis and experimental results.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: One limitation of the method lies in the increased computational cost due to the use of neural networks.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: The assumptions required for our theoretical results are described in the main paper, and the proofs are provided in the appendix.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We provide detailed experimental settings and the code.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We have provided a ZIP file containing the experimental code.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We provide detailed experimental settings and the code.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We provide detailed experimental losses in the form of mean ± standard deviation measured from 5 independent experiments in the appendix.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We include a description of the hardware.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We adhere to the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: There is no societal impact of the work performed.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We used the official code of SympNet. Although the information about the license is not available online, we properly acknowledged the code.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: The paper does not introduce any new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [NA]

Justification: The core method development in this research does not involve LLMs as any important, original, or non-standard components.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel method that uses a neural network to learn highly accurate numerical solvers for physical systems. It builds upon discrete gradient methods, which inherently guarantee the conservation or dissipation of energy. The key innovation is using a neural network to optimize a corrective term within this physically-consistent framework. This approach ensures the learned solver respects fundamental physical laws by design, while training on data makes it significantly more accurate. Experiments show it excels in long-term simulations of complex and chaotic systems, outperforming standard geometric integrators.\n\nSTRENGTHS AND WEAKNESSES:\nPros:\n\n1. The paper is well-written in general.\n\n2. The proposed UEPI is technically sound, with detailed derivation towards its guaranteed physical properties.\n\n3. The experiments covers a diverse set of physical systems, showing the expressive power of the proposed approach.\n\n\nCons:\n\n1. There are some PINN work that also studies energy-conservative/disppastive systems, such as [1,2]. The authors only compared with  AVF, Itoh-Abe method and time-symmetric Itoh-Abe method. A lot of PINN methods should also be discussed and considered. \n\n2. Using a neural network at each step of the integration process is more computationally expensive than traditional methods. While the paper argues this cost is justified by the significant accuracy gains, it remains a notable overhead, especially for high-dimensional systems. Is there any comprehensive time comparison analysis regarding the computational overhead, in terms of prediction length, complexity of physical systems, etc? Have the authors study scheduling policy that only applys the neural network at certain integration steps, similar to [3]?\n\n[1] Physics-Informed Regularization for Domain-Agnostic Dynamical System Modeling.\n\n[2] Time-Reversal Symmetric ODE Network.\n\n[3] PhysDiff: Physics-Guided Human Motion Diffusion Model\n\nQUESTIONS:\nSee above. I am happy to raise my score if my questions are well-addressed in the weakness section.\n\nLIMITATIONS:\nSee above", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes to learn an energy-preserving numerical integrator for energy-based dynamical systems (e.g., Hamiltonian mechanics or dissipative systems), building upon the the discrete gradient method. The key idea is to augment the discrete gradient with a learnable skew-symmetric term, modeled by a neural network. The authors claim that the proposed method generalizes the traditional discrete gradient method and is theoretically capable of universally approximating energy-preserving integrators. The proposed method is evaluated on various dynamical systems and outperforms conventional methods such as the AVF and Ito-Abe integrators.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n- The proposed method appears to work well empirically, as the authors validate their approach across a range of physical systems, including the Henon–Heiles ODE, the two-body problem, the double pendulum, the dissipative pendulum, the Duffing oscillator, and the KdV PDEs.\n\n**Weaknesses**\n\n- The proposed approach requires learning from trajectory data to approximate a proper skew-symmetric matrix using a neural network. In other words, it necessitates obtaining a sufficient amount of high-quality trajectory data generated by a high-order integrator. Moreover, the performance of the learned numerical integrator heavily depends on the quality of the provided data. Therefore, the actual overall computational cost of the proposed method should include the time needed to generate the training data, the training time of the neural network, and the integration time during testing. However, current discussions of computational cost appear to focus solely on post-training integration time, overlooking these significant additional costs.\n\n- While the paper is generally well written, it is not entirely clear how the proposed method can universally approximate dissipative discrete gradient integrators. In line 204, the authors claim that the universal approximation properties (Theorem 4.5 for energy-preserving systems and Theorem 4.6 for dissipative systems) follow directly from Theorem 4.4. However, since Theorem 4.4 is valid only for conservative systems, and the authors state in lines 188–190 that it is impossible to derive an analogous result for dissipative systems, it remains unclear how Theorem 4.6 is justified.\n\nQUESTIONS:\n- Could the authors discuss the actual computational cost of the proposed method and compare it with that of the high-order method used to generate the training trajectories?\n\n- Could the authors provide a more detailed explanation of how the proposed method handles dissipative systems? For example, how is Theorem 4.6 derived? Additionally, why can the MLP-based **skew-symmetric** matrices approximate general **dissipative** integrators?\n\n- Is the learned neural skew-symmetric matrix generalizable across different integrator orders? For instance, if it is trained using a 4th-order discrete gradient scheme, can it still yield good results when tested with a 1st-order integrator, or vice versa?\n\n- There are some typos, e.g., in line 118, \"...if a matrix $S$ is skew-symmetric, then $u^T \\mathbf{G} u = 0$…\", ...\n\nLIMITATIONS:\nYes, the authors acknowledge the computational cost as a limitation of the proposed method. However, I believe this issue is more critical than currently addressed. As noted in the Weaknesses section, a more thorough discussion is needed to support their efficiency claims.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses the challenge of accurately simulating physical systems while preserving their intrinsic energy laws. Traditional numerical methods often fail to maintain these invariants, leading to instability. UEPI proposes a novel hybrid approach, integrating neural networks with discrete gradient methods to learn highly accurate, structure-preserving integrators from data. Physical phenomena are often described by energy-based theories, requiring numerical methods that preserve energy conservation or dissipation for accuracy and long-term stability. The paper demonstrates superior accuracy over  diverse systems like chaotic Hamiltonian, dissipative, and nonlinear PDEs.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n\n- The UEPI methodology is built upon discrete gradient methods, numerical schemes for differential equations with energy conservation or dissipation laws. The formulation ensures that energy preservation / dissipation is built-in into the formulation.\n\n### Weaknesses\n\n- Currently looking at the results, it seems like the proposed method is a better discrete gradient method. However, it is not clear where it stands compared to other numerical integrators. A simple example would be for the chaotic Henon-Heiles model [1] satisfies invariants even more consistently than the proposed method.\n- Henon-Heiles is very moderately chaotic in the tested regime (E < 1 / 6) [2]. Claiming that this method is general enough and works for chaotic systems is not validated with proper experiments.\n- Several examples (double pendulum / two body problem) can be solved with a DAE solver or with a ODE Mass Matrix solver. Additionally, comparison with Sympnets has been provided, but other data-driven methods that can conserve invariants (often outperforming Sympnets) like COMET [3], Stabilized Neural ODEs [4], etc. are omitted from discussion.\n\n\n[1] https://docs.sciml.ai/DiffEqDocs/stable/examples/classical_physics/#H%C3%A9non-Heiles-System\n\n[2] https://adsabs.harvard.edu/full/1964AJ.....69...73H\n\n[3] https://arxiv.org/abs/2208.10387\n\n[4] https://arxiv.org/abs/2306.09739\n\nQUESTIONS:\n- What is the precision at which the experiments are run? At low-precision (FP32) common in machine learning, it is almost impossible for any solver (learned or otherwise) to capture chaotic behavior [1]\n- Since the paper proposes better numerical solvers (and not a better discrete gradient method), additional benchmarks using DAEs, symplectic integrators, etc. (at sufficient precision) should be demonstrated.\n\n[1] https://www.stochasticlifestyle.com/how-chaotic-is-chaos-how-some-ai-for-science-sciml-papers-are-overstating-accuracy-claims/\n\nLIMITATIONS:\nNo, see the questions section.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a method for learning numerical integrators from data for deterministic energy-conserving or dissipative systems. The key idea is to use the neural network to parametrize the discrete gradients in the classical implicit discrete-gradient methods. The learned discrete gradient is adaptive to the time-stepping size of the training data; thus, it can achieve large time-stepping while keeping the accuracy. Numerical results show that the proposed method can learn accurate large time-stepping integrators for various differential equations, including chaotic Hamiltonian systems (the Henon-Heiles Model and the two-body problem, the double pendulum), dissipative pendulum, and the KdV equation.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n- The paper proposes a robust method for learning a data-driven numerical integrator that incorporates energy conservation laws into the design of neural network approximations.   \n- The paper validates the proposed approach in a diverse set of systems, including nonlinear Hamiltonian systems and dissipative systems. \n- The paper is well-organized, and the main idea is clearly presented. Additionally, the paper provides a detailed description of the experiments and the code, ensuring the results are reproducible. \n \nWeakness: \nThe paper is solid overall. The following are minor weakness points: \n- Lack of statistical robustness tests. Since the proposed integrator is learned from data, it is important to discuss its dependence on training data, including data size and the methods used to generate training data (single or multiple trajectories, which can be interpreted as the distribution of the training data). Additionally, it is crucial to test the learned integrator with different initial conditions, not just at the end of the training trajectory. Then, report the mean and standard deviation of the errors (in time, as shown in the last plot of Figure 3) for each example.\n- Missing computational time for the examples, except the first example. Computation efficiency is a major motivation of the proposed method over classical methods (which use a smaller time step size). However, the complex NN approximation will take more time to compute the implicit integrator. It is unclear if the proposed method is more efficient. Therefore, it is important to report computation time for each example.  \n- Theorem 3.1 and Theorem 4.2 are more like definitions or simple facts. They are not sufficient to be called a theorem. In particular, the last sentence of Theorem 4.2 is not clear enough to be a statement in a theorem. \n- The KdV example lacks the details, and the results are rough.\n\nQUESTIONS:\n- What is the matrix $\\bar{S}$ in Eq.(5)? Is it the same as the matrix $S$ in Figure 1? How to derive or approximate $S$ or $\\bar{S}$ in applications? \n \n- Line 263: is $u^{(i)}$ a typo for $u^{(i-1)}$? Is there any reason for selecting such random training data?\n\n- Line 306+: the natation x is a typo of q. What if the system is stiff?\n\n- KdV example: What is the training data, and how is it generated? How does the training data affect the performance of the proposed model?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-03"}