Efficiently Parameterized Neural Metriplectic Systems
Authors: Anthony Gruber
Kookjin Lee
Haksoo Lim
Noseong Park
Nathaniel Trask
Venue: ICLR 2025 Poster (2025)
Keywords: metriplectic systems
structure preservation
energy conservation
entropy stability
neural ODEs
Primary area: learning on time series and dynamical systems
TL;DR: General metriplectic systems are parameterized in a way which is universally approximating, bounded in error, and scales quadratically with the size of the system.

Abstract: Metriplectic systems are learned from data in a way that scales quadratically in both the size of the state and the rank of the metriplectic operators.  In addition to being provably energy-conserving and entropy-stable, the proposed neural metriplectic systems (NMS) approach includes approximation results that demonstrate its ability to accurately learn metriplectic dynamics from data, along with an error estimate that indicates its potential for generalization to unseen timescales when the approximation error is low. Examples are provided to illustrate performance both with full state information available and when entropic variables are unknown, confirming that the NMS approach exhibits superior accuracy and scalability without compromising on model expressivity.

Published as a conference paper at ICLR 2025
EFFICIENTLY PARAMETERIZED NEURAL METRIPLEC- TIC SYSTEMS
Anthony Gruber*
Kookjin Lee
Center for Computing Research Sandia National Laboratories Albuquerque, NM. USA adgrube@sandia..gr
School of Computing and Augmented Intelligence
Arizona State University
Tempe, AZ. USA
kookjin, Tee.nneeefee lee@asu.edi
Haksoo Lim
Noseong Park
Big Data Analytics Laboratory Yonsei University
Big Data Analytics Laboratory
Korea Advanced Institute of Science and Technology Daejeon, Korea noseong.. kaist. cs@gma com
Seoul, Korea
Manaksso96@nave:.
Nathaniel Trask
School of Engineering and Applied Science University of Pennsylvania Philadelphia, PA. USA
ntrask@seas. upenn.
ABSTRACT
Metriplectic systems are learned from data in a way that scales quadratically in both the size of the state and the rank of the metriplectic operators. In addition being provably conserinn and entropy-stable, the proposed neural metriplectic systems (NMS) approach includes approximation results that demonstrate its ability to accurately learn metriplectic dynamics from data, along with an error estimate that indicates its potential for generalization to unseen timescales when the approximation error is low. Examples are provided to illustrate performance both with full state information available and when entropic variables are unknown, confirming that the NMS approach exhibits superior accuracy and scalability without compromising on model expressivity.
INTRODUCTION
The theory of metriplectic, also called GENERIC, systems Morrison 1986); Grmela and Öttinger (1997) provides a principled formalism for encoding dissipative dynamics in terms of complete thermodynamical systems that conserve energy and produce entropy. By formally expressing the reversible and irreversible parts of state evolution with separate algebraic brackets, the metriplectic formalism provides a general mechanism for maintaining essential conservation laws while simulta- neously respecting dissipative effects. Thermodynamic completeness implies that any dissipation caught within metriplectic system through the generation of entropy, allowing for a holistic
treatment which has already found use in modeling fluids Morrison (1984] 2009), plasmas Kaufman and Morrison (1982); Materassi (2012), and kinetic theories Kaulman (1984); Holm et al. (2008).
From a machine learning point of view, the formal separation of conservative and dissipative effects makes metriplectic systems highly appealing for the development of phenomenological models. Given data which is physics-comee coossrs-cnnst or exhibits some believed properties, a metriplectic system can be learned exhibit the same properties with clearly identifiable conservative and dissipative parts. This allows for more nuanced understanding of the governing dynamics via an evolution equation which reduces to an idealized Hamiltonian system as the dissipation is taken to zero. Moreover, elements
"crreepoondigg author

---

Published as a conference paper at ICLR 2025
in the kernel of the learned conservative part are immediately understood as Casimir invariants, which are special conservation laws inherent to the phase space of solutions, and are often useful for understanding and exerting control on oowdimensiooal structure in the system. On the other hand, the same benefit of metriplectic structure as a "direct sum" of reversible and irreversible parts makes it challenging to parameterize in efficient way, since delicate degeneracy conditions must be enforced in the system for all time. In fact, there are no methods at present for learning general metriplectic systems which scale optimally with both dimension and the rank of metriplectic data-an issue which this work directly addresses.
Precisely, metriplectic dynamics on the finite or infinite dimensional phase space P are generated by a free energy function(al) F P - R. F = E+S defined in terms of pair E, S P + R representing energy and entropy, respectively, along with two algehraic brackets (..). C (P) (P)
Co which are bilinear derivations on (P) with prescribed symmetries and degeneracies
O. Here (:.)) is an antisymmetric Poisson bracket, which is a Lie algebra realization on functions, and [:-]] is a degenerate metric bracket which is symmetric and positive semi-definite. When P C [2" for some n > 0, these brackets can be identified with symmetric matrix fields L P - Skew,, M P Sym, (IR) satisfying {F, G)
{ [E,
VF LVG and [F. = VF MVG for all functions F, G E (P) and all states a E P. Using the degeneracy conditions along with V I and abusing notation slightly then leads the standard equations for metriplectic dynamics,
= (a, r) + a F] = (x. E) + [x, S LVE + MVS,
which are provably energy conserving and entropy producing. To see why this is the case, recall that LT = - L. It follows that the infinitesimal change in energy satisfies
E = à. VE LVE. VE + MVS. VE LVVE VE + VS MVE = 0,
and therefore energy is conserved along the trajectory of æ. Similarly, the fact that MT = M is positive semi-definite implies that
Š VS = LVE VS + MVS VS = -VE LVS + MVS VS = [ M 2 0,
so that entropy is nondecreasing along æ as well. Geometrically, this means that the motion of trajectory æ is everywhere tangent to the level sets of energy and transverse to those of entropy, reflecting the fact that metriplectic dynamics are combination of noncanonical Hamiltonian (M 0) and generalized gradient (L = 0) motions. Note that these considerations also imply the Lyapunov stability of metriplectic trajectories, as can be seen by taking E as a Lyapunov function. Importantly, this also implies that metriplectic trajectories which start in the (often compact) set K = {= E(x)= < E(x,)) remain there for all time.
In phenomenological modeling, the entropy S is typically chosen from Casimirs of the Poisson bracket generated by L, i.c those quantities C e (P)()) such that LVC = O. On the other hand, the method which will be presented here, termed neural metriplectic systems (NMS), allows for all of the metriplectic data L, M, E, S to be approximated simultaneously. removing the need for Casimir invariants to be known assumed ahead of time. The only restriction inherent to NMS is that the metriplectic system being approximated is nondegenerate (c.f. Definition 3.1 a mild condition meaning that the gradients of energy and entropy cannot vanish at any point æ in the phase space. It will be shown that NMS alleviates known issues with previous methods for metriplectic learning, leading to easier training, superior parametric efficiency, and better generalization performance.
Contributions. The proposed NMS method for learning metriplectic models offers the following advantages over previous state-of-ther -ttt--art-:: (1) It approximates arbitrary nondegenerate metriplectic dynamics with optimal quadratic scaling in both the problem dimension and the rank r of the irreversible dynamics. (2) It produces realistic, thermodynamically consistent entropic dynamics from unobserved entropy data. (3) It admits universal approximation and error accunulation results given in Proposition (37 and Theorem 3.9 (4) It yields exact energy conservation and entropy stability by construction, allowing for effective generalization to unseen timescales.
2 PREVIOUS AND RELATED WORK
Previous attempts to learn metriplectic systems from data separate into "hard" and "soft" constrained methods. Hard constrained methods enforce metriplectic structure by construction, that the
2

---

Published as a conference paper at ICLR 2025
defining properties of metriplecticity cannot be violated. Conversely, methods with soft constraints relax some aspects of metriplectic structure in order to produce wider model class which is easier to parameterize. While hard constraints are the only way to truly guarantee appropriate generalization in the learned surrogate, the hope of soft constrained methods is that the resulting model is "close enough" to metriplectic that it will exhibit some of the favorable characteristies of metriplectic systems, such as energy and entropy stability. Some properties of the methods compared in this work are summarized in Table
Soft constrained methods. Attempts to learn metriplectic systems using soft constraints rely on relaxing the degeneracy conditions LVS MVF = 0. This the approach taken in Hernández et al. 202 termed SPNN, which learns an iimost-metriplectte model parameterized with generic neural networks through a simple L penalty term in the training loss, Lpnn LV + SM This widens the space of allowable network parameterizations for the approximate operators L, M. While the resulting model violates the first and second laws of thermodynamics, the authors show that reasonable trajectories are still obtained, at least when applied within the range of timescales used for training. A similar approach is taken in Hernández al. (2021a), which targets larger problems and develops an almost-metripletic model reduction strategy based on the same core idea.
Hard constrained methods. Perhaps the first example of learning metriplectic systems from data was given in González et al. (2019) in the context of system identification. Here, training data is assumed to come from a finite element simulation, so that the discrete gradients of energy and entropy can be approximated as V (xx) Ax, V5(x)= Bx. Assuming fixed form for L produces constrained learning problem for the constant matrices M, A, B which is solved to yield a provably metriplectic surrogate model. Similarly, the work Ruiz et al. (202I) learns M, E given L, S by considering a fixed block-wise decoupled form which trivializes the degeneracy conditions, i.e.
L= ++ 0: 0 0] and M = [0 0: 0 +). This line of thought is continued in Xu et al. (2022) and Xu ct al. (023), both of which learn metriplectic systems with neural network parameterizations by assuming this decoupled block structure along with specialized forms for the metriplectic data. A somewhat broader class of metriplectic systems are considered in Gruber et al.] (2023b) using tools from exterior calculus, with the goal of learning metriplectic dynamics on graph data. This leads to a structure-preserving network surrogate which scales linearly in the size of the graph domain, but also cannot express arbitrary metriplectic dynamics due to the specific modeling choices for L, M.
A particularly inspirational method for learning general metriplectic systems was given in Lee et al. (2021) and termed GNODE, building on parameterizations of metriplectic operators developed in Ot tinger 2014). GNODE parameterizes learnable reversible and irreversible bracket generating matrices L, M in terms of state-independent tensors & E (R")ss and Ç E @4; for a, R, % u, S n, the authors choose Las(w) = Σ, 075 and Mas(#) = Σn Sap,Bo E00" E, where 8a 0F/Oxa. & is totally antisymmetric, and Ç is symmetric between the pairs (ax, μ)) and (3,v) but antisymmetric within each of these pairs. The key idea here to exchange the prob- lem of enforcing degeneracy conditions LVE = MVS 0 in matrix fields L, M with the problem of enforcing symmetry conditions in tensor fields E, C. which is comparatively easier but comes at the expense of underdetermining the problem. In GNODE, enforcement of these symmetries is handled by generic learnable 3-tensor E (R2))8 along with learnable matrices D E Sym,,(R) and A E Skew, (R) for S S S n, leading to the final parameterizations Sasy- = 37 (Ear,- - Eari + Egya - Éso + ETa E7 and Sap,,8r = Σ., Ai Dec Along with learnable energy and entropy functions E, S parameterized by multi- layer perceptrons (MLPs), the data L, MM Iearned by GNODE guarantees metriplectic structure in the surrogate model and leads to successful learning of metriplectic systems in some simple cases of interest. However, note that this is a highly redundant parameterization requiring (3) + r()) + 1I+ + 2 learnable scalar functions, which exhibits On + rn?) scaling in the problem size because of the necessity to compute and store (t) entries of & and r (() entries of A. Additionally, the assumption of sate-independence in the bracket generating tensors E, Ç is somewhat restrictive, limiting the class of problems to which
GNODE can be applied.
A related approach to learning metriplectic dynamics with hard constraints was seen in Zhang
let al. (2022). which proposed series of GFINN architectures depending how much of the information L, M, E, S is assumed to be known. In the case that L, M are known, the GFINN
energy and entropy are parameterized with scalar-valued functions f P where f I R
3

---

Published as a conference paper at ICLR 2025
(E S) is learnable and A R" R" is orthogonal projection onto the kernel of the (known) operator A (L or M). It follows that the gradient Vff o AkerA) = A V lies in the kernel of A, so that the degeneracy conditions guaranteed at the expense of con- straining the model class of potential energies/entropies. Alternatively, in the case that all of L, M, E., S are unknown, GFINNs use learnable scalar functions f for F,, parameterized by MLPs as well as two matrix fields Q Q ER'x with rows given by q. (S!V for learnable skew-symmetric matrices S S s. E, S. Along with two triangular (r x r) ma- trix fields Tt, TM. this yields the parameterizations L(x) Q (T)
Ty (=)) (m)
and (z) = Qa ( (TM T ( (a:), which necessarily satisfy the symmetries and degeneracy conditions required for metriplectic structure. GFINNs are shown to both increase ex- pressivity over the GNODE method as well as decrease redundancy, since the need for an explicit order-3 tensor field is removed and the reversible and irreversible brackets are allowed to depend explicitly on the state æ. However, GFINNs still exhibit cubic scaling through the requirement of rn(n 1) + r² + 2 (rn²) learnable functions, which is well above the theoretical minimum required to express a general metriplectic system and leads to difficulties in training the resulting models.
Model reduction. Finally, it is worth mentioning the closely related line of work involving model reduction for metriplectic systems, which began with Öttinger (2015). As remarked there, preserving metriplecticity in reduced-order models (ROMs) exhibits many challenges due to its delicate require- ments on the kernels of the involved operators. There are also hard and soft constrained approaches: the already mentioned Hernández et al. (2021a) aims to learn close-to- motetripeleitttcccccccc data-driven ROM by enforcing degeneracies by penalty, while Gruber et al. (2023a) directly enforces metriplectic structure in projection- based ROMs using exterior algebraic factorizations. The parameterizations of metriplectic data presented here are related to those presented in Gruber et al. (2023a), although NMS does not require access nonzero components of the gradients VE, VS. It is also worth mentioning the influential monograph Dorst et al. (2007) which contains many examples of exterior algebraic methods for computer scientific applications.
FORMULATION AND ANALYSIS
The proposed formulation of the metriplectic bracket-generating operators L, M used by NMS is based on the idca of exploiting structure in the tensor fields E, C to reduce the necessary number of degrees of freedom. In particular, it will be shown that the degeneracy conditions LVS = MVE- = 0 imply more than just symmetry constraints on these fields, and that taking these additional constraints into account allows for a more compact representation of metriplectic data. Following this, results are presented which show that the proposed formulation is universally approximating on nondegenerate systems fc.f. Definition 3.1 and admits generalization error bound in time.
3.1 EXTERIOR ALGEBRA
Developing these metriplectic expressions will require some basic facts from exterior algebra, of which more details can be found in, e.g., Tu (0017 Chapter 19). The basic objects in the exterior algebra AV over the vector space V are multivectors, which are formal linear combinations of totally antisymmetric tensors on V. More precisely, if T(V) denotes the two-sided ideal of the free tensor algebra T(V) generated by elements of the form v & e (v E V), then the exterior algebra the quotient space AV ~ T(V)/I(V) equipped with the antisymmetric wedge product operation A. This graded algebra is equipped with natural projection operators pk AV + ~VV which map between the full exterior algebra and the th exterior power of V. denoted A A whose elements are homogeneous k--vecooss More generally, given an n-manifold M with tangent bundle TM, the exterior algebra MT is the algebra of multivector fields whose fiber over M is given by AT,..
For the present purposes, it will be useful to develop a correspondence between bivectors B E (R())) and skew-symmetric matrices B E Skew,, (TR), which follows directly from Riesz representation in terms of the usual Euclidean dot product. More precisely, supposing that e eI, en are the standard basis vectors for R", any bivector B E A 2TR can be represented as B = Exes Bie, A e, where
Bid TR. denote the components of B. Define grade-lowering action of bivectors vectors through
4

---

Published as a conference paper at ICLR 2025
right contraction (see e.g. Section 3.4 of Dorst et al. (2007) ), expressed for any vector and basis bivector e, Ñey as (e, / e,,) v == (e, vue; - (e, e,e, [e:,,e; It follows that the action of B is equivalent
to
U Σ Bd v)e (e, v)e,) = Σ B uje ee - EB = EB = Bu, [5
2.3
where = R is skew-symmettic matrix representing B, and we have re-indexed under the second sum and applied that B = BB for all i.j. Since the kernel of this action the zero bivector, is straightforward to check that this string of equalities defines an isomorphism M A²R" + Skew,, (R) from the 2nd exterior power of I to the space of skow-symmetric (n x matriccss over I: in what follows, will write B ~ B rather than B M(B) for notational convenience. Note that a correspondence in the more general case of bivector/matrix fields follows in the usual way via the fiber-wise extension of M.
3.2 LEARNABLE METRIPLECTIC APERATORS
It is now possible to explain the proposed NMS formulation. First, note the following key definition which prevents the consideration of unphysical examples.
Definition 3.1. A metriplectic system on K R" generated by the data L, M, E, will be called nondegenerate provided VE, VS + 0 for all x K.
With this, the NMS parameterizations for metriplectic operators predicated algebraic result proven in Appendix A
Lemma 3.2. Let K C RR. For all E K, the operator L - Raxa satisfies L -L
and LVS O for some S - R, VS + O, provided there exists non-unique bivector field A: U + A Rg and equivalent matrix field = such that
VS AS=A-
1 |SS
L
A^
AVSAVS.
2
Similarly, for all x E K positive semi-definite operator M K - RaxD satisfies MT = M and M V 0 for some E K + RR, VE 0, provided there exists a non-unique matrix-valued B K RxT and symmetric matrix-valued D K Rrrr such that and
VE
VE
M
Doe
A
VE
R
6
VE
V
VE
8.
b
VE
b'
VE
Σ Det
6'
VE
b'
VE
VE
2
V
8,
where b² denotes the th column of B. Moreover, using Pj- (I- VIVA
to denote the
orthogonal projector onto Span (
(Vf)+, these parameterizations of L.M equivalent to the matricized expressions L = Ps APS and M = M PEBBBTB
Remark 3.3. Observe that the projections appearing these expressions are the minimum necessary for guaranteeing the symmetries and degeneracy conditions necessary for general metriplectic structure. In particular, conjugation by P, respects symmetry and ensures that both the input and output to the conjugated matrix field lie in Span + Note that this implies that the class of nondegenerate metriplectic systems cannot be parameterized in a way that scales aub-quadratically in the state dimension n.
Lemma 3.2 demonstrates specific parameterizations for L, M that hold for any nondegenerate metriplectic data and are core to the NMS method for learning metriplectic dynamics. While generally underdetermined, these expressions are in sense maximally specific given no additional information, since there is nothing available the general metriplectic formalism to determine the matrix fields A, BTB Span Span(V (VE), respectively. The following Theorem, also proven in Appendix A provides rigorous correspondence between metriplectic systems and these particular parameterizations.
5

---

Published as a conference paper at ICLR 2025
Theorem 3.4. The data L, M, E.S form a nondegenerate metriplectic system in the state variable a E K if and only if there exist a neew-yymmetris A K - Skewn (R), symmetric postive semidefinite D K Sym, and generic B
K + R2xr such that
LVE MVS Ps APs VE + PE B
Remark 3.5. As mentioned, the proposed parameterizations for L, M are not one-to-one but properly contain the set of valid nondegenerate metriplectic systems in E, S. This is due to both the orthogonal projections employed, which induce non-uniqueness in the L, M matrix fields, as well as the lack of guaranteed Jacobi identity for L. Recall that the Jacobi identity is necessary for the matrix field L to generate a true Poisson manifold structure, and hence for o to be the velocity field of a truly metriplectic system. For i.j, S n, the Jacobi identity is given in components E, L Ljs- + L Lus Lær0e Lij 0. However, this requirement is not often enforced in algorithms for learning general metriplectic (or even symplectic) systems, since is considered subordinate to energy conservation and it is well known that both qualities cannot hold simultaneously in general Zhong and Marsden (1988). Future work will compare the alternative choice of parameterizations which enforce the Jacobi identity explicitly, maintaining Poisson structure while sacrificing strong-form energy conservation.
3.3 SPECIFIC PARAMETERIZATION
Now that Theorem 3.4 has provided a model class which is rich enough to express any desired metriplectic system, it remains to discuss what NMS actually learns. First, note that it unlikely to be the case that E, S are known priori, so it is beneficial to allow these functions to be learnable alongside the governing operators L, ... For simplicity, energy and entropy F,, S are parameterized as scalar-valued MLPs with tanh activation, although any desired architecture could be chosen for this task. The skew-symmetric matrix field A = Auni - A used to build L is parameterized through its strictly lower-triangular part Acri using a vector-valued MLP with output dimension (2). which guarantees that the mapping Atri +> A above is bijective. Similarly, the symmetric matrix field D K chol is parameterized through its lower-triangular Cholesky factor Kchol, which a vector-valued MLP with output dimension ). While this choice does not yield a bijective mapping Kehol D unless, e.g.. D is assumed be positive definite with diagonal entries of fixed sign, this does not hinder the method in practice. In fact, D should not be positive definite in general, as this equivalent to claiming that M positive definite on vectors tangent the level sets of E. Finally, the generic matrix field B is parameterized as a vector- nr. Remarkably, the exterior algebraic expressions in Lemma 3.2 require less redundant operations than the corresponding matricized expressions in Theorem 3.4 and therefore the expressions from Lemma 3.2 are used when implementing NMS. This can be seen from the proof of Lemma 3.2 in Appendix where matricization of these expressions requires computing term which is identically zero due to symmetry considerations. Figure summarizes the proposed NMS architecture.
MLP with output dimension
Remark 3.6. With these choices, the NMS parameterization of metriplectic systems requires only (1/2) (nn + + (n --- rr) + 2 learnable scalar functions, in contrast to
(3) r r(²) + (*++)) 2 for
the GNODE approach in Lee et al. 22021) and rn(n - 1) + r + 2 for the GFINN approach in Zhang
al. (2022). In particular, NMS is quadratic in both n, r with no decrease in model expressivity, in contrast to the cubic scaling of previous methods.
Calcuirad
Table 1: Properties of the metriplectic architectures compared.
xits)
+(6+)
wathlel
Name
Physics Bias Restrictive
Scale
B(s)
L((s)
NODE
None
No
Lincar
Auiz))
SPNN
Soft
No
Quadratic
GNODE
Hard
Yes
Cubic
GFINN
Hard
No
Cubic
NMS Hard
No
Quadratic
Figure 1: A visual depiction of the NMS architecture

---

Published as a conference paper at ICLR 2025
3.4 APPROXIMATION AND ERROR
Besides offering a compact parameterization of metriplectic dynamics, the expressions used in NMS also exhibit desirable approximation properties which guarantee a reasonable bound on state error over time. To state this precisely, first note the following universal approximation result proven in Appendix A
Proposition 3.7. Let K C R" be compact and E, K - R be continuous such that LVS MVF = 0 and V F. VS V 0 for all x: E K. For any > 0, there exist two-layer neural network functions E. K + R L: K + Skew (R) and M K + Sym, such that E V + on K, M is positive semi-definite, LVŠ = MV E O for all æ E K, and each approximate function E-close to its given counterpart on K. Moreover, if L, M have k continuous derivatives on K then so do L, M.
Remark 3.8. The assumption E K of the state remaining in a compact set V is not restrictive when at least one of E, : IR" + R. say E, has bounded sublevel sets. In this case, letting 2p =0) follows from E < O that E())(()) E(xo) + So E(x((T) dr I E(ro), so that the entire trajectory xt) lies in the (closed and bounded) compact set K = { E(x) S E(xo)} C R".
I Leaning on Proposition 3.7 and classical universal approximation results in Li 1996). it is further possible to establish the following error estimate also proven in Appendix which gives an idea of the error accumulation rate that can be expected from this method.
Theorem 3.9. Suppose L, M, E, S are nondegenerate metriplectic data such that L, M have at least one continuous derivative, E, S have Lipschitz continuous gradients, and least one of E, S have bounded sublevel sets. For any defined by two-layer neural networks such that, for all T > 0,
there exist nondegenerate metriplectic data L, M, E, S
(1 21 dt
VeaaT - 2e + T + 1,
SE
where a, E R are constants depending on both sets of metriplectic data and à V())) M (+))S(())
Remark 3.10. Theorem 3.9 provides a bound on state error over time under the assumption that the approximation error in the metriplectic networks can be controlled. On the other hand, notice that Theorem can can also be understood as generic error bound on any trained metriplectic networks L, M, E, S provided universal approximation results are not invoked in the estimation leading to eb.
This result confirms that the error in the state x for a fixed final time T tends to zero with the approximation error in the networks L, .,M E, S, one would hope based on the approximation capabilities of neural networks. More importantly, Theorem also bounds the generalization error of any trained metriplectic network for appropriate (and possibly large) e equal the maximum approximation error on K, where the learned metriplectic trajectories are confined for all time. With this theoretical guidance, the remaining goal of this work is demonstrate that NMS is also practically effective at learning metriplectic systems from data and exhibits reasonable generalization to unseen timescales.
ALGORITHM
Similar to previous approaches in Lee et al. (2021) and Zhang al. (2022). the learnable parameters in NMS are calibrated using data along solution trajectories to given dynamical system. This brings up an important question regarding how much information about the system question is realistically present in the training data. While many systems have known metriplectic form, it is not always the case that one will know metriplectic governing equations for a given set of training data. Therefore, two approaches are considered in the experiments below corresponding to whether full or partial state information is assumed available during NMS training. More precisely, the state x = (x", (x" x") will be partitioned into "observable" and "unobservable" variables, where x" may be empty in the case that full state information is available. In a partially observable system x" typically contains positions and momenta while x contains entropy or other configuration variables which are more
7

---

Published as a conference paper at ICLR 2025
Algorithm Training neural metriplectic systems
1: Input: snapshot data X E Raxxs, each column 2, 2t,,; Ms), target rank r 2 1, batch size
n4 1.
2: Initialize networks Atri, B, Kehoi, E, and loss
0
3: for step in Nstops do
4: Randomly draw batch P = {[te2: Lz I===
5: for (t, x) in do
6: Evaluate (a)) (xx). K choo (x), F(x), ((x))
7:
Automatically differentiate E, S to obtain V ( (S(x))
8:
Form A = (x) ( and (xx) K Kohol (x) K choo (
9:
Build ((x) a M (x) according to Lemma 3.2
10:
Evaluate à L(x) V ((x) (x)
11:
Randomly draw 711 ni and form t, = =t+ nyAt for j
12:
T1 ....... 1t ODEsolve (z, fi....tt) 3+=1-1 Σj Loss( 2.)) I.
13:
14:
end for
15: Rescale L -
16: Update Auri B, Kehol, E, through gradient descent on L.
end for
difficult observe during physical experiments. In both cases, NMS will learn a metriplectic system in æ according to the procedure described in Algorithm
Note that the batch-wise training strategy in Algorithm T requires initial conditions for x" in the partially observed case. There are several options for this, and two specific strategies will be considered here. Suppose the data are drawn from the training interval [0, T] with initial state 2g and final state ær. The first strategy sets 2 0, 2 (where is the all ones vector), and x = 1/s. < 8 T, so that the unobserved states are initially assumed to lie on a straight line. In the absence of other information, this ensures that the entropy will be satisfy the monotonicity constraint necessary for metriplectic structure. The second strategy is more sophisticated, and involves training diffusion model to predict the distribution of x given æ". Specific details of this procedure are given in Appendix H Note that other strategies are also possible, such as allowing the unobserved initial states to be trainable in Chen al. (2020).
EXAMPLES
The goal of the following experiments show that NMS effective even when entropic information cannot be observed during training, yielding superior performance when compared to previous methods including GNODE, GFINN, and SPNN discussed in Section 2 The metrics considered for this purpose will be mean absolute error (MAE) and mean squared error (MSE) defined in the usual way as the average t! (resp. squared (?) error between the discrete states x; E E x, For brevity, many implementation details have been omitted here and can be found in Appendix B An additional experiment showing the effectiveness of NMS in the presence of both full and partial state information be found in Appendix D
Remark 5.1. To facilitate more equal parameter count between the compared metriplectic meth- ods, the results of the experiments below were generated using the alternative parameterization D = KKT where K K + prxr' is full and r' 2 Of course, this change does not affect metriplecticity since D still positive semi-definite for each K.
5.1 TWO GAS CONTAINERS
The first test NMS involves two ideal gas containers separated by movable wall which removed time la allowing for the exchange of heat and volume. In this example, the motion of the state
8

---

Published as a conference paper at ICLR 2025
(q
S p
S
is governed by the metriplectic equations:
2 (x(x)
E2(x)
p
m
3 2L - q
9. k
9N
1
1
5, 4E, (z)
5, 4E (x)
Ej 2
E2(x)
Ei
E (x)
where (q. p) are the position and momentum of the separating wall, S1, .2 are the entropies of the two subsystems, and the internal energies E1, E2 are determined from the Backur-Tetrodi equation for ideal gases, S:/NkB = In In(ev, (VV ) I i < 2. Here, m denotes the mass of the wall, 27.
is the total length of the system, and V; is the volume of the ith container. As in Lee et al. 2021 0.5 fix the characteristic macroscopic unit of entropy while ensures the argument of the logarithm defining E, is dimensionless, This leads to the total entropy S(x) S + S2 and the total energy ((x) (1/2m)p² + Ei(x) + E2(x), which are guaranteed to be nondecreasing and constant, respectively.
2022) Nkg = and
102.25
The primary goal here is to verify that NMS can accurately and stably predict gas container dynamics without the need to observe the entropic variables S₁, S2. To that end, NMS has been compared to GNODE, SPNN, and GFINN on the task of predicting the trajectories of this metriplectic system time, with results displayed in Table 2 More precisely, given intial condition æo and interval 0 < Irain < Ivalid < 4est, each method is trained on partial state information (in the case of NMS) full state information (in the case of the others) from the interval [0, étain]] and validated on (4train, [vania] before state errors in p only are calculated on the whole interval [0, [tss] As can be from Table 2 unfair comparison, avoiding the unphysical behavior which often hinders roft-constrained methods like SPNN. The energy and instantaneous entropy plots in Figure further confirm that the strong enforcement of metriplectic structure guaranteed by NMS leads to correct energetic and entropic dynamics for all time.
and Figure 2 NMS remarkably accurate over unseen timescales even in this
SPNN
GNODE
GFINN
NMS
1.2
1.5
1.0
0.5
100
10
15
20
35
40
(a) Hamiltonian H
(b) Position
2m
-0.5
0.0
2.5
10
15
20
30
20
100
40
80
100
(c) Momentum
(d) Energy E = H >, E.
(c) Instantancous Entropy $
Figure 2: The ground-truth truth and predicted position, momentum, instantaneous entropy, and energies for the two gas containers example in the training (white), validation (yellow), and testing (red) regimes.
5.2 THERMOELASTIC DOUBLE PENDULUM
Next, consider the thermoelastic double pendulum from Romero (2009) with 10-dimensional state variable (q1
42
P1
P2
S S2), which represents highly challenging benchmark for

---

Published as a conference paper at ICLR 2025
Table Prediction errors for x² measured in MSE and MAE on the interval [0, [tstt] in the two gas containers example (left) and the test set in the thermoelastic double pendulum example (right).
NODE
SPNN
GNODE
GFINN
NMS
NODE
SPNN
GNODE
GFINN
NMS
MSE MAR
.12
04
134 10
.6 10
.07
.01 + .2.02 .08 + +.06
MSE MAF
01
+ 01
442 01
40 # 03
.38 + .03
.55 + 10
26 + .1
25 +
13
,13 4.03
04
+ 03
.66 + 04
43 + m
42 + .07
metriplectic methods. The equations of motion in this case are given for 1 sis 22ii200 a
q:
P;
p; -0 (Ei (x) E2())), $1
$2 K (TTTT2 T 1),
m
where x > 0 is a thermal conductivity constant (set to 1), m, is the mass of the ith spring (also set to 1) and T: 0s, E, its absolute temperature. this case, qi P: E R represent the position and momentum of the ith mass, while S; represents the entropy of the jth pendulum. As before, the total entropy S(x) = S + S2 is the sum of the entropics of the two springs, while defining the internal
energies (x = (1 (Im X:)² + In A + e S:-lud; - 1, A qel, = 122 - qıl, leads to the total (1 /2222m222222 + (m)) + (2(2)).
energy F(xz) (1 (1/2m1)lp ppp
The task in this case is prediction across initial conditions. As in Zhang et al. 2022. ). 100 trajectories are drawn from the ranges in Appendix and integrated over the interval [0, 40 with At = 0.1, with an 80/10/10 split for reaining/validationneeset
Here all compared models are trained using full state information. As seen in Table 2 NMS is again the most performant, although all models struggle to approximate the dynamics over the entire training interval. is also notable that the training time of NMS is greatly decreased relative GNODE and GFINN due to its improved quadratic scaling; representative study to this effect is given in Appendix
CONCLUSION
Neural metriplectic systems (NMS) have been considered for learning finite-dimensional metriplectic dynamics from data. Making use of novel non-redundant parameterizations for metriplectic operators, NMS provably approximates arbitrary nondegenerate metriplectic systems with generalization error bounded in terms of the operator approximation quality. Benchmark examples have shown that NMS is both more scalable and more accurate than previous methods, including when only partial state information is observed. Future work will consider extensions of NMS metriplectic systems with the aim of addressing its main limitation: the difficulty of scaling NMS (among all present methods for metriplectic learning) realistic, 3-D problems of the size that would be considered in practice. A promising direction is to consider the use NMS in model reduction, where sparse, large-scale systems are converted small, dense systems through a clever choice of encoding/decoding..
ACKNOWLEDGEMENTS
Sandia National Laboratories is multimission laboratory managed and operated by National Technol- ogy & Engineering Solutions of Sandia, LLC, wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy's National Nuclear Security Administration under contract DE-NA0003525. This paper describes objective technical results and analysis. Any subjective views or opinions that might be expressed in the paper do not necessarily represent the views of the U.S. Department of Energy or the United States Government. This article has been co-authored by an employee of National Technology & Engineering Solutions of Sandia. LLC under Contract No. DE-NA0003525 with the U.S. Department of Energy (DOE). The employee owns all right, title and interest in and to the article and is solely responsible for its contents. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this article or allow others to do so, for United States Government purposes. The DOE will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan https:/wwww.energy www.:/nnrgg.ooo/doowlo..oooooooe www/doowowwdd//oeeeeeeeooeooeeee celooe-ppuubcccccceeeee access--pas. The work of A. Gruber and N. Trask is supported by the John Neumann fellowship Sandia, the U.S.
10

---

Published as a conference paper at ICLR 2025
Department of Energy, Office of Advanced Computing Research under the "Scalable and Efficient Algorithms Causal Reasoning. Operators, Graphs and Spikes" (SEA-CROGS) project, and the DoE Early Career Research Program. K. Lee acknowledges support from the U.S. National Science Foundation under grant IIS 2338909. N. Park and H. Lim acknowledge support from the Korea Advanced Institute of Science and Technology (KAIST) grant funded by the Korea government (MSIT) (No. G04240001, Physics-inspired Deep Learning), as well as the Institute for Information & Communications Technology Planning Evaluation (IIT)) grants funded by the Korea government (MSIT) (No. 2020-0-01361 361,0 Artificial Intelligence Graduate School Program (Yonsei University)). The authors thank Jonas Actor for careful proofreading and suggestions regarding the presentation of Appendix A
REFERENCES
Z. Chen, J. Zhang, M. Arjovsky, and L. Bottou. Symplectic recurrent neural networks. In International Conference on Learning Representations, 2020. URL bttps::/openzeeview wwt.:nnumt/taaaae..m BkGYPREtPr
J. Dormand and J. Prince. family of embedded runge-kutta formulae. Journal of computational and applied mathematics, 6(1):19 19-1192220 26, 1980.
L. Dorst, D. Fontijne, and S. Mann. Geometric Algebra for Computer Science: An Object-oriented Approach Geometry. Morgan Kaufmann, Amsterdam, 2007.
D. González, Chinesta, and F. Cueto. Thermodynamically consistent data-driven computational mechanics. Continuum Mechanics and Thermodynamics, 31 3111):239-253, 2019.
M. Grmela and H. C. Öttinger. Dynamics and thermodynamics of complex fluids. i. development of general formalism. Phys. Rev. E, 56:6620-6632, Dec 1997. doi: 10. 103/PhysReee..56662220 URL https // //link. aps
56. 6620
A. Gruber, M. Gunzburger, L. Ju, and Wang. Energetically consistent model reduction for metriplectic systems. Computer Methods in Applied Mechanics and Engincering, 404:1 15700, 2023a. ISSN 0045-7825, doi: https://doi. 10. 101 601 w/j. cma 2022. 1 15709, URLhtttss:/ / wwww. sc sciencedi com/ sc science/ article/pii httpee///0004522222220
A. Gruher, K. Lee, and N. Trask. Reversible and irreversible bracket-hased dynamics for deep graph neural networks. In seirrt--eventhh Conference on Neural Information Processing Systems, 2023b.
Q. Hernández, A. Badías, D. González, F. Chinesta, and E. Cueto. Deep learning of thermodyyamiic--wwre reduced-order models from data. Computer Methods in Applied Mechanics and Engineering. 379:113763. 2021a.
Q. Herúández, A. Badías, D. González, F. Chinesta, and E. Cueto. Structure- Structure-preservingg neural networks. Journal of Computational Physics, 426:109950, 2021b.
D. D. Holm, V. Putkaradze, and C. Tronci. Kinetic models of oriented aelf-assembly. Journal of Physics A: Mathematical and Theoretical, 444340,344010 aug 2008 doi:
URL
https::/dxx... dei. wwt.:ddii..rgg./../ 10. 0000088811111
A. N. Kaufman. Dissipative hamiltonian systems: A unifying principle. Physics Letters A, 100(8):419 URL //www
422, 1984. ISSN 0375-9601. doi: http://...........oo000300000000
sciencedirect. www.ccm.cccoo/ttttt http://eww......eeeaaaaaaaaaaaaa
A. N. Kaufman and P. J. Morrison. Algebraic structure of the plasma quasilinear equations. Physics Letters
A. 4868) 1982. ISSN 0375-9601 doi: http:://ko..og///...oooo77/.6629
URL
https:// //wp:///wwww. http://wwwwcciecccc www.sciiiiennc........... hwt.eeeddrrettccooeeeeeeoeeeeeee htt.a//cc..........o.......taaaa
D. P. Kingma and Ba. Adam: A method for stochastic optimization. arXiv preprint 4rXiv:1412.6980, 2014.
K. Lee, N. Trask, and Stinis. Machine learning structure preserving brackets for forecasting irreversible processes, Advances Neural Information Processing Systems, 34:5696-5700 2021.
K. Lee, N. Trask, and P. Stinis. Structure-peeserving sparse identification of nonlincar dynamics for data-driven modeling. Mathematical and Scientific Machine Learning, pages 65-80 80. PMLR, 2022.
X. Li. Simultancous approximations of multivariate functions and their derivatives by neural networks with one hidden layer. Neurocomputing, 12(4)3327-343, 1996.
11

---

Published as a conference paper at ICLR 2025
H. Lim, M. Kim, Park, and N. Park. Regular time-series generation using sgm. arXiv preprint arXiv:2301.0852 2023.
E. Materassi, M.; Tassi, Metriplectic framework for dissipative magnett--hydrodynamics Physica D: Nonlinear Phenomena, 2012.
P. Morrison. Thoughts on brackets and dissipation: old and new. In Journal of Physics: Conference Series, volume 169, page 012006. IOP Publishing, 2009.
P. J. Morrison. Some observations regarding brackets and dissipation. Technical Report PAM-228, Center for Pure and Applied Mathematics, University of California, Berkeley, 1984.
J. Morrison. paradigm joined hamiltonian dissipative systems. Physica Nonlinear Phenomena, 18(1):410-41 419, 1986. ISSN 0167-2789. doi: //www.sciencee www.ssecieneieee...mmm com/ http:///ce.aaattccc/////////////
URL Intpps
C. Öttinger. Invevesiblee dynamics, onsager-casimir symmetry. and an application turbulence. Phys. Rev. 90:042121, Oct 2014.
H. C. Öttinger. Preservation of thermodynamic structure in model reduction. Phys. Rev. E, 91:032147, Mar 2015.
1. Romero. Thermodynamically consistent time-stepping algorithms for non- lincar thermomechanical systems. International Journal for Numerical Methods in Engineering, 79(6):706-732, 2023/05/14 2009.
D. Ruiz, D. Portillo, and I. Romero. A data-driven method for dissipative thermomcchanies. PapC-PapprsOnniiee 54(19):315-33 320020,, 2021. 2021.
X. Shang and H. C. Öuinner. Structure-preservinn integrators for dissipative systems based on reversible- irreversibee splitting. Proceedings of the Royal Society A, 276(2234):201944444 2020.
Song. Sohl-Dickstein, Kingma, A. Kumar, Ermon, and B. Poole. Score-based generative modeling through stochastic differential equations. CoRR, abs/2011.13456, 2020. URL httphttss:/::ii..ooott abs 22011 1. 13456
Särkkä and A. Solin, editors. Applied stochastic differential equations, volume 10. Cambridge University Press, 2019.
Differential Geometry: Connections, Curvature, and Characteristic Classes. Springer International Publishing, 2017.
Vincent. connection between score matching and denoising autoencoders. Neural Computation, 23(7): 1661-1674, 2011.
B. Xu, Y. Chen, T. Matsubara, and T. Yaguchi. Learning generic systems using neural symplectic forms. In International Symposium Nonlinear Theory and Its Applications, number A2L-D-03 in IEICE Proceeding Series, pages 29-32. The Institute Electronics, Information, and Communication Engineers (EEICE), 2022.
B. Xu, Y. Chen, T. Matsubara, and T. Yaguchi. Equivalence class learning for GENERIC systems. In ICML Workshop on New Frontiers Learning, Control, and Dynamical Systems, 2023.
Z. Zhang, Y. Shin, and G. Em Karniadakis, Gfinns: Generic formalism informed neural networks for deterministic and stochastic dynamical systems. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 380(2229):20210122 2022.
G. Zhong and J. E. Marsden. Lie-poisson hamilton-jacee jacobi theory and lie- lie-poisson integrators. Physics Letters A. 133(3):134 139, 1988.
A PROOF OF THEORETICAL RESULTS
This Appendix provides proof of the analytical results in Section of the body. First, the parameteri- zations of L, M in terms of exterior algebra are established.
12

---

Published as a conference paper at ICLR 2025
Proof of Lemma 3.2 First, it is necessary to check that the operators L, M parameterized this way satisfy the symmetries and degeneracy conditions claimed in the statement. To that end, recall that a b abt - bat, meaning that (abt baT)T = bba = -a A. b. follows that AT = = - where A denotes the reversion of A, i.e., Ā = Eiej A A e;. Therefore, we may write
==
= -
A
[S
|
showing that L = - L. Moreover, using that
(bΛc) = aa (b = (a. cjb -a (a- b)c.
it follows that
LVS A VS (AVSAVS) VS AVS AVS = 0,
V
since VS- AVS = -VS AVS = O. Moving to the case of M, notice that M DDrv" @ v² for a particular choice v, meaning that
M
Σ Duc & Dt) = D.N'' &v'
Des & r'
D v' M,
8,t
3,8
2,5
8,t
since D is symmetric matrix. Additionally, is straightforward to check that, for any 1 s Sr.
VE= = 6* be VE VE
Sº
VE
VE= -V - 6a VE= = 0.
So, follows immediately that
MVE D
@ 0')
VE = Σ Doe Dos
VE) =
8,t
Now, observe that
1 (AVS(VS)) VS(AV)))
==
V
(AVS(VST) + VS(VS) A)
V
A A VS(VS)
VS(VS)
APS
|VS|
since AT = - and hence DT Av = for all E RR. Similarly, it follows that for every S
PB = b² 6a VE VE,
VE
and therefore M is expressible as
M = (PP(P)))) = BDBBTPP
With Lemma 3.2 established, the proof of Theorem 3.4 is straightforward.
Proof of Theorem 3.4 The "if" direction follows immediately from Lemma 3.2 Now, suppose that L and M define metriplectic system, meaning that the mentioned symmetries and degeneracy conditions hold. Then, it follows from LVS = 0 that the projection PS LP3 L leaves L invariant, so that choosing A = L yields Ps AP L. Similarly, from positive semi-definiteness it follows that M = UAUT = PLUAUT for some colunn-orthomormal and D A yields
and M V E
U E xr and positive diagonal A Tr Therefore, choosing
M P BDB desired.
13

---

Published as a conference paper at ICLR 2025
Looking toward the proof of Proposition] 3.7 we also need to establish the following Lemmata which give control over the orthogonal projectors E. Ps- First, we recall how control over the L 00 norm Hox of a matrix field gives control over its spectral norm |-|.
Lemma A.1. Let A K + gux" be a matrix field defined on the compact set K C. I" with m continuous derivatives. Then, for any & > there exists a two-layer neural network Å K - Raxx
such that SuPMEK A A and SuPœEK V A (total) derivative operator of order k.
V
< S k where V is the
Proof. This will be a direct consequence of Corollary 2.2 in Li 1996) provided we show that |A| S c|A) for some > O. To that end, if an ... 0, 0
n) denote the nonzero singular
values of A, it follows that for each æ E K,
A A = 01 S V + o7
Σ V 1.j
Aig
Auj 2 =
A F
On the other hand, it also follows that
A
A
Σ 1.j
Agj
Aij
Σ 1.
max Aij
Aij
="
max Aij
Aij
A
À
and therefore the desired inequality holds with c n. Now, for any it follows from Li (996) that there exists two layer network A with continuous derivatives such that sup K A A
z/n and s v A
V
< < E for all 1 S k S m. Therefore, it follows that
8
sup #eK!
A -
< n sup æEK
A
< n = 6,
completing the argument.
Next, bound the deviation in the orthogonal projectors P PS.
Lemma A.2. Let f R > R be such that f * on the compact set C R". For any E >
there exists a two-layer neural network f K + R such that v o 0 K, supæek f-
e, S.SUP V v s, and suPzek P7- P
Proof. Denote f and consider any 0 K R. Since |m|
S [6]
+ |/ 6). it follows for all
E K that whenever a öl < (1/2) inf (p).
|o|
|o inf EEk
|0| > O,
so that * 0 K, and since the square function monotonic,
inf أ| |n|
>
inf mek
|v|
inf mek
|a|
inf /w|2
#EK
4
m mek
On the other hand, we also have |o| < | + | - - < |o|| + (1/2) inf /ol, so that, adding and subtracting TuT and applying Cauchy-Schwart Schwarz, it follows that for all æ K.
10
|o|l0
T < maxf[u) |o हl
2//o| + inf
|0|] ö/.
øEK
Now, by Corollary 2.2 Li 1996), for any there exists two-layer neural network f: such that
inf 2
sup ZE K
v
min
inf mek 2supaee vo| + inf onf | 4 SE, SSS,
14

---

Published as a conference paper at ICLR 2025
and also sup f
<
Letting
Vf. follows that for all æ E K.
oot
a - T8T
2/ + inf inf 2
P P
ö/,
|u|2
|ö|
min
|o| [0]
min |0| 2
and therefore, taking the supremum both sides and applying the previous work yields the desired estimate,
2 2 |o| infæek
sup Dc K
P <
P
sup
| ö| <
mek
With these intermediate results established, the proof of the approximation result Proposition 3.7 proceeds as follows.
Proof of Proposition 3.7 Recall from Theorem 3.4 that we can write L = P (Aui A
and similarly for L. Notice that, by adding and subtracting Pg Am and Pg Am it follows that for all x E K,
Ps Ps P Asrips
(P} - P))) Auris
+ P3
Auri - Aui
Ps + P3 Auri (P') -P3)
$ Ps Ps | +
Aini
Aut
+
Aui
Ps
PS
s |Atrll:
max
Acri
Ps.
$
+ Atri
A
where we have used that Ps. PS have unit spectral norm. By Lemma A.1 for any s > there exists
two layer neural network A.n such that suPweK Auri Am a two-layer network 5 with VŠ + on K such that
f. and by Lemma A.2 there exists
sup mE
Ps
PS
min &, max
sup mEEK
Aeril:
sup mek
Agri
follows that S, VŠ are e-close S. VS on K and
sup mek
2 max
Acril:
Am
Ps -
Therefore, the estimate
sup i < sup
Ps Amp
PS
A
E + 1) = E, <
øEK
æEK
4
implies that s-close to L on as well.
Moving to the case of M, that for all æ E K, by writing M = UAUT = Kehol KT chol for
K UAA/ and repeating the first calculation with Kehol place of Auri and P in place of Ps.
P Kehol KT shot PE - P Kchol Jhat P.l
< 2 max { [Kcholl, Ketual
P
P +
Keho KT chol
K
KT chol
Moreover, if Kehol K chhol < (1/2) Kehotl for all K then similar arguments used
in the proof of Lemma A.2 yield the following estimate for all æ E K.
K Kehol chol
KT
Kehol KT
chol
< 2 max [Khoolh, Kechal
Kehool Kechol
< 2 ehhol + inf (chool) Kehoo
Kethol
15

---

follows that Published as a conference paper at ICLR 2025
As before, invoke Lemma that
A.1 to construct two-layer lower-triangulas network Kehol such
-1e
sup
K
Kchol
<
min
inf øek
Kchool,
2 sup K K + inf œe
Keholl|
ØEK K
øeK'
well (using Lemma A.2
network E satisfying V 0 on K and
sup
PE
P
min
E, max
sup DEK
K
Kcholl: sup Øek
K
Kebol|
Øek
4
Again, it follows that E, V are e-close E, VE on K, and by the work above we conclude
ΦEK
sup
M M = sup øek
PE
Kehol K
chot
PE
P
chol
KT P E
chol
2
+ e
2
=
as desired.
now possible to give a proof of the error bound in Theorem 3.9 Recall the [[0, T)) error
metric ||>|| and Lipschitz constant y, defined for all æ; y IR" and L Lipschitz continuous functions
f as
||0||| /
2 (f(x) [(y)] S Tog x y!.
Proof of Theorem 3.9 First, note that the assumption that one of E, S (without loss of generality, say E) has bounded sublevel sets implies bounded trajectories for the state æ as in Remark 3.8 so may assume E K for some compact K C R". Moreover, for any > 0 it follows from Proposition 3.7 that there are approximate networks É, S which are E-close to E, K. Additionally, it follows that É, S have nonzero gradients V V,, VŠ which are also e-close to the true gradients VE, VS K. This implies that for each
K, E É + (E E) S E + so it follows that the sublevel sets {x E(x) I m) } S {æ E(x) I + e} are also bounded. Therefore, we may assume (by potentially enlarging K) that both x, & E K lie in the compact set K for all time.
Now, let y 2. The next goal is to bound the following quantity:
l0 V(=)))((>+++ M(z) M -(aV)) M
= -[(
M = 1Ù +sl- + (M(x)VS(a) (LLV)VEE)))-LLVVEEE))EE
(())VE()))
To that end, notice that by adding and subtracting L(z)VE(a), (()) V fol- lows that
ÚE = ((x)() - VE(#)) + [(xx) ixx) VF()))
+ +(((x)-i()))
- L(x) V
+ (()) -VE())VV(())
VÉ(#))
By Proposition 3.7 there exists two-layer neural network L with one continuous derivative such
that supzek L < <, which implies that İ Lipschitz continuous with (uniformly well- approximated) Lipschitz constant. Using this fact along with the assumed Lipschitz continuity of VE and the approximation properties of the network E already constructed then yields
| <
[Ve sup |L| + Li sup /VE] 1yl + sup i
+ sup VE æEK
=: a + bbg
Similarly, by adding and subtracting ((x)VS(() M x VS) M
Ús = M(x) - + (M(x) M M() VS(Ø) VS())
+ (M(x) - VS( M (V(()) Vš(+)) -
16

---

Published as a conference paper at ICLR 2025
By Proposition 3.7
there exists two-layer network M with continuous derivative such that < E, with M Lipschitz continuous for the same reason as before. follows from
S
M M
this and SUPEER VS V & that
Úsl I
Lvs sup M + L MM
sup V |ya +
sup
M
+
sup ØEK
V
=: aslyl + bs.
ØEK
øEK
ØEK
Now, recall that yl = y |y| is bounded by
S y/ by Cauchy Cauchy-Schward Schwarz, and therefore the time derivative of ( y)
a y < yel
jus| = (ag + as)lyl e(b + bş) =: alyll + b.
This implies that O.yy alyl so multiplying by the integrating factor e-t and integrating in time yields
v() S sb / (a(t-r) dr =E- (e
where we used that (0)) since the initial condition the trajectories shared. Therefore, the 2 error in time can be approximated by
/
13 |/1 di S a² (2a0 - 26 + +7 T
ulll
establishing the conclusion.
B EXPERIMENTAL AND IMPLEMENTATION DETAILS
This Appendix records additional details related to the numerical experiments in Section 5 For each benchmark problem, a set of trajectories is manufactured given initial conditions by simulating ODEs with known metriplectic structure. The metriplectic forms of these systems given in the subsections below. For the experiments in Table only the observable variables are used to construct datasets. since entropic information assumed be unknown. Algorithm summarizes the training of the dynamics models used for comparison with NMS.
Algorithm 2 Training dynamics models
1: Input: snapshot data X E __xx,, each column æ, x(t., Ms),, target rank 2 1
2: Initialize loss L = 0 and networks with parameters
3: for step in N do
4: Randomly draw an initial condition (lo,, vo.) where k E ns
5: 3 .t; ODEsolve( (2o ãt, (l,...,.l, (1)
6: Compute the loss L((=".... x7), (ā)....... 2())
Update the model parameters θ via SGD
8: end for
For each compared method, integrating the ODEs is done via the Dormand Prince method (do- pris) Dormand and Prince (1980) with relative tolerance 10-7 and absolute tolerance 10-9. The loss is evaluated by measuring the discrepancy between the ground truth observable states 2 and the approximate observable states the mean absolute error (MAE) and mean squared error (MSE) metrics, which are defined in the usual way via the ll norms respectively for true data æ(t.) and approximate data #(t;):
MAE(@, x)
The Σ a(L)) - -(4))) s=1
MSE x)
Σ ((t,)) x(t+)l2-
The model parameters (i.e., the weights and biases) are updated by using Adamax Kingma and Ba (2014) with an initial learning rate of 0.01. The number of training steps is set 30,000, and the model parameters resulting in the best performance for the validation set are chosen for testing. Specific information related the experiments in Section 5 is given in the subsections below.
17

---

Published as a conference paper at ICLR 2025
For generating the results reported in Table implemented the proposed algorithm in Python 3.9.12 12 and PyTorch 2.0.0. Other required information is provided with the accompanying code. All experiments conducted Apple M2 Max chips with 96 GB memory. To provide the mean and the standard deviation, experiments are repeated three times with varying random seeds for all considered methods.
B. TWO GAS CONTAINERS
The metriplectic equations of motion in the two gas containers example are given terms of the state 2 = (q S1 S2) With the energy E(x) (1/2m)pp + E (æ) + (x) and
entropy S(x) Si S2 defined in Section 5 it follows that LV (xr) + M VS where
VS (0 1 1) and
0
0
-1
0
0
0
0
L
M (xx)
0
TT²
T
7
0
-(T)))
T2
A 28
28
12-413
VE(x)=
3
T₃(x) = 0s. Ex(x).
Ti (xx (2
As mentioned the body, the two gas container (TGC) problem tests models' predictive capability (i.e., extrapolation in time). To this end, one simulated trajectory is obtained by solving IVP with a known TGC system and an initial condition, and the trajectory of the observable variables is split into three subsequences, [0, l.ain. (fuain (vall, and (Ival (ua)] for training, validation, and test with 0 < Eraii < Eval < fest-
In the experiment, sequence of 100,000 timesteps is generated using the Runge-Kutta 4th- order (RK4) time integrator with step size 0.001. The initial condition is given as æ = (1, 2, 103.2874, 103.2874) following Shang and Öttinger (2020). The training/validationnnss split is defined by Auain 20, fval = 30, and frett = 100. For fair comparison, all considered models are set to have a similar number of model parameters, 2,000. The specifications of the network architectures are:
NMS: The total number of model parameters 1959. The functions Auii B, K E, S are parameterized as MLPs with the Tanh nonlinear activation function. The MLPs pa- rameterizing Auti B, K Chhol, E are specified as hidden layer with 10 neurons, and the on parameterizing S is specified
hidden layers with 25 neurons.
The total number of model parameters is 2179. The black-box NODE is param- eterized as an MLP with the Tanh nonlinear activation function, 4 hidden layers and 25 neurons.
NODE:
The total number of model parameters is 1954. The functions E and S are parame- terized as MLPs with the Tanh nonlinear activation function; each MI.P specified hidden layers and 20 neurons. The two 2-tensors defining L and M are defined as learnable matrices.
SPNN:
The total number of model parameters is 2343. The functions E and S parameterized as MIPs with the Tanh nonlinear activaton function; each MI.P is specified 2 hidden layers and 30 neurons. The matrices and 3-tensors required learn L and M are defined as learnable 3 x matrices and 3 x 3 x 3 tensor.
GNODE:
GFINN:
The total number model parameters is 2065. The functions and S are parameterized as MLPs with Tanh nonlinear activation function; each MLP is specified as hidden layers and 20 neurons. The matrices to required to learn L and M are defined as K learnable x 3 matrices, where K is set to 2.
18

---

Published as a conference paper at ICLR 2025
B.2 THERMOELASTIC DOUBLE PENDULUM
The equations of motion in this case are given for 1 2
Pi p: -dq (Ei()) + E2(x)), $1 = T2- 82 = x -),
q.
where > is a thermal conductivity constant (set to 1), m, is the mass of the طt spring (also set to 1) and T: = =Os; E, its absolute temperature. this case, qi E IR² represent the position and momentum of the 7th mass, while St represents the entropy of the 1th pendulum. As before, the total entropy S(x) = S1 + S2 is the sum of the entropies the two springs, while defining the internal energies
1 (In 2=) + In + ei -1 A1 = |
E((x)
X 1q2 quls
2
leads to the total energy E(x) = (1 (1/2m1)lpil (1/2m²)lpp) metriplectic with matrix fields L, M, given by
E1 (xx) + (2()).
This system is
0
0
0
0
0
0
0
0
0
1
0
0
0
0
0
(xx)
0
(x))
0
0
0
0
0
0
1
0
0
0
0
T
(T)))
0
0
0
0
TT2 (x)
where 7; = 00; E,
The thermoelastic double pendulum experiment tests model prediction across initial conditions. In this case, 100 trajectories are generated by varying initial conditions that are randomly sampled from
[0.1,1.1] [-0.1,0.1] [2.1. 2.3] [-0.1,0.1]] [-1.9.2.1] [0.9.1.1] [-0.1, 0.1] x [0.9,1.1] x [0. C 1R Each trajectory is obtained from the numerical integration of the ODEs using an RK4 time integrator with step size 0.02 and the final time
40, resulting in the trajectories of split into 80/10/10 for training/valiin walg//aaaaaaiittesss...eeseessse sets. For
length 2,000. The resulting 100 trajectories a fair comparison, all considered models are again set to have similar number of model parameters, 2,000. The specifications of the network architectures are:
The total number of model parameters is 2201. The functions A, B, K, E, are pa- rameterized as MLPs with the Tanh nonlinear activation function. The MLPs parameterizing are specified as hidden layer with 15 neurons.
NMS:
NODE: The total number of model parameters is 2005. The black-box NODE param- eterized as MLP with the Tanh nonlinear activation function, 2 hidden layers and 35 neurons.
The total number of model parameters is 2362. The functions E and S parame- terized as MLPs with the Tanh nonlinear activation function; each MLP is specified as hidden layers and 20 neurons. The two 2-tensors defining and M are defined learnable 3 3 matrices.
SPNN:
GNODE:
The total number model parameters is 2151. The functions E S are parameterized MLPs with the Tanh nonlinear activaton function; each MLP is specified as 2 hidden layers and 15 neurons. The matrices and 3-tensors required to learn L and M defined learnable 3 matrices and 3 3 tensor.
The total number of model parameters is 2180. The functions E and S are parameterized as MLPs with Tanh nonlinear activation function; each MLP specified hidden layers and 15 neurons. The matrices to required to learn L and M are defined as K learnable matrices, where K set to
GFINN:
C ADDITIONAL FIGURES: TWo GAS CONTAINERS AND THERMOELASTIC DOUBLE PENDULUM
presents the ground -ruth and predicted trajectories of q and ý for the two gas containers example, providing confirmation that the learned metriplectic vector field å closely approximates the
Figure 3
19

---

Published as a conference paper at ICLR 2025
true metriplectic vector field. The values pictured are obtained by evaluating the different dynamics functions (either the ground truth or those learned by the different parameterization) along the trajectories of q and p. Figure shows that the proposed NMS method produces the most accurate learned velocities é: p. The MAF between the ground-truth metriplectic vector field and the predicted computes [0.2652, 0. 1355, 0.2285, 0.0546) for [SPNN, GNODE, GFINN, NMS), confirming that NMS yields the lowest error in this quantity. Note that only the observed state variables are included this computation no access the unobserved variables assumed during training.
SPNN
GNODE
GFINN
NMS
0
0.5
-2
-1.0
20
30
10
20
25
4
(b)
Figure 3: The ground-truth and predicted trajectories of / and obtained via evaluating the (learned) dynamics functions.
Figure 4 depicts trajectories of the ground-truth and the predicted position (q). momentum (p), entropy (S), and energy (E) for the thermoelastic double pendulum example. Figures show that the trajectories produced by NMS follow the ground-truth trajectories closer than those produced by the baseline methods. Figures [{g) and 4 demonstrate that the trajectories of entropy S and energy E produced by GNODE, GFINN, and NMS are thermodynamically consistent (i.c., non-decreasing entropy and conserved energy).
and
SPNN
GNODE
GFINN
NMS
63-
(a) Q1x
(b) Qiay
(c) f2x
(d) (2>
-03
(e) Pla
(1) Ply
(g) P2r
(h) P2y
-1-
-2
30
30
(i) S
(i) E
Figure 4: The ground-truth and predicted position, momentum, entropy, and energy for the ther- moclastic double pendulum. Figures illustrate that the trajectories generated by NMS follow the ground-truth ones closer than those of other methods do.
20

---

Published as a conference paper at ICLR 2025
D ADDITIONAL EXPERIMENT: DAMPED NONLINEAR OSCILLATOR
Consider a damped nonlinear oscillator of variable dimension with state x = (q p S) whose motion is governed by the metriplectic system
p p= kssin 1 YP $ == mT
P m
q
Here q, p Rn denote the position and momentum of the oscillator, S the entropy of a surround- ing thermal bath, and the constant parameters m, 7, are the mass, damping rate, and (constant) temperature. This leads to the total energy E(x) = (1/2m)lpl k cos + TS, which is readily seen to be constant along solutions (t). This system has metriplectic expression in terms of E, and the matrix fields
0 0 0
L(x)=
-
M(x)
7/ -/91
0
m
It is now verified that NMS can accurately and stably predict the dynamics of a nonlinear oscillator (q p S) in the case that n 1, 2, both when the entropy observable as well as when is not. As before, the task considered is prediction in time, although all compared methods NODE, GNODE, and NMSknow are now trained on full state information from the training interval, and test enrors are computed over the full state x the extrapolation interval (Ivanid, frest]. which is 150% longer than the training interval. addition, another NMS model, NMS was trained using only the partial state information xx (q. P) and tested under the same conditions, with the initial guess for x:" generated as in Appendix [H] As can be seen in Table 3 NMS is more accurate than GNODE NODE in both the 1-D and 2-D nonlinear oscillator experiments, improving previous results by up to two orders of magnitude. Remarkably, NMS produces more accurate entropic dynamics even in the case where the entropic variable S unobserved during NMS training and observed during the training of other methods. This illustrates another advantage of the NMS approach: because of the reasonable initial data for S produced by the diffusion model, the learned metriplectic system produced by NMS remains performant even when metriplectic governing equations are unknown and only partial state information is observed.
Table Experimental results for the benchmark problems with respect MSE and MAE. The best scores are in boldface.
I-D D.N.O.
T.G.C.
2-D D.N.O.
MSE
МAE
MSE
MAE
MSE
MAE
NMS diff
.0170
.1132
.0045
.0548
.0275
.1456
NMS known
.0239
.1011
.0012
.0276
.0018
.0357
NODE
.0631
.2236
.0860
.2551
.0661
.2066
GNODE
.0607
.1976
.0071
.0732
.2272
.4267
To describe the experimental setup precisely, data is collected from single trajectory with initial condition = (2, 0, 0) following Lee et al. (201)). The path is calculated at 180,000 steps with time interval of 0.001, and is then split into trainingvvaliatiionnees Lval 90 and hest 180. Specifications of the networks used for the experiments in Table are:
sets before using frain 60,
NMS: The total number of parameters is 154. The number of layers for Auri B, Kehnt, E, S is selected from (1,2.3) and the number of neurons per layer from 15,10,15). The best hyperparameters are 1 hidden layer with 5 neurons for each network function.
GNODE: The total number of model parameters is 203. The number of layers and num- ber of neurons for each network is chosen from the same ranges as for NMS. The best hyperparameters are 1 layer with 10 neurons for each network function.
NODE: The total number of model paramters 3003. The NODE architecture formed by stacking MLPs with Tanh activation functions. The number of blocks is chosen from (3,4,5)
21

---

Published as a conference paper at ICLR 2025
and the number of neurons of each MLP from (30,40.50). The best hyperparameters are and 30 for the number of blocks and number neurons, respectively.
E ADDITIONAL EXPERIMENT: HAMILTONIAN NEURAL NETWORK FOR TWO GAS CONTAINERS
As briefly mentioned in the main body, from the definition metriplectic dynamics, restricting M = leads to Hamiltonian systems. In this section, we present the results of training Hamiltonian systems only on the observable variables (g. p) from the two gas container example problems. We use the same dataset used in Section 5.1 and the same experimental setup described in Section B.I
For parameterization the Hamiltonian dynamics, we explicitly consider the canonical Poisson matrix L J
is standard in Hamiltonian neural networks (HNNs), along with an MLP to parameterize the Hamiltonian function H = He p). resulting in the dynamics
-1
Og H -1 0 0 LVH.
(I)
Figure presents the trajectories of position and momentum predicted by the trained HNN. It essentially shows that imposing an inductive bias (i.e., energy conservation), which does not match with the underlying physical system (i.e., dissipation), results in catastrophic failure in prediction. ((gure]5())oofiinnnn that the learned Hamiltonian kept constant over numerical simulation.
1.2
0.32
0.50
-05
022
0.25
(a)
(b)
(c) He
Figure 5: [Hamiltonian neural networks] The ground-truth and predicted trajectories of position and momentum for the gas containers example, along with the learned Hamiltonian. Notice that the incorrect inductive bias of conservation leads to catastrophic failure in network prediction.
ADDITIONAL EXPERIMENT: DAMPED THERMOELASTIC ROD
As an additional benchmark problem, we consider a damped thermoelastic rod, which discretizes an infnite-dimenssonall metriplectic system. Consider a I-dimensional elastic rod, with coordinate s e [0, I]. which evolves following the dynamics defined as:
p
p $
R V'(g)
m
LVE M
(2)
m
Here, denotes a given potential function, 7 denotes a constant controlling the rate of dissipation, m(s) denotes the mass density, and e(s) denotes the internal energy representing the conversion mechanical energy into heat. The energy function given by
1/' ( + v(((s)) + + e(s), 2m((s)
p(s))
E(g.p.e) H( +()) S(()
(3)
where H(g.) p) denotes the Hamiltonian.
22

---

Published as a conference paper at ICLR 2025
GFINN
NMS
-1
-20
(a) (first component)
(b) (last component)
-2.0
-25
-30
-32
100
(c) Pi (first component)
(d) P5o (last component)
Figure The ground-trutt truth and predicted trajectories of positions and momenta for the thermoelastic rod are depicted with = 50.
Figure 7: MSEs (in the logio scale) of predictions obtained by NMS and GFINN for varying N.
In numerical computation, this continuous system discretized, leading to & being represented by vector RN, where N is the number of discretization points. This discretization results in vectorized representation of p. denoted as q: E RN, and a discretized version of Eq. (2), where the state variables are [q, p, S]. In particular, the resulting metriplectic system is given terms of
VS (0 0 1) and
I 0
0
0
V'(g)) P
I
P
L(x)= =
-I
0
0 0)
M(x) 7
M
V
O
OT
0
p
p
In the experiments, we largely follow the setup used in the two gas container experiments, i.e., testing models' predictive capability through temporal extrapolation. One simulated trajectory is obtained by solving an IVP with known thermoelastic rod system, and the trajectory of observed variables is then split into three subsequences, [0, 1. (furain fval. and (feat, Les] for training, validation, and testing with 0 furin (xal < heess In the experiment, the proposed NMS on spatial discretizations of varying size N = (50, 75, 100), which results in systems of dimension { {101, 151, 201), respectively. The performance of NMS is then compared to the performance of GFINNS, which are the current state-of- thr--rrrt in metriplectic learning.
set fuain = 30, vaal 40, and liest 50. We train
Tigure/ōlshore the ground truth and predicted trajectories of the first and the last components of q and for the case of N 50, where it can be seen that the trajectories predicted by NMS are significantly better aligned with the ground truth trajectories compared those of GFINN, particularly in the extrapolation regime (40 < t < 50). Figure shows the results of training GFINN and NMS on thermoelastic rod systems with varying N, where GFINN and NMS instances with comparable model sizes are considered. The results suggest that the proposed NMS is parameter efficient, supporting the theoretical conclusions in Section 3
G SCALING STUDY
To compare the scalability the proposed NMS architecture design with existing architectures, dif- ferent realizations of GNODE, GFINN, and NMS are generated by varying the dimension of the state variables, n = {1,5, 10, 15, 20, 30, 50). The specifications of these models (i.e., hyperparameters) set that the number of model parameters is kept similar between each method for smaller values ofn. n. For example, for n = 1, the number of model parameters is ~00,000 for each architecture. The results in Figure |(a) confirm that GNODE scales cubically in while both GFINN and NMS scale quadratically. Note that only a constant scaling advantage of NMS over GFINN can be seen from this plot, since is fixed during this study.
is also worthwhile to investigate the computational timings of these three models. Consider- ing the same realizations of the models listed above, i.e., the model instances for varying
23

---

Published as a conference paper at ICLR 2025
{1,5, 10, 15, 20, 30, 50), 1,000 random samples of states {x(0) 1,000
then fed to the dynamics function [(x(*) ( (((()) M (x Ti=1 are generated. These samples
for i 1, ...... 1000, and the computational wall time of the function evaluation via py Torch's profiler API is measured. The results of this procedure are displayed in Figure 8(b) Again, it is seen that the proposed NMSs
require less computational resources than GNODEs and GFINNs.
x
GNODE
GFINN
NMS
3.2
5.0
Campatetioonl lime
mode
4.8
4.6
4.4
10 15 20
30
50
43
4,4
4.5
4.6
4.7
number of states, (n)
number oodd peremetera
(a) State dimension n versus number of model parameters
(b) Model parameters versus wall time microseconds
Figure 8: A study of the scaling behavior of GNODE, GFINN, and NMS.
H DIFFUSION MODEL FOR UNOBSERVED VARIABLES
Recent work in Lim et 2023) suggests the benefits of performing time-series generation using diffusion model. This Appendix describes how this technology is used to generate initial conditions for the unobserved NMS variables in the experiments corresponding Table More precisely, we describe how to train a conditional diffusion model which generates values for unobserved variables a given values for the observed variables x
Training and sampling: Recall that diffusion models add noise with the following stochastic differential equation (SDE): dx(t) f(t, x())dd + g())dw EE [0, 1],
where w E dim(() is a multi-dimensional Brownian motion, f(t, : Rdim(x) - ddi((x) a vector-valued drift term, and g j0, - IR is scalar-valued diffusion function.
For the forward SDE, there exists a corresponding reverse SDE:
() = [(., x()) (1) (oggoo((pp)))ddd gg))))++++())) ((),dw
which produces samples from the initial distribution at This formula suggests that if the score function, V log p( (((x)),, is known, then real samples from the prior distribution p(x) ~ N o?) can be recovered, where (u, a vary depending on the forward SDE type.
In order for model Mg to learn the score function, it has to optimize the following loss:
L(0)= = (x x(t)) - Ve(e; logp(x(t))]]
where / is uniformly sampled over [0, 1] with appropriate weight function X(1) [0. 1 + IR. However, using the above formula computationally prohibitive. Thanks to Vincent (20TT), this loss can be substituted with the following denoising score matching loss:
(0) = E, (o) II| Mo x()) Vx( log (o)x((())x()))]
Since score- based generative models follows a certain Gaussian distribution Särkkä and Solin (2019). and therefore the gradient term Vx(o) log (()) can be analytically calculated.
affine drift term. the transition kernel ((x()))x(0))
24

---

Published as a conference paper at ICLR 2025
Experimental details On the other hand, the present goal is to generate unobserved variables æ" given values for the observed variables x" = (q. p), ie., conditional generation. Therefore, our model has to learn the conditional score function, V (1) log p(=" x" (t) z)). For example, in the damped nonlinear oscillator case, S(t) is initialized
perturbed E [0, 1). from which the model takes the concatenation q, p. (t) as inputs and learns conditional the score function Ss(e) log(S(t))q, p).
For the experiments in Table 3 diffusion models are trained to generate x² variables on three benchmark problems: the damped nonlinear oscillator, two gas containers, and thermolastic double pendulum. On each problem, representative parameters such varied, with the total number of cases denoted by N. Full trajectory data of length 7 is then generated using standard numerical integrator (e.g.. dopri5), before it is evenly length L. Let V, U denote the total number of variables and the number of unobserved variables, respectively. It follows that the goal generate U unobserved variables given V U observed ones, i.e., the objective is to generate data of shape (NT) L, L, U) conditioned on data of shape (NT/L, U). After the diffusion model has been trained for this task, the output data is reshaped into size (N. W, U), which is used to initialize the NMS model. Note that the NODE and GNODE methods compared to NMS in Table 3
or thermal conductivity
into [T/L.] pieces of
use full state information for their training, i.e., æ Ø in these cases, making it comparatively easier for these methods to learn system dynamics.
As in other diffusion models e-g. [Song et al. (2020), U-net architecture is used, modifying 2-D) convolutions to I-D ones and following the detailed hyperparameters described in [Song et al. (2020). Note the following probability flow ODE seen in Song et al. (2020):
1 f(t. x(t) - (L) di,
dx(t)
Although models trained to mimic the probability flow ODE do not match the perofrmance of the forward SDE's result in the image domain, the authors of et al. (2023) observe that the probability flow ODE outperforms the forward SDE in the time-series domain. Therefore, the probability flow ODE is used with the default hyperparameters of [ong et al. (2020).
25


Reviews:
[{"review_id": "GbKRiotZf1", "reviewer": "Reviewer_gp5d", "summary": "In this work, the authors present a neural network model (NMS) for learning the dynamics of metriplectic systems from trajectory data. It is based on a parameterization of certain scalar- and matrix-valued fields using neural networks that differs from prior works that enforce hard constraints on the degeneracy condition of metriplectic systems and results in a model size that scales quadratically in the system dimenion. The proposed method is demonstrated empirically to perform well in the learning of two low-dimensional metriplectic systems compared to prior methods.", "questions": "See questions in the \"Weaknesses\" section above regarding physics background, training strategy, and alternative methods.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The proposed neural network model for parameterizing the metriplectic system is novel to my knowledge, and it is well-motivated by the theoretical results on metriplectic operators and approximation errors presented in Section 3.2 and 3.4. Compared to several methods from prior literature, the proposed NMS method holds an advantage in terms of model size as well as empirical performances. The writing of the paper is clear overall.", "weaknesses": "**Physics background and relation to Hamiltonian systems:** Having relatively little prior knowledge about metriplectic systems, I would appreciate an expanded introduction of its physical motivation and some concrete examples for illustrating the general governing equation on the top of Page 2. For example, what do L and M look like in the two examples used for the numerical systems, or in general Hamiltonian systems -- will L be the \"J\" matrix in Hamiltonian systems (padded with zeros for the extra dimensions) and lose its independence on the state x? \n\n**On the decoupled block-wise structure:** The authors mentioned prior works such as Ruiz et al. (2021) and Xu et al. (2022 & 2023) which proposed to parameterize metriplectic systems assuming a decoupled block-wise structure and discussed their inability to express general metriplectic dynamics. But I would appreciate some examples of these more general metriplectic systems encountered in practice. In particular, do the two systems studied in Section 5 admit the decoupled block-wise structure? If so, it would be reasonable to expect that those more restrictive methods are also tested in the experiments as baselines.\n\n**Comparison with Hamiltonian learning:** If one ignores the entropy states and focuses only on the observable states (positions and momenta), it looks like the two examples in Section 5 can just be learned as Hamiltonian systems. In that case, does the NMS method reduce to e.g. the Hamiltonian Neural Network (HNN) from [1]? If not, perhaps HNN should also be added as a baseline method to compare against in Table 2.\n\n**Choices of the initial unobserved states:** Regarding the initial unobserved states in batch-wise training. The authors mentioned two interesting strategies to handle the missing initial unobserved entropy states (first question: which one was used in the experiments whose results are reported in the main text?), one of which is to assume that the entropy values increase linearly in time. Is there a justification behind this? (A further question seems to be: are the entropy states and their dynamics uniquely determined by the observable states?) Besides the two strategies considered by the authors, the initial state optimization proposed in [2] for learning Hamiltonian systems might also be an alternative to consider.\n\n**Test systems are low-dimensional:** Another limitation, as acknowledged by the authors, is that the empirical performance of NMS on larger-scale, realistic metriplectic systems has not yet been demonstrated.\n\nA minor issue: misplaced parentheses for citations on Page 1 (probably due to mixing up \\citep with \\citet).\n\nReferences:\n\n[1] Greydanus et al., \"Hamiltonian Neural Networks\", NeurIPS 2019.\n\n[2] Chen et al., \"Symplectic Recurrent Neural Networks.\", ICLR 2020.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rBOkfXc0Z2", "reviewer": "Reviewer_ZdKG", "summary": "Metricplectic systems model systems that satisfy energy conservation as well as entropy constraints and can used to model thermodynamic and other system that require such constraints. In the context of machine learning it is of interest to learn such systems from data.\nPrior methods like GNODE exchange the problem of enforcing degeneracy constraints with the problem of enforcing symmetry constraints which underdetermines the problem and at the same time have a redundant parameterization of the problem which leads to high (cubic) complexity.\nThe proposed method exploits structure in the tensor fields to reduce the number of parameters.\nThe paper demonstrates structure in the degeneracy constraints beyond what can be captured by symmetry constraints leading to a lower parameter parameterization.\nA further result shows that the proposed formulation universally approximates metricplectic systems that are non-degenerate and shows a generalization result.", "questions": "Besides lower complexity the method achieves better loss values compared with all the other methods. Why is that even on the simple two-gas problem the other methods, say GNODE, do not achieve the same accuracy even with full state information? Is it due to a restrictive parameterization assumption?  Some intuition would be useful.", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The proposed approach appears to be novel and more general than prior work and allows for all metriplectic data to be approximated simultaneously. Prior work assumes special forms to satisfy the constraints on entropy and energy. This allows the method to model a greater class of systems. The modeling assumptions are also quite mild and only require non-zero gradients for energy and entropy.\n\nThe claims and objectives of the paper are clear. The literature review is quite extensive and the experiments treat a number of prior baseline and validate the claims of the paper of better generalization at lower complexity.", "weaknesses": "The paper is difficult to read and there is not a lot of intuition to understand the source of the complexity reduction in Lemma 3.2 and Theorem 3.4 for a non-expert.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0l5GoqI7cj", "reviewer": "Reviewer_ufvB", "summary": "The paper presents a new parameterization scheme for learning metriplectic systems. The parameterization is constructed based on computations using properties of the exterior algebra. The constraints can be seen as hard constraints instead of soft constraints. Proof of the construction and proof of universal approximation are provided as well as proof of growth rate of error with respect to time. An algorithm has been proposed, and numerical experiments are carried out to justify the effectiveness of the proposed scheme.", "questions": "0. How original is the approach to using exterior algebra to parameterize hard-constrained structures? The reviewer believes some papers that deal with structure preservation using exterior algebra exist in the literature. If the authors also know such papers, the authors could have remarked on that.\n\n1. In Remark 3.5, it is claimed that \"the proposed parameterizations for L, M are not one-to-one but properly contain the set of valid nondegenerate metriplectic systems\". Are the authors trying to say that the proposed parameterizations are not injective but subjective onto the set of possible nondegenerate metriplectic systems? I think so from the proof. The author could clarify what they mean by \"properly contain\".\n\n2. Again in Remark 3.5, the authors said that the Jacobi identity is not enforced in the algorithm, which causes the parameterization to be not one-to-one. It is not clear to the reviewer how are these related. From the reviewer's viewpoint, the parameterization is not one-to-one because the construction is via orthogonal projection, then of course there will be more than one parameterizations that give the same metriplectic system. However, it is unclear how this is linked to the Jacobi identity not being enforced. The author should clarify this point. Besides, in the last sentence of Remark 3.5, it is said that the structure and energy conservation cannot be simultaneously preserved, which, as far as the reviewer knows, applies in the case of symplectic integrators, but here the context is not symplectic integrators. The authors could clarify this point.\n\n3. In page 6, it is claimed that \"the exterior algebraic expressions in Lemma 3.2 require less redundant operations than the corresponding metricized expressions from Theorem 3.4, and therefore the expressions from Lemma 3.2 are used when implementing NMS\". The author should clarify how the construction in Lemma 3.2 requires less redundant operations.\n\n4. On top of page 8 (lines 381-382), the first strategy to deal with unobserved states is to assume a line between the all 0 vector to the all 1 vector. Can this be justified?\n\n5. In Algorithm 1, the input, xs = x(ts, μs), but what is μs? It seems that it is not introduced.\n\n6. There is a typo \"Lemmata\" in line 641.\n\n7. The algorithm in the end still needs structure-preserving integration, which means for each system under consideration, a structure-preserving integrator needs to be applied. In the numerical experiment, how are the integrators chosen?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The subject being investigated is of significant importance in AI for Science. The topic is closely related to the so-called structure-preserving machine learning. The paper is in general well written.\n\n2. The idea of the proposed parameterization is novel. The construction of the parameterizations that satisfy the hard constraints is obtained through exterior algebra computations. The construction can be seen as obtained by orthogonal projection, which is quite natural. In Theorem 3.4, the statement is \"if and only if\", which is a nice result for machine learning purposes. There are universality analysis and growth rate of error analysis, which are theoretically convincing.\n\n3. The numerical experiments are carried out quite extensively, together with comparisons with other machine learning schemes in the literature.", "weaknesses": "The main weakness of the paper is that, some of the remarks and claims are not clearly explained. The reviewer will state the details in the \"Question\" session.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wYqyYfQoUy", "reviewer": "Reviewer_QHfC", "summary": "This paper introduces a novel framework for learning metriplectic dynamics, which describe systems that conserve energy while generating entropy. The key contribution of the proposed approach lies in its design, which ensures both energy conservation and entropy stability. Mathematically, the paper presents a parameterization method that employs neural networks to model four functions, $L$, $E$, $M$, and $S$, such that $L \\nabla S = M \\nabla E = 0$. Here, $L$ is an antisymmetric matrix-valued function, $M$ is a symmetric matrix-valued function, and $E$ and $S$ are scalar functions.\n\nUnder the assumption that $\\nabla E,\\nabla S\\neq 0$, the proposed parameterization of metriplectic systems requires less learnable scalar functions than existing methods. Numerical results show that  the proposed approach exhibits superior accuracy that existing methods for  learning metriplectic dynamics from data.", "questions": "1. While it is evident that the proposed parameterization requires fewer learnable scalar functions, this alone does not directly imply superior performance. Could the paper provide an explanation of why the proposed approach achieves higher accuracy?\n\n2. The paper claims that the core advantage of the proposed method is its efficiency. Could the paper provide evidence demonstrating that the method requires less computational time or other resources, such as memory?\n\n3. Could the paper provide that specific formula of metric use to compare the performance?\n\n4. As the true governing functions (the right hand side of the ODE) are known, could the paper show the results of learning these governing functions?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1.  The emphasis on energy conservation and entropy stability is crucial for modeling realistic physical systems. The authors demonstrate that their approach maintains these properties, which is essential for ensuring that the learned models are physically meaningful and applicable to real-world scenarios\n\n2. The proposed parameterization of metriplectic systems requires less learnable scalar functions than existing methods. \n\n3. The paper is well-structured and clearly written, making complex concepts accessible to a broader audience. The examples and comparisons with previous methods help to illustrate the advantages of the proposed approach effectively.", "weaknesses": "1. Although the paper presents empirical results demonstrating the model's performance, the range of examples may be limited. Specifically, in both examples, the entropy is given by $S = s_1 + s_2$, resulting in a constant gradient $\\nabla S = 0 $. This contradicts the core assumption in the paper that $ \\nabla S \\neq 0 $. \n\n2. The requirement that the metriplectic system being approximated is nondegenerate—i.e., the gradients of energy and entropy must not vanish ($\\nabla E, \\nabla S \\neq 0$)—may limit the applicability of the method. Although this is claimed to be a mild condition, both of the examples considered in the paper contradict this core assumption. In addition to the entropy $S$ having a zero gradient ($\\nabla S = 0$), the energy in the investigated examples is also degenerate. From the reviewers' understanding, steady-state is a fundamental concept in physical systems, characterized by the energy reaching an extremum, meaning the energy gradient is zero ($\\nabla E = 0$). The reviewers are not aware of any physical systems with non-zero energy gradients. To better demonstrate the applicability of the proposed method, the reviewers recommend that the paper provide a detailed discussion of systems that satisfy the key non-degeneracy assumption. Additionally, it would be beneficial to include examples that adhere to this assumption to validate the effectiveness of the algorithm.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)