Latent Mixture of Symmetries for Sample-Efficient Dynamic Learning
Authors: Haoran Li
CHENHAN XIAO
Muhao Guo
Yang Weng
Venue: NeurIPS 2025 poster ()
Keywords: dynamic learning; sample efficiency; symmetry preservation; Lie transformations; geometric deep learning
Primary area: machine_learning_for_sciences
TL;DR: This paper proposes a novel geometric deep learning model to preserve the underlying mixture of symmetry transformations for sample-efficient dynamic learning

Abstract: Learning dynamics is essential for model-based control and Reinforcement Learning in systems operating in changing environments, such as robotics, autonomous vehicles, and power systems. However, limited system measurements, such as those from low-resolution meters, demand sample-efficient learning. Symmetry provides a powerful inductive bias by characterizing equivalent relations in system behavior to improve sample efficiency. While recent methods attempt to discover symmetries from data, they typically assume a single global symmetry group and treat symmetry discovery and dynamic learning as separate tasks, leading to limited expressiveness and error accumulation. In this paper, we propose the Latent Mixture of Symmetries (Latent MoS), an expressive model that captures symmetry-governed latent factors from complex dynamical measurements. Latent MoS focuses on dynamic learning while locally preserving the underlying symmetric transformations. To further capture long-range temporal equivalence, we introduce a hierarchical architecture that stacks Latent MoS blocks across multiple time scales. Numerical experiments across diverse physical systems demonstrate that Latent MoS significantly outperforms state-of-the-art baselines in interpolation and extrapolation tasks while offering interpretable latent representations suitable for future geometric and safety-critical analysis.

Latent Mixture of Symmetries for Sample-Efficient Dynamic Learning
Haoran Li
Chenhan Xiao Muhao Guo Yang Weng School of Electrical, Computer and Energy Engineering
Arizona State University, Tempe, AZ 85281
{1haoran, cxiac20, mguo26, yweng2) edu
Abstract
Learning dynamics essential for model-based control and Reinforcement Learn- ing in engineering systems, such robotics and power systems. However, limited system measurements, such as those from low-resolution sensors, demand sample- efficient learning. Symmetry provides powerful inductive bias by characterizing equivariant relations in system states to improve sample efficiency. While recent methods attempt discover symmetries from data, they typically assume single global symmetry group and treat symmetry discovery and dynamic learning as separate tasks, leading to limited expressiveness and error accumulation. In this paper, we propose the Latent Mixture of Symmetries (Latent MoS), an expressive model that captures a mixture of symmetry-govent dynamical measurements. Latent MoS focuses on dynamic learning while locally and provably preserving the underlying symmetric transformations. To further capture long-term equivariance, we introduce a hierarchical architecture that stacks MoS blocks. Numerical experiments in diverse physical systems demonstrate that Latent MoS outperforms state-of- -fate-of-the-arr baselines in interpolation and extrapo- lation tasks while offering interpretable latent representations suitable for future geometric and safety-critical analyses.
latent factors from complex
1 Introduction
Learning dynamic models is fundamental to modern physical and control systems, especially when exact governing equations are unavailable due to incomplete records, device aging, or privacy constraints (121 Accurate dynamic modeling enables robust control and Reinforcement Learning 3. and supports model-based analysis for performance and safety guarantees [4] []. However, sample-efficient learning remains challenge in practice, where data scarcity arises from low- resolution meters, communication failures, or storage limitations. For example, smart grids, SCADA (Supervisory Control and Data Acquisition) systems typically report measurements every 2 ~ 10 seconds, while critical events may occur within milliseconds [6] 77.
To mitigate data scarcity issues, existing methods can be grouped into four main categories. The first category interpolates the low-resolution data by assuming sparsity [8, low-rankness [. or manifold geometry [1] The second category imposes known physical laws as constraints on the The third category utilizes Bayesian frameworks that demand accurate prior distributions 15.15) In general, all of the above methods require certain domain-specifte assumptions or knowledge. The fourth category requires little prior knowledge and leverages Ordinary Differential Equation (ODE) solvers to construct continuous ODE process, e.g., the family of Neural ODEs 19]. Although Neural ODE- based methods are well-suited for sparse data, they often employ generic architectures such multilayer perceptrons [16] 20 This lack of inductive bias hinders generalization, leading to overfitting rather than capturing true dynamics.
learning process IT
To address these limitations, propose to use the underlying symmetries in dynamical systems. Symmetry, defined as a group of transformations that leave a system's 's behavior or properties equivari-
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

ant (2,, is prevalent across various domains. For instance, rotational, reflectional, and translational symmetries are leveraged in robotics [22). 3D vision data (23], power systems (24. fluid dynamics 25 and networked systems 266 Why can symmetry increase sample efficiency? Intuitively, sym- metry transformations enable a system state to represent a large set of equivariant states, reducing the amount of data needed for training. A rich line of work has shown the improvements of enforcing symmetry priors into equivariant neural networks (27-30) with some applications to sample-efficient RL 331333 Some other works investigate symmetry- -based data augmentation [34] and regularization
35 for dynamic modeling. They rely on clear symmetry knowledge. For systems with unknown
symmetries, several recent methods propose to discover symmetries from data 38 42).
However, these methods nnur large modeling errors when applied to complex dynamical systems due to the following limitations: (1) Decoupled learning and accumulated errors. Symmetry discovery and dynamic modeling are typically treated as separate processes, leading to error propagation. (2) Limited expressiveness of symmetry representations. Most approaches assume that single (linear or nonlinear) symmetry group governs at least a segment of data evolution. In practice, however, the underlying transformations exhibit substantial variability: both in which symmetries (often multi-modal and context-dependent) and when they occur (ranging from short to long terms across distant segments). To summarize, dynamical systems often do not adhere to strict, rigid symmetry constraints, but instead exhibit forms of symmetry breaking. This means that while approximate symmetry structures are present, they may be only partially preserved vary across regimes. Our work identifies symmetry breaking in two orthogonal axes: a mixture of groups and short- or long-time dependence (see Section 2.1). The mixture of groups appears in literature [43] and the locally adaptable symmetry breaking is discussed in 46). They are independently discussed, but method addresses both in unified framework.
relevant
To address these challenges, we introduce the following modeling strategies. First, grounded in Lie theory [47], we propose a dynamic learning model that preserves local symmetries in the latent space, thereby eliminating the need for symmetry discovery algorithms. Second, rather than relying on single symmetry group, we consider mixture of transformations inspired by the expressive cexture-based model [48]. We refer to the overall framework as the Latent Mixture of Symmetries (Latent MoS). Latent MoS encodes system evolution mixture of latent flows, each governed by a distinct Lic group. This design enables samplc-efficient learning, as each flow captures a local symmetry transformation that defines a neighhorhood of equivariant observations. Finally, to extend beyond local equivariance, we develop a hierarchical architecture by stacking multiple MoS blocks with different temporal resolutions. Namely, a higher level of MoS is responsible for modeling longer-term symmetries. In summary, our contributions are as follows:
Latent MoS Framework:
We propose the Latent Mixture of Symmetries (Latent MoS), model that achieves high sample efficiency.
novel
Theoretical Foundations: We establish guarantees the approximation capacity of Latent MoS and its ability to preserve symmetry structures induced by Lie group transformations.
Empirical Validation: We demonstrate that Latent MoS outperforms state-of-the-an models across variety of dynamical systems, particularly in low-resolution scenarios.
2 Preliminaries
Let x E X C 18" denote the system state. The discretized dynamics of the system are given by a(ti+1)= f(x(t))) where f : X - X' is unknown dynamic function. The objective is to learn a model that approximates f using historical observations (x(+)))eN, which span the time interval from to to tN but can be low-resolution and irregularly sampled. For systems with inherent uncertainty, the goal becomes learning the transition probability distribution p(x(t++1) | xxt;)). For notational simplicity, we focus on the deterministic setting and use f in the following derivations. In addition, the goal can also be extended to multi-horizon prediction, e.g., long-term forecasting.
2.1 Equivariance for Efficient Dynamic Learning
Intrinsic data scarcity challenges the learning process. Hence, we introduce the equivariance: Definition Let be a symmetry group and a nonlinear group action be n'(g, :) G -+ X. Then, the dynamic governing function f is G-equivariant if E G. G, E X: (IT'(g.x((..))
(I)
2

---

When (g.-) is known or learned, existing methods use it to augment data 34 or impose symmetry constraints as regularization during training [37], thereby improving sample efficiency. While many
recent models adopt this equivariance concept 32 40 37
we propose the following symmetry-
breaking generalizations to suit more complex dynamical behaviors:
Temporal Dependence. (g. may depend on time and past observations, requiring the learning model be temporally adaptive. This extension is described in Section 3.2
Mixture of Groups: Multiple symmetry groups [GI..... Gx J. such as the orthogonal group, the similarity group, the Fuclidean group, and the affine group, may simultaneously influence the system dynamics. The mixture model is discussed in Section 3.2
Short- and Long-Term Equivariance: Symmetry relations may exist across multiple temporal scales. For example, xtt,) and n' (g. xxt))) may correspond to states that either temporally close or far apart, depending on the choice of E G. However, traditional symmetry discovery methods often fail capture long-term symmetries due to limited temporal modeling. Our approach to multi-scale symmetry is discussed in Section 3.3
2.2 Nonlinear Group Action Decomposition
The nonlinear -)) is hard to analyze. 40 provides effective decomposition for compact Lie group G and a continuous group action [g (g E G):
Tg(g,' .) = Saec o [g(g) o fencs
where fenc : :X' - 2 and Six: state space X' and latent space Z TR n, G GL. is a group representation such that [g)g) E m is matrix that can transform a latent vector 2 E Z by matrix multiplication. GL is the general linear group. Under mild assumptions, authors in 40 prove the universal approximation of a nonlinear group action using the decomposed format in Equation (2). Consequently, we only need investigate the equivariant relation on Z, which can be written as follows:
X' are encoder and decoder neural networks between the
Vg E G, z (44) Z, (()==(((_1)) [s(T)())=((4)),
where f2= = Sene o fo Jaee is the dynamic function in Z. It's easy to verify that when Equations 2
and (3 hold and the encoder and decoder are well-trained (i.c., face feae = I, where I is the identity matrix), the equivariant relation in Equation also holds.
2.3 Time-Series Latent Space Construction
To construct the latent space for time-series measurements, we introduce Latent ODE 18 since it can
explicitly model a continuous ODE process in 2, capable of interpolating missing data at arbitrary times. In Latent ODE, ODE-RNN is the encoder (fene) to process (z((:))ieMs and generate a latent vector 2 (to). The decoder (facc) employs a Neural ODE [16] to reconstruct the input state sequence and predict the future state. Mathematically, fen: approximates the posterior:
= Eo), Po: Σn = fenc ((x((t;) (f=(tt))ieN.))
where @ is the parameter set for fene that approximates the mean and the variance of the posterior The encoding process is shown on the bottom left of Fig. The decoder uses z(ln) as the initial latent state and solves the ODE problem for inference:
=(tu) ~ p(z(to)), z(ta)= = ODESolve(ha, z tt)), xtts)
(t(=(tt))((t)))
(5)
where 0 is the parameter set for Saec. p(z((o)) is zero-mean Gaussian, h 2 + 2 is neural network to represent the derivative i(),, and pe includes several MLP layers to approx- imate the conditional distribution. Thus, face consists of both ha and Pa. The overall training maximizes the evidence lower bound (ELBO): ELBO(Ø, () = Ez(to)<go log po (p(((tt)))x)) KL[9 (z ((t))(t(((tttt heN, )lp()((t))] For the deterministic scenario, we can also minimize the Mean Square Error (MSE). The bottom right of Fig. shoows the decoding process.
3 Method
In this section, we introduce dynamic learning model that intrinsically preserves complex sym- metries (Section(..) (.T) without symmetry discovery. By embedding geometric priors into the latent dynamics in Equation (3), we enforce physically grounded, symmetry--peseervigg structure with theoretical guarantees (Section(3.) This foundation extends to a mixture model (Section 3.2) and generalizes to multi-scale temporal equivariance (Section 3.3
3

---

3.1 Short-term Equivariance Preservation for A Single Latent Symmetry
We begin with the ideal case where the dynamics are governed by symmetry transformations of a single Lie group G. Then, we examine the functional class of dynamic models that have the potential to preserve equivariance. An expressive class is the dynamic function that can commute with the transformation in G. For example, the rotating dynamies in a 2D circle can commute with the rotational action in SO(2), and thus are -quiivaaara SO is the special orthogonal group. In general, to study the commutative property, we demand the dynamic itself to be Lie group action. Specifically, we consider the nonlinear Lie group action (h, .) such that:
(a(h, = Sdee o (h) o Senc. 2(14+1) = ==(((())) == (<(())z(4);
where we use the subscript d to respect the dynamics. h E H and H is another Lie group that represents dynamics. By the universal approximation in Section 2.2] (h, -) can be decomposed, as shown in the first equality in Equation (6). Thus, we only need to Tocus on the property of the matrix transformation (h) operating in Z. The following lemma illustrates how (h) can commute with [g) to preserve the equivariance.
Lemma 1. Assume that G and H S Gre Lie groups whose elements are defined in Equation (6) and Equation 3), respectively. Define the centralizer of H in G as: Ca(h) == {g E G| (g) (h)) Ta(h) n (g)}. J. If (h) is nontrivial (i.e., contains elements other than the identity), then Vg Ch the relation for (g(g) in Equation
B is preserved.
The proof is in Appendix A.1 The commutative property in the centralizer (h(h) defines an equivalent format for G-equivariance. In Lemma [. the first condition (h E H C G) is ideal by assuming globally constant dynamic function and single Lie group G. Hence, in Section 3.2 we design the learning model capable of capturing mixed and time-variant symmetries. The second condition (Cc(h) is nontrivial) doesn't 't always hold. Thus, we establish the following conditions.
Proposition 1 (Sufficient Conditions for Nontrivial Centralizer). Let H C Aff(m) be an affine Lie subgroup whose elements are represented in homogeneous coordinates as affine transformations:
An bn
(t)
Ta (a(h)=(1) = 0
1
where Ad GL(m). bn E RmM and 2tt [z(t), 1|T E <m=+1 to denote the augmented latent vector in homogeneous coordinates to support affine transformations. Suppose that for Vg. h E H, it holds that A, Ag. Agbn = bb and Anby= by. Then we have H C (C) In particular, (a(h) is nontrivial and contains at least all of elements in H.
The proof is provided in Appendix A.2 In Equation we adopt the upper-triangular affine matrix form because offers a unified and compact representation for broad class of Lie transformations, including rotation, scaling, and translation, and their higher-order multiplications. Subsequently, show in Corollaries 3 that they satisfy the commutativity conditions in Proposition [ thus having equivariance guarantees.
Corollary 1 (Equivariance of Planar Rotation Transformation). Let in and U2 be orthonormal vectors and P := [un [z] E R=*x Consider the planar rotation transformation: grot
I'm + RRg- PT 0
[cos 0 -sin 0
where Ro sin 0 cos 0
E (SO(2), Im is the m x m identify
matrix. Then, Tot and its Lie subgroup H C (0mm) satisfy the conditions in Proposition 1
The restriction planar rotation arises because general rotations are non-commutative. E.g., die rotates along the x-axis and then z-axis, or along the z-axis and then x-axis, will cause different results. In contrast, planar rotations (fixed rotation axis) are commutative. The result is rigorously proved in Appendix A.3] meaning that the state should rotate along one axis at least locally to preserve rotation symmetry. If rotation continuously changes the axis. symmetry is completely removed. Although Re is two-dimensional, the learnable projection matrix can lift the dimension to increase the expressiveness of frot.
Corollary 2 (Equivariance of Translation and Scaling Transformation). Denote the translation Im v.
dynamics: Ara = 0
and its Lie subgroup H c E(m), where E(m) is an Euclidean group.
1
They satisfy the conditions in Proposition Denote the scaling dynamics: a=ca [diag(7) 0"
0
4

---

and its Lie subgroup H C Sim(m), where Sim(m) is similarity group. They satisfy the conditions in Proposition L
Corollary 3 (Equivariance of Second-Order Composed Transformations). The second-order composi- tion of the planar rotation, translation, and scaling transformations defined in Corollaries
7 satisfies sea
the commutativity conditions in Proposition [ Specifically, for any #()), (2) , a
we have that a (2) and its Lie subgroup H (m) satisfy the conditions in Proposition
In corollary ! to 3 we add : to the matrix transformation to imply that it's a candidate dynamic approximation as we don't 't know the ground truth. The proof is provided in the Appendix AA.3 to A.5
3.2 Latent Mixture of Symmetries with Preserved Equivariance
Proposition identifies a rich set of Lie groups that may appear in complex systems. In this section, we introduce our model to capture the mixture of diverse symmetries and maintain high representational power. Fig. D demonstrates the main architecture. The core is a Mixture-of oxf-etxerts (MoH) model, which employs gating mechanism 49 to automatically select the appropri- ate Lie group transformations. In addition, we make the MoS module üime-varian for time-dependent symmetries. Thanks to the model's extensibility, new Lie group ex- perts that are not characterized by Propo- sition | can be flexibly incorporated. We leave such extensions for future work. Specifically, let h E R be the output of gating neural network, activated by Softmax function. Treating each symme- try transformation as an expert, the kth en- try h[k]] provides the weight for the expert.
Mixture of Symmetriies (MoS)
#rot
# tra
A am
tra
sca
(ta)
a( Ar)
(fa)
Eg
ODE-RNN
s (1241)
x(fN)
rtoo
z(f,)
Ar
Time
EN
Figure 1: The Latent MoS framework, Compared to Latent ODE, Latent MoS structures the latent dynamics by using an MoE to select symmetries. Different colors in the MoS box imply different gate weights.
Then, Latent MoS linearly mixes the latent flows 2n (t++1) such that:
K Σ [R] (2((1)1) = Σ h(k) t 2(4))
K
8(++1))
(8)
where f& is the htth expert, parameterized to represent Lie group action with a geometric structure from Proposition[] We adopt a linear mixture because real-world systems often exhibit multiple latent symmetry flows acting simultaneously (e.g.. damping and oscillation), which cannot be represented by single Lie group action. The linear mix allows the gating network to combine these expert transformations into expressive latent dynamics, while still preserving the interpretability of each individual symmetry component.
In Equation (8), h and म. are made of time-dependent. Specifically, let the time interval [lo. LN] be evenly divided into T. disjoint subintervals:
[Lo, [N]
U [10) ((++1),, t=0
where (0)= lo + AT, AT IN to
T.
Then, for each subinterval, Rt and h are piecewise constant and locally dependent on both the initial time t()) and the latent state 2(((),, i.e.. designs for मे are shown below. We eliminate the superscript (!) for simplicity.
)..., h := (((1)))((())) and ft. := ft. (t()), )() The specific
Rotational symmetry. The group action æ (t) E SE(m), where SE(m) is a special Euclidean group. To parameterize froo (1, (()), by Corollary 1 we first learn a projection matrix P(1, (()) E
Rmx2 whose columns form orthonormal basis of 2D subspace in Rm Specifically, neural network outputs an unconstrained matrix (t, (t)) | which is then orthonormalized via QR decomposition: (()) = ((1) =(1),, and we define P(t, z()) == Q((,, 2()). We also parameterize planar rotation matrix Rg e SO(2) using learnable scalar angle A(t, =(t). The
5

---

overall transformation then defined as: (1, =()) Im + P(Rg 12)
E SE(m),
0
which applies learned planar rotation within dynamically selected 21) subspace while leaving the rest of the space invariant and preserving the affine structure.
Translational symmetry. The action t (t, (t)) E B(m), where E(m) is an Euclidean group.
Given a learnable velocity vector v(t, (t)) E I""" we define: A (t, (t)) I v(4, ((,=((()))
Scaling symmetry. The group action #000 (to (t)) E Sim(m), where (im(m) a similarity group. For ratio vector 7(t, xt)) E R", we have A5mm (t, x diag (7, (()))
0
Second-order multiplications. To capture coupled symmetry effects, we consider the multiplication of multiple Lie group actions. For example, multiplying translational action (t, with scaling action 7 (t, (t)) yields a new transformation f f (t, z (t)) = म "t (t)) - (t, (t)). More generally, the multiplication of Lie group actions remains valid Lie group action when defined over a closed subgroup (e.g., SE(m), E(m), (im(m). etc.).
Fig. Iillustrates how the gating mechanism enables a mixture of symmetry transformations to govern the latent dynamics. Compared to the Latent ODE model introduced in Section 2.3 our Latent Mixture of Symmetries (Latent MoS) replaces the unstructured ODE update step (i.e., ODESolve in Equation ( with a structured,
formulation as given in Equation (8). According
to Proposition D each latent flow & (t) in Equation S preserves an underlying equivariance, which facilitates more sample-efficient learning.
Discrete vs. continuous updates. Equation (8) adopts a discrete update formulation based on Lie group actions. However, our framework also admits a continuous-time integral by computing the matrix logarithm of the group action to obtain its infinitesimal generator. Specifically, for each expert, we define Èu (t, z(t)) == log (t, ((t)),, where É. E g is the Lie algebra element corresponding to सt. E G. This enables continuous integration of the latent trajectory via the Lie algebra. Specifically, for the ttth flow, the evolution of 2t)) is governed by the differential equation d di (1) & (1, x()) ((), whose solution over small interval [L2, + Ar) is given by the matrix integral: (t, + At) = exp (I%, (fM++A' (s)) ds) =(tx). While this formulation enables
interpolation and handling of irregular time intervals, computing the integral and exponential map can be computationally expensive. By the first-order approximation of Lie group flows (50), we
can approximate the matrix integral ( + At) R exp
(AV- (1,, ((1)) ((44). With a slight abuse of notation, we then define the group action म사. (t , (t(()) = exp At (x() =((()))
where t, E ((+1)). This leads to the the discrete At update:
E(t: + At) = Σ h(t 1(10), ((t(0)) 1=1
(a(tt),x(t()) E(tz).
(10)
Combing this equation with the Equation we can obtain a high-resolution latent sequence (Ai) (ž(4n), El1n + A/), 244 + 2 Ä((N At), (((x)), which can be treated as features for interpolation and extrapolation, as shown in Equation (5). The bottom right part of Fig. demonstrates the process: the green observations are sparse, but the pink latent vectors are dense.
3.3 Multi-scale Latent MoS for Short- and Long-term Equivariance
The transformation fx (t()), x((+))) defined only within the local interval ((+))))))), may fail to capture long-term equivariant relations. Specifically, there may exist symmetry transformation (g(g) that maps 2(4,) to point outside the current interval to preserve the dynamics, but will be missed by locally defined models. To address this, we propose a simple hierarchical structure. Intuitively, we consider different durations for a constant dynamic function, illustrated in Algorithm
L' > L' if level, j corresponds to a higher and coarser resolution level.

---

Algorithm 1 Multi-level Latent Sequence Generation
Require: Initial full latent state ((6) E R time step Al, number of subintervals at different levels (T((), [ ..... I.(5)).
1: Partition (tu) into [z (tu)
(tu)), where each 2 (to) E m/5
2: for each level 8 = to S do
3: Set I. T,(*)_ Use Equation
compute the subintervals {100, 1111
4: for each subinterval [(() )(++)) do.
5: Generate MoS(s)
( )) and gates h (t(0) z(5) (t(0)). (t
6: end for
7: Use Equation 10 to gain trajectory
(() uu.N (At) (t)) 2 (to At), ..... (l)
8: end for
9: return N0.N At) = concat
(At.)
O:N (At) along the feature dimension.
4 Experiments
We test the following datasets, fully described in Appendix B (1) Complex Nonlinear ODE
Systems. Similar (37. we employ spiral datasets, Glycolytic oscillators (biochemical system), and Lotka-Volterra systems (the interaction between a predator and prey population). (2) Residential
Electricity Consumption. We gather real-world electricity data, publicly available at (31 (3)
Photovoltaic Solar Energy. We introduce a publicly available Photovoltaic (PV) dataset [54] for solar power generations. (4) Power System Event Measurements. This is 10-dimensional dataset (see Appendix [ produced by high-order and time-dependent ODE system (55)57]. (5) Air Quality System. UCI Repository provides measurements of metal oxide chemical sensors in air quality monitoring system 58 (6) Electrocardiogram (ECG) signals. Recorded electrical signals from patient's 's heart in the UCR Time Series Archive [59]. These test systems are representative, encompassing nonlinear and high- digeeniimnsional dynamics, as well as real-world applications in power systems, weather systems, and biomedical domains, of which exhibit disturbances and noise. For these datasets, the interpolation and extrapolation tasks are illustrated in Appendix B To evaluate these systems, we adopt a range of state-of-the-ar methods based on Neural ODES and Transformer, as detailed in Appendix C Model architectures and training time comparisons provided in Appendix [D] and Appendix E.3 respectively. For each method, we run 10 times and report the average errors.
4.1 Effective Symmetry Preservation in the Latent Space 2
First, we validate whether Latent MoS can cor- rectly preserve the symmetry. We test the model in the ODE systems. As shown in the left part of Fig. 2 the true data are governed by linear or nonlinear rotational and slight scaling symmetry. Then, we set the data drop rate to be 90%. We utilize Principal Component Analysis
Ground Truth
Latent ODE
Latent MoS
Spiral
reduce the 15 15-dimensional latent vector (m = 15) in to 2D for visualizations in the middle and right parts of Fig. 2 The result suggests that even with limited input data, latent MoS captures the correct and highly interpretable latent geometry, which can used for the geometry- based analy- ses in dynamical systems. However, Latent ODE will produce unstructured latent trajectories that risk overfitting. The predicted curve visualiza- tions are provided in Appendix F.2
Glycolytic Oscillator
Lotka- Voltema
pea.
poa
pea
Figure 2: Raw data (left) and latent trajectories for
Latent ODE (middle) and Latent MoS (right).
In the Glycolytic system, even when all symmetry experts are included, model achieves a low MSE of 0.0033. Importantly, the gating weights are highly concentrated: the dominant expert, i.e., a second-order transformation (rotation x scaling), receives weight of 0.97, while the next highest (scaling x rotation) receives 0.025, and the remaining experts collectively contribute less
7

---

Table Test Mean Squared Error (MSE) (x 10 for extrapolation tasks under varying drop rates.
Drop
EqSINDy
RNN-A,
ODE-RNN
NCDE
Comtlormer
ODE
MoS
90%
1.79
2.01
1.01
1.16
1.34
1.07
1.21
1.13
1.04
0.62
Spical
60%
0.88
0.96
0.19
0.98
1.31
1.09
1,09
0.89
0.99
0.54
30%
0.45
0.78
0.33
0.95
1.11
1.09
1.10
0.93
0.96
0.33
9098
10:23
13.12
0.52
13:44
12.11
13.57
12.92
2.47
9.88
0.32
Glycolyte
60SS
8.81
13.06
0.20
13.29
11:23
7.54
0.77
0.49
3.88
0.15
30%
8.97
10.98
0.33
13.28
11.10
3.51
0.06
0.16
2.67
0,06
Locka
60%
5.34
11.23
0.75
5.35
12.28
4.25
14.51
5.48
0.18
0.13
30%
3.98
10.75
0.35
5.32
11.10
2.49
14,50
1,80
0.09
0.05
7.88
7.67
8:69
3.25
Load
60%
4.98
4.27
5.75
6.39
15.19
5.49
9.81
3.61
2,42
3058
3.14
3.06
5.10
5.93
11.19
5.14
7.89
6.27
2.96
1.33
90%
20.13
22,98
17.16
10:53
25.33
19.82
24.50
20.44
11.26
8.83
Solar
2.62
30%
16.82
18.37
11:38
7.36
19.84
15.53
15.31
16.08
7.28
3.48
90%
7.72
8.21
8.04
9:43
10.45
8.09
10.05
8.65
8.02
5.32
7.95
9.08
30%
5.89
5.62
7.69
8:41
8.98
6.74
7.22
7.58
6.78
2.53
90%
10:19
9.22
9.38
10:50
11.20
8.44
8.28
10:56
9.55
5.90
Air quality
60%
10:07
8.71
8.91
8.86
10.59
7.11
10.38
12:21
7.89
4.10
9.13
8.98
7.95
9.05
9:44
90%
5.98
6.92
3.48
4.33
5,33
3.49
4.08
3.91
3.04
1.08
ECG
60%
5.13
5.12
2,02
3.23
4.59
2.13
2.11
4.10
1.03
0.91
1.84
than 0.005. This concentration indicates that the model identifies and relics on the correct symmetry transformation.
4.2 Overall Evaluations on Diverse Datasets with Different Data Drop Rates
We evaluate the model performance under varying drop rates: 90%, 60%, and 30%. The complete results are shown in Fig. 3 (see also Table i3 Appendix E.T for interpolation tasks and Table for extrapolation tasks. In general, Contiformer, Latent ODE, and the proposed Latent MoS all achieve strong performance across both continuous ODE-based systems and real-world datasets with high disturbances. This attributed to their shared ability to reconstruct latent trajectories during decoding, effectively capturing the underlying system dynamics. Among them, Latent MoS leverages geometric priors to structure the latent trajectorics (see Fig. - For instance, while Contiformer and Latent ODE struggle with high-frequency Glycolytic oscillations, Latent MoS successfully encodes the data into a latent space where linear Lie group action-based dynamics can be more easily estimated. Across all datasets, Latent MoS achieves relative improvements ranging from 15% to 97%. contrast, Informer and Autoformer only perform well under low drop rates, as they rely on pre-interpolation and lack the capacity to model the underlying continuous dynamies directly.
EqSINDY
RNN-AI
NCDE
Latent ODE
EqGP
ODE-RNN
Contiformer
Latent MoS
Spina
Giycolyrie
Lodka
Load
0.1
MSE (logssale)
0.001
60%
900%
60%
90%
60%
90%
60%
90%
Soles
Power event
Air quility
ECG
0.1
0.001
30%
60%
90%
30%
60%
90%
30%
60%
90%
30%
60%
90%
Drop rate
Drop rate
Drop rate
Drop rate
Figure 3: Test MSE in log- scalec for interpolation tasks.
Table
Appendix
E.1 shows values.
4.3
Sensitivity Analysis on Noise, Gating Mechanisms, and Latent Dimensionality
We use the Glycolytic system as an example for sensitivity analysis under a 90% data drop rate. Similar trends observed across other systems. Fig. 4 shows the interpolation test set MSE under

---

various conditions. First, we introduce Gaussian noise with increasing standard deviations. The MSE for both Latent ODE and Latent MoS rises gradually, reflecting the impact of noise. This demonstrates that they are robust to noise perturbations and capable of approximating the underlying average trend. Second, we vary the latent dimensionality m. Latent MoS exhibits a sharp drop MSE when m > This suggests that higher provides sufficient capacity for approximating the nonlinear Lie group actions described in Equation (2). while still preserving linear operations within Z. In contrast, Latent ODE lacks structured inductive biases in its latent space, and its performance remains relatively unaffected by increased dimensionality. Third, we examine the effect the gating mechanism by allowing the top Ko K gates with the highest scores to open and vary Ku- The error remains when Ko E [1, 4] because Glycolytic system contains rotational and scaling symmetries. However, when Ko becomes large, Latent MoS tends to overfit. Empirically, we find that setting Kg 4 yields strong performance, though further improvements can be achieved through hyperparameter tuning.
0.010
0.15
0.10
0.008
MSE
0.10
Latent MoS
0.05
Latent ODE
0.006
0.00
0.05
0.15
0.2
20
Std of noise
Latent dimension
Number of open gutes
Figure 4: Results sensitivity analysis for Glycolytic systems under 90% data drops.
Scalability in high-dimensional systems. Consistent with empirical findings in Latent ODEs [8] we observe that Latent MoS achieves strong performance on the 10- (O-dimensional power event dataset by setting m = 30. We hypothesize that this efficiency stems from underlying physical correlations.
4.4 Ablation Study
Using the same setting as in Section we conduct an ablation study. As shown in Fig. the presence of sparse gates crucial for selecting relevant symmetries in our MoS framework. Furthermore, in the Glycolytic system, removing scaling or rotational symmetries sig- nificantly increases the MSF. Interestingly, performance improves when unnecessary symmetries, such as transla- tional or certain second-order components, are excluded. When the translational expert is removed (which is un- necessary for Glycolytic dynamics), the MSE decreases to 0.001 and the gating weight for rotation scaling increases to 0.99.
w/o reconder-order w/o scaling w/o translation w/o rotation w/o gates w/o sparsity gates Latent MoS
0.000
0.005
0.010
0.015
MSE
Figure 5: Results in ablation studies.
This demonstrates that while the gating works well, a simplified expert set can enhance efficiency and robustness. To achieve the best performance, we can perform a validation- procedure to progressively remove unnecessary experts and compare the resulting MSE. Importantly, the gating mechanism remains valuable in this process: it helps identify the dominant expert(s) within candidate subset, effectively narrowing down the search space.
4.5 Dynamic Learning for Control
We introduce an optimal frequency control problem in power systems. The task is stabilize the generator's 's frequency. Past methods [60] assume an idealistic second-order ODE model, i.e., swing equations, to model frequency dynamics. However, real-world dynamics are higher-order with unknown physical parameters. Hence, we utilize latent MoS to learn it. In general, we compare model-free proximal policy optimization (PPO) 611, model-based policy optimization (MBPO) with a Gaussian process (MBPO-GP) (62]. MBPO with Latent MoS (MBPO-MoS), and MBPO with the swing equation (MBPO-SE). By the end time, our MBPO-MoS stabilizes near zero with a deviation -0.024, the lowest among all methods. In contrast, PPO remains highly unstable ( (00.928) MBPO-GP yields 100, and MBPO-SE yields -0.115. The general cost (integral the Euclidean norm of frequency deviations) in the control process is 0.081 (MBPO-MoS), 0.163 (MBPO-SE 0.257 (MBPO-GP), and 0.51 (PPO). Our learned dynamics Latent MoS can much better predict the future frequeney, hence largely improving control performance.

---

Related Work
Dynamic Modeling Against Data Scarcity, Existing approaches to dynamic modeling under data scarcity fall into four categories. The first focuses interpolation: model-based methods (e.g.. multidimensional interpolation [63] physical estimations (64)) assume explicit system behavior,
while optimization- based techniques (e.g., compressed sensing [8) matrix completion [15] rely on
low-rank or sparsity assumptions to ensure tractability. Data-driven models from signal processing and machine learning [65 [2] can adapt to complex patterns but often neglect domain-speciffe structures. The second category incorporates physical knowledge through governing equations 12 666, enabling convergence even from sparse data. The third uses Bayesian inference inject priors and quantify uncertainty [33155, though performance depends heavily on prior quality. The fourth employs neural ODE solvers [16], including Latent ODEs [18] and Neural CDBs (17), which model latent catinuous-time flows decodable to system states but require high-resolution data, as ODE integral errors accumulate (67].
Symmetry-bas -based Learning. Enforcing symmetry priors in equivariant neural networks improves generalization and sample efficiency for applications like image processing (27-30 and RL [3133]
However, they typically demand prior knowledge to handcraft symmetry transformations. To discover unknown symmetry knowledge, pioneer work involves a restrictive search space [68]70] 42 [8], e.g.,
a discrete symmetry group [3 Recent work extends this study to generate and approximate different In particular, the framework based on Generative Adversarial is highly generalizable to finding different symmetries in the latent space, grounded on the theory of Lie algebra. Nevertheless, GAN known to have training instability [71] and their methods may deliver fallacious, trivial, or unclear solutions 41
kinds of symmetries 40
41
37]
Networks (GANs) 44
37
Learning. Existing approaches that incorporate symmetry breaking in machine learning can be categorized into three main groups (1) Learnable or non-stationary weights relax strict equivariance by allowing model parameters to adapt locally [6). (2) Soft regularization encourages but does not strictly enforce equivariance, typically through auxiliary loss terms [43). (3) mennetry-breaking sets provide constructive strategy by introducing minimal auxiliary inputs to selectively break output symmetry (72). Our model falls into the first category, employing locally adaptive, learnable weights capture approximate symmetries.
Symmetry-breatt
Conclusion, Limitation, and Future Work
We present Latent MoS, sample-efficient framework for learning dynamical systems that explicitly preserves equivariance through structured latent transformations. Motivated by the observation that real-world systems often exhibit symmetry breaking. where strict group invariances hold only approximately or locally, Latent MoS captures these effects through mixtures of symmetry groups and adaptable latent flows. Grounded in Lie group theory, our approach provides physically interpretable and effective means of modeling system dynamics under symmetry constraints. Although the set of predefined candidate transformations does not cover the full space of possible Lie group representations, empirical results demonstrate consistent improvements across wide range of physical systems. The modular design of Latent MoS further enables natural extensions to incorporate additional or more complex symmetry transformations.
Latent MoS is primarily suited for systems that exhibit least local symmetries. While systems entirely lacking symmetry, such as highly chaotic or purely stochastic processes (e.g.. unconstrained turbulence), may fall outside its intended scope, such cases are rare in practice. Most real-world phys- ical, biological, and engineered systems exhibit local or approximate symmetries due to conservation laws, structural regularities, or recurring patterns.
In future work, we plan to extend Latent MoS in two directions. First, we will develop automated mechanisms to refine and constrain the candidate set of symmetries, using validation- based selection or search strategies, so that the model can achieve the best performance even when the true symmetries uncertain. Second, we aim to explore physically meaningful symmetry structures for broader geometric analyses in dynamical systems, including stability, safety, and contraction properties.
10

---

References
[1] Haoran Li and Yang Weng. Physical equation discovery using chysics-consistent neural network (penn) under incomplete observability. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 925-933, 2021.
[2] Haoran Li, Yang Weng, and Hanghang Tong. Console: Convex neural symbolic learning. Advances in Neural Information Processing Systems, 35:5766-5774, 2022.
[3] Anusha Nagabandi, Gregory Kahn, Ronald S. Fearing, and Sergey Levine. Neural network dynamics for model-based deep reinforcement learning with model-free fine-tuning, In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 7559-7566, 2018.
[4] Felix Berkenkamp, Matteo Turchetta, Angela Schoellig, and Andreas Krause. Safe model-based reinforcement learning with stability guarantees. Advances in neural information processing systems, 30, 2017.
[5] Wenqi Cui and Baosen Zhang. Lyapunov-regullrizzed reinforcement learning for power system transient stability. IEEE Control Systems Letters, 6:974-979, 2021.
[6] Jian Li, Ming Zhao, and Kai Zhang. Low-quality and missing data handling in power grids using neural networks. IEEE Transactions Smart Grid, 2024. to appear.
[7] Chenhan Xiao, Yizheng Liao, and Yang Weng. Distribution grid line outage identification with unknown pattern and performance guarantee. IEEE Transactions on Power Systems, 2023.
[8] David L. Donoho. Compressed sensing. IEEE Transactions on information theory, 52(4):1289 1306, 2006.
[9] Yuejie Chi, Yue M Lu, and Yuxin Chen. Nonconvex optimization meets low-rank matrix factorization: An overview. IEEE Transactions on Signal Processing, 67(20):523--526), 2019.
[10] Mark A Iwen, Felix Krahmer, Sara Krause-Solberg, and Johannes Maly. On recovery guarantees for one-bit compressed sensing on manifolds. Discrete & computational geomelry, 65:953-998, 2021.
[11] Zhao Chen, Yang Liu, and Hao Sun. Physics-informed learning of governing equations from scarce data. Nature communications, 12(1):6136, 2021.
[12] Haoran Li, Zhihao Ma, Yang Weng, Haiwang Zhong, and Xiaodong Zheng. Low-dimensional ode embedding to convert low-resolution meters into "virtual" pmus. IEEE Transactions on Power Systems, pages 1-13, 2024.
[13] Jesper Kjær Nielsen, Mads Græsbell Christensen, Taylan Cemgil, Simon Godsill, and Søren Holdt Jensen. Bayesian interpolation and parameter estimation in a dynamic sinusoidal model. IEEE transactions on audio, speech, and language processing. (9(())1986-11)) 2011.
[14] Nathaniel J Linden, Boris Kramer, and Padmini Rangamani. Bayesian parameter estimation for dynamical models in systems biology. PLoS computational biology, ((110)e0010651) 2022.
[15] Ming Yi, Meng Wang, Tianqi Hong, and Dongbo Zhao. Bayesian high-rank hankel matrix completion for nonlinear synchrophasor data recovery. IEEE Transactions on Power Systems, 2023.
[16] Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. In Advances in Neural Information Processing Systems, pages 6571-6583, 2018.
[17) Patrick Kidger, James Morrill, James Foster, and Terry Lyons. Neural controlled differential equations for irregular time series. Advances in Neural Information Processing Systems, 33:6696-6700 2020.
[18] Yulia Rubanova, Ricky TQ Chen, and David K Duvenaud, Latent ordinary differential equations for irreuularlyssmppled time series. Advances in neural information processing systems, 32, 2019.
11

---

[19] Haoran Li, Muhao Guo, Yang Weng, and Hanghang Tong. Neural predictive control to coordinate discrete-and continuous- time models for time-series analysis with chtrool-heoorttccl improvements. arXiv preprint arXiv:2508.01 00025000000133,. 2025.
[20] Emilien Dupont, Arnaud Doucet, and Yee Whye Teh. Augmented neural odes. Advances in neural information processing systems, 32, 2019.
[21] Giovanni Russo and Jean-Jacques E Slotine. Symmetries, stability, and control in nonlinear systems and networks. Physical Review E-Statisticall, Nonlinear, and Soft Matter Physics, 84(4):041929, 2011.
[22] Shengchao Yan, Baohe Zhang, Yuan Zhang, Joschka Boedecker, and Wolfram Burgard. Learn- ing continuous control with geometric regularity from robot intrinsic symmetry. In 2024 IEEE International Conference Robotics and Automation (ICRA), pages 49-55, 2024.
[23] Congyue Deng, Or Litany, Yueqi Duan, Adrien Poulenard, Andrea Tagliasacchi, and Leonidas J. Guibas. Vector neurons: A general framework for -euui-cquva networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 12200-12209, October 2021.
[24] J Lewis Blackburn. Symmetrical components for power systems engineering. CRC press, 2017.
[25] John David Crawford and Edgar Knobloch. Symmetry and symmetry- -rreektg bifurcations in fluid dynamics. Annual Review of Fluid Mechanics, 23(1):341-387, 1991.
[26] Cristian- Vasile, Mac Schwager, and Calin Belta. Translational and rotational invariance in networked dynamical systems. IEEE Transactions on Control of Network Systems, 5(3):822- 832, 2018.
[27] Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling. E (n) equivariant graph neural networks. In International conference machine learning, pages 9323 9332. PMLR, 2021.
[28] Lingshen He, Yuxuan Chen, Yiming Dong, Yisen Wang, Zhouchen Lin, et al. Efficient equivariant network. Advances Neural Information Processing Systems, 34:5290-5302 2021.
[29] David Ruhe, Johannes Brandstetter, and Patrick Forré Clifford group equivariant neural networks. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, volume 36, pages 62922-62990. Curran Associates, Inc., 2023.
[30] Simon Batzner, Albert Musaelian, Lixin Sun, Mario Geiger, Jonathan P Mailoa, Mordechai Kornbluth, Nicola Molinari, Tess E Smidt, and Boris Kozinsky. E (3)-equivariant graph neural networks for data-efficien and accurate interatomic potentials. Nature communications, 13(1):2453, 2022.
[31] Dian Wang, Robin Walters, and Robert Platt. equivaraaa reinforcement learning. In International Conference on Learning Representations, 2022.
[32] Amab Kumar Mondal, Vineet Jain, Kaleem Siddiqi, and Siamak Ravanbakhsh. Eqr: Equivariant representations for data-efficient reinforcement learning. In International Conference on Machine Learning, pages
PMLR, 2022.
[33] Joshua McClellan, Naveed Haghani. John Winder, Furong Huang, and Pratap Tokekar, Boosting sample efficiency and generalization in multi-agent reinforcement learning via equivariance. arXiv preprint urXiv:2410.0255: 2024.
[34] Johannes Brandstetter, Max Welling, and Daniel E Worrall. Lie point symmetry data aug- mentation for neural pde solvers. In International Conference on Machine Learning, pages 2241-2250 PMLR, 2022.
[35] In Huh, Eunho Yang. Sung Ju Hwang, and Jinwoo Shin. Time-reversal symmetric ode network. Advances in Neural Information Processing Systems, 191190016-9027,2000
12

---

[36] Tara Akhound- Akhound-Sadegh, Laurence Perreault-Levasseur, Johannes Brandstetter, Max Welling, and Siamak Ravanbakhsh. Lie point symmetry and physics-informed networks, Advances in Neural Information Processing Systems, 36, 2024.
[37] Jianke Yang, Wang Rao, Nima Dehmamy, Robin Walters, and Rose Yu. Symmetry-informed governing equation discovery. arXiv preprint arXiv:2405. 16756, 2024.
[38] Miguel Abreu, Luis Paulo Reis, and Nuno Lau. Addressing imperfect symmetry: novel symmetry-leariigg actor-critic extension. Neurocomputingg, 614:128771, 2025.
[39] Pablo Calvo-Barlés Sergio G Rodrigo, Eduardo Sánchez-Burilly and Luis Finding discrete symmetry groups via machine learning. Physical Review E, 145(44) 2024.
[40] Jianke Yang, Nima Dehmamy, Robin Walters, and Rose Yu. Latent space symmetry discovery. arXiv preprint arXiv:2310.00105, 2023.
[41] Jianke Yang. Robin Walters, Nima Dehmamy, and Rose Yu. Generative adversarial symmetry discovery. In International Conference on Machine Learning, pages 39488-39508. PMLR, 2023.
[42] Samuel Otto, Nicholas Zolman, J Nathan Kutz, and Steven L Brunton. A unified framework to enforce, discover, and promote symmetry in machine learning, arXiv preprint ar0022231..0021,, 2023.
[43] Hyunsu Kim, Hyungi Lee, Hongseok Yang, and Juho Lee. Regularizing towards soft equiv- ariance under mixed symmetries. In International Conference on Machine Learning, pages 16712-16727. PMLR, 2023.
[44] Rui Wang, Robin Walters, and Rose Yu. Approximately equivariant networks for imperfectly symmetric dynamics. In International Conference on Machine Learning, pages 23099.23091. PMLR, 2022.
[45] Rui Wang, Elyssa Hofgard, Han Gao, Robin Walters, and Tess Smidt. Discovering symmetry breaking in physical systems with relaxed group convolution. arXiv preprint arXiv:2310.02299, 2023.
[46] Tycho van der Ouderaa, David W Romero, and Mark van der Wilk. Relaxing equivariance constraints with non-stationary continuous filters. Advances in Neural Information Processing Systems, 35:33818-233 18-33830, 2022.
[47] Alexander A Kirillov. An introduction to Lie groups and Lie algebras, volume 113. Cambridge University Press, 2008
[48] Josip Česié, Ivan Marković, and Ivan Petrović. Mixture reduction on matrix lie groups leee signal processing letters, 24(11): 2017.
[49] Saeed Masoudnia and Reza Ebrahimpour. Mixture of experts: literature survey. Artificial Intelligence Review, 12:275-293, 2014.
[50] Ernst Hairer, Marlis Hochbruck, Arieh Iserles, and Christian Lubich. Geometric numerical integration. Oberwolfach Reports, 3(1):05--88) 2006.
[51] Marija Ilie, Le Xie. and Qixing Liu. Engineering IT-enabled sustainable electricity services: The tale of two low-cost green azores islands, volume 30. Springer, 2013.
[52] Haoran Li, Muhao Guo, Marija Ilie, Yang Weng, and Guangchun Ruan. External data- enhanced rectreepressntttioon for adaptive probabilistic load forecasting. arXiv preprint arXiv:2506.23200 2025.
[53] Haoran Li, Muhao Guo, Yang Weng, Marija Ilie, and Guangchun Ruan. Exarnn: An environment-drivee adaptive rnn for learning non-stationary power dynamics. arXiv preprint arXiv:2505.17488, 2025.
13

---

[54] Matthew Boyd. Nist weather station for photovoltaic and building system research. National Institute of Standards and Technology, Gaithersburg, MD. Technical Note, (1913). 2016.
[55] Haoran Li, Yang Weng, Evangelos Farantatos, and Mahendra Patel. An unsupervised learning framework for event detection, type identification and localization using pmus without any historical labels. In 2019 IEEF Power & Energy Society General Meeting (PESGM), pages 1-5, 2019.
[56] Haoran Li, Yang Weng, Evangelos Farantatos, and Mahendra Patel. A hybrid machine learning framework for enhancing pmu-based event identification with limited labels. In 2019 Interna- tional Conference on Smart Grid Synchronized Measurements and Analytics (SGSMA), pages 1-8, 2019.
[57] Haoran Li, Zhihao Ma, Yang Weng, Erik Blasch, and Surya Santoso. Structural tensor learning for event identification with limited labels. IEEE Transactions on Power Systems, 38(6):5314 14- 5328, 2023.
[58] Saverio Vito. Air Quality. http:///....rroo..24///////////
UCI Machine
Learning Repository, 2008.
DOL:
[59] Yanping Chen, Eamonn Kcogh, Bing Hu, Nurjahan Begum, Anthony Bagnall, Abdullah Mueen, and Gustavo Batista. The uer time series classification archive, July 2015. www.cs. uur edu/ "eammonnttiii time series _dtaa/
[60] Hamad Alduaij, Yang Weng, and Haoran Li. Beyond monotonic droop: Expanding feasible control regions for optimal frequency regulation. IEEE Transactions on Power Systems, pages 1-14, 2025.
[61] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint vrXiv:1707.06477 2017.
[62] Marc Peter Deisenroth, Dieter Fox, and Carl Edward Rasmussen. Gaussian processes for data-efficient learning in robotics and control. IEEE transactions on pattern analysis and machine intelligence, 37(2):008-423, 2013.
[63] Christian Habermann and Fabian Kindermann. Multidimensional spline interpolation: Theory and applications. Computational Economics, 30: 153-199, 2007.
[64] Mauricio D Sacchi, Tadeusz J Ulrych, and Colin J Walker. Interpolation and extrapolation using a high-resolution discrete fourier transform. IEEE Transactions on Signal Processing, 46(1):31-38, 1998.
[65] Kai Fukami, Koji Fukagata, and Kunihiko Taira. Machine-learring -ased spatio-temporal super resolution reconstruction of turbulent flows. Journal of Fluid Mechanics, 909:A9, 2021.
[66] Yeonjong Shin, Jerome Darbon, and George Em Karniadakis. On the convergence of physics informed neural networks for linear second-ordes elliptic and parabolic type pdes. arXiv preprint UrXiv:2004.0800 2020.
[67] Birgit Hillebrech and Benjamin Unger. Certified machine learning: A posteriori error estimation for physies-informed neural networks. In 2022 International Joint Conference on Neural Networks (ICCN), pages 1-8. IEFF, 2022.
[68] Gregory Benton, Mare Finzi, Pavel Izmailov, and Andrew G Wilson. Learning invariances in neural networks from training data, Advances neural information processing systems, 33:17605-17616 2020.
[69] Allan Zhou, Tom Knowles, and Chelsea Finn. Meta-learning symmetries by reparameterization. arXiv preprint arXiv:2007.00233, 2020.
[70] David W Romero and Suhas Lohit. Learning partial equivariances from data. Advances in Neural Information Processing Systems, 35:36466-36678, 2022.
14

---

[71] Lars Mescheder, Andreas Geiger, and Sebastian Nowozin. Which training methods for gans do actually converge? International conference on machine learning. pages 3481-3490. PMLR, 2018.
[72] YuQing Xie. Equivariant symmetry breaking sets. Massachusetts Institute of Technology, 2024.
[73] Evgeni E SEL'KOV. Self-oscillations in glycolysis 1. simple kinetic model. European Journal of Biochemistry, 4(1):79-86) 1968.
[74] Haoran Li, Yang Weng, Evangelos Farantatos, and Mahendra Patel. An unsupervised learning framework for event detection, type identification and localization using pmus without any historical labels. In 2019 IEEE Power & Energy Society General Meeting (PESGM), pages 1-5, IEEE, 2019.
[75] Haoran Li, Zhihao Ma, and Yang Weng. transfer learning framework for power system event identification. IEEE Transactions on Power Systems, 37(6):4424 4435, 2022.
[76] General Electric Energy Consulting. General Electric Concorda PSLF. 2018.
[77] Engineering Texas A&M University. Illinois 200-Bus System: ACTIVSg200. 2016.
[78] Dheeru Dua, Casey Graff, et al. Uci machine learning repository. 2017.
[79] Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David Sontag, and Yan Liu. Recurrent neural networks for multivariate time series with missing values. Scientific reports, 8(1):1-12, 2018.
[80] Yuqi Chen, Kan Ren, Yansen Wang, Yuchen Fang, Weiwei Sun, and Dongsheng Li. Contiformer: Continuous-time transformer for irregular time series modeling. Advances in Neural Information Processing Systems, 36, 2024.
[81] Haoyi Zhou, Shanghang Zhang. Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 11 11106-11115, 2021.
[82] Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long, Autoformer: Decomposition trans- formers with auto-correlation for long-term series forecasting. Advances in neural information processing systems, 34:22419 224330, 2021.
15

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No]. or [NA].
[NA] means either that the question Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will published with the paper.
The reviewers of your paper will be asked use the checklist as one of the factors their evaluation. While "[Yes] is generally preferable to "[No]" ", it is perfectly acceptable to answer "[No] providedd proper justification is given (e.g.. "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering binary way, we
"[No] or "[NA]" is not grounds for rejection. While the questions are phrased acknowledge that the true answer is often more nuanced, please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The material can be found in Abstract and Introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The discussion of limitations of the work can be found in Conclusion section.
16

---

Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification:
The assumption and proofs can be found in Appendix A.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: The dataset detaits can be found in Appendix B. The baseline details can be found in Appendix C. The details of experiment setting be found in Appendix D.
Guidelines:
The answer NA means that the paper does not include experiments.
17

---

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g.. with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome describe the particular way they provide for reproducibilityy In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: we will submit code and data as supplementary materials. After the review process, we will upload the code to Github.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
//nips.cc/public/eee. www.eeddeee.oeuuummmcecomd ssi ssionyolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
18

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification:
The details of experiment setting can be found in Appendix D.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to a level detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: The statistical errors are reported in Experiments, showing significant improve- ments in our methods.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split. initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (c.g. negative еттог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer:
[Yes]
Justification: The resource is described in Appendix D, and the run time is described in Appendix E.3.
Guidelines:
The answer NA means that the paper does not include experiments.
19

---

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
We have carefully read the codes and make sure it's well conducted.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: [TODO]
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: [TODO]
20

---

Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [NA]
Justification: [TODO]
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of in the package should be provided. For popular datasets, paperssithcode com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re- re-packaged, both the original license and the license of the derived asset (if has changed) should provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: [TODO]
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
21

---

Answer: [NA]
Justification: [TODO]
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: [TODO]
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: [TODO]
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non non-standard components.
Please refer to LLM policy fattps://neuripe: cc/ /ttp:://s//..ooooeeeeee///////// for what should or should not be described.
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. There many potential societal consequences of our work, none which we feel must be specifically highlighted here.
22

---

A Detailed Proofs
A.1 Proof of Lemmal
Lemma 1. Assume that G and H C G are Lie groups whose elements are defined in Equation (6) and
Equation (3), respectively. Define the centralizer of H in G as: Cc(h) == {g E (Ta(h) (a(T) (g)}. If CC(h) is nontrivial (i.e., it contains elements other than the identity), then Vgg E Ce(h), the relation for (g(g) Equation 3 preserved.
Proof. For Vy E (C), we have
(=(0)((t(+1) = ==(0)Ta(()=((t)
(11)
==(())=(())((4))
(12)
==((T(()=((L)),
(13)
where the first equality follows from Equation (6). the second by the definition of the centralizer, and the third again by Equation (6).
L
A.2 Proof of Proposition
Proposition (Sufficient Conditions for Nontrivial Centralizer), Let C. Af(mm be Lie subgroup whose elements are represented in homogeneous coordinates as affine transformations:
Ta (a(h) = na(h)) z(t)
A 0
by (t)) 1 1
(14)
1
where Ad GL(m), bn E RTm and latent vector in homogeneous coordinates to support affine transformations. Suppose that for dg h E H, it holds that Ag Ar, An Ag and Agb, = bb- Then have H C C.(h). In particular, Ca(h) is nontrivial and contains at least all of elements in H.
z(t) (z(t), 1]T R to denote the augmented
Proof. We notice that
[,,
by
A
bn
Ag
Aabs + bo
Ne(o) Ta(h)=
(15)
0
1
0
and
[A
bi
Ag
bg
An
Anby + bb
na(h) Fa(g))
(16)
0
0
1
0
1
Since for Vg. Ag A, and A, = bR. then have (g(g) Ta(h) Ta(h) Tg(g).
shows that every pair of elements h E H commute under Td. Thus, E Ca(h), Vig E H. Thus. the centralizer nontrivial and contains at least H.
23

---

A.3 Proof of Corollary 1
Corollary 1 (Equivariance of Planar Rotation Transformation). Let u1 and u2 be orthonormal
vectors and P == [u1 U2] E R=xx Consider the planar rotation transformation: wrot
I + P(Ro- P2 0
cos -inn
where Ro = sin cos E SO(2), Im is the m x m identify
matrix. Then, no and its Lie group c S0(m) satisfies the conditions in Proposition
Proof. For the rotation matrix Qo = 1m + P(Rg - 12) we have
QaQ, = (Lm + P(Rg 12) (1m + P(R, - )2)
Lm P(Rg 12)P" P(R, - 12) P(Rg 12)PPT + P(R, 12)
mm P(Ro (2) 12)P P(R 12)PT +P(R, 12)P
and
Q,Q0 = (1m + P(R, - 12) m + P(Rs - )
mm P(R, 12)P P(Rs 12)PT P(R, 12)PPT + P(Rs 12)PT
Im P(Ry (R) 12) P(R, 12)PTT T + P(Ro 12)PT.
where the equality holds due to PT P Iz. Since we have
coss 0
sin cos 0
cos
- sin 7
cos cos y - sin 8sin - cos sin 7 - sin cos sin cos cos 0 sin sin 0 sin y + cos cos
RaR,
sin 0
siny
cos 7
=
coos(e + 7
sin(f + +7)
Roty Rs
s0 7)
cos(0 + 7)
we also have QoQ, = Q-Q0-
A.4 Proof of Corollary
Corollary 2 (Equivariance of Translation and Scaling Transformation). Denote the translation and its Lie group H C E(m), where E(m) is an Euclidean group.
dynamics: tra
Im
v
0
1
They satisfy the conditions Proposition Denote the scaling dynamics: a (diag(y) 1
and its Lie group H Sim(mn) where mi is similarity group. They satisfy the conditions in Proposition 7
Proof. In the case of translation transformation atra for identity matrices, we have Im4m Im me- Also, we have I, x v for Yo IR. Therefore translation transformation a satisfies the condition in Proposition [ In the case of scaling transformation a for diagonal matrices, we have diag(r, )diag(?2) = ddia(?)))dia(())) ). Also, we have diag(Yr) x 0 = 0. Therefore scaling
transformation sca satisfies the condition Proposition
24

---

A.5 Proof of Corollary 3
Corollary 4 (Equivariance of Second-Order Composed Transformations). The second composi- tion of the planar rotation, translation, and scaling transformations defined in Corollaries
7 satisfies
the commutativity conditions Proposition | Specifically, for any #0), (2)) E { sea
we have that (1))g ((2) and Lie group C (m) satisfy the conditions in Proposition
Proof. To show that a (2) satisfies the commutativity conditions in Proposition show that
need to
A ( (g1) (92) (1) (h1) (2))) ( (h1) (h2)+ (91) (2)))(1)(00))))))))))))) (g)). Vg1 92. h1, E H.
Since (11, .(2) (#rot, Atre # which already satisfy the commutativity conditions shown in Corollaries
2 we have
* ) (h2) = (bx) (g1). Vgi, H
( (12) ( (2) (h2) = # (12) (g2), Yga, h2 E H
can be commutative, i.c., Thus, the initial condition is reduced to showing that (1) and ((2)
* (g) (h) (2)) (h)+(1 4g. fg H
Consider two transformations =(1, (()) E rort gero ) ) their general affine transformation form is:
(A)) ((K) 1 for k 1,2,
where ( E Rmxm and b
(8)
R We have:
A
A(2) B (2)
+ (1)
and #1))(1) (4(2)0)
A + (2))
# (2)
0
0
According to Proposition I commutativity holds if:
( ( (2) and ( +b (1) 2
We now verify these two conditions for all combinations:
Linear Part: A -
If both are diagonal scaling matrices, this holds since diagonal matrices commute.
If both are planar rotations in the same subspace, they commute by the abelian property of SO()).
either matrix is Im (as in translation), the product trivially commutes. one is scaling (diagonal) and the other is a planar rotation, they commute within the rotation subspace or when scaling is isotropic.
Translation Part: A (2) (1) ( (1) (2)
If both transformations translations, A A Im and the equality holds trivially.
one is translation and the other is rotation or scaling, we have:
g ((2) ( = (())
since translation has (A) Im. satisfying the equality.
If both non-translation (rotation scaling), (1) ( 0. and the equality
holds.
In all cases, the matrix product and translation terms satisfy the required commutation conditions. Therefore,
# (1) (2) (1)
and the composed transformations satisfy the conditions of Proposition
25

---

B Tasks, Data Generation, and Preprocessing
B.1 Interpolation and Extrapolation Tasks
We adopt a standard formulation consistent with prior work on continuous time and irregularly sampled time series, such Latent ODE, Contiformer, RNN-A,, etc. The setup as follows:
(1) Interpolation. Given a time series with time points (to- ... fx). we condition on the subset of points from (to: (....... -vv) with data drop rate (30%, 60%, 90% in Experiments) and reconstruct the full set of points in the same time interval.
(2) Extrapolation. We split the time series into two parts (lo.... .(222) and (1 Lx). We input the first half of the time series and predict the second half. We apply the same random drop procedure only the input half, and the model is tasked with predicting the entire future segment beyond the observed range.
B.2 Complex ODE systems
Spiral dataset. We generate 80 trajectories with 60 timesteps. The spiral system is a two-dimensional system with rotation and scaling symmetries, characterized by the infinitesimal generator
2201 O2 (i.e.,
in matrix terms). It is governed by:
-1 0
Φ 00.121 22,
(17)
22 = 0.122,
We also create more complex ODE systems with nonlinear symmetries.
Glycolytic Oscillator. We generate 100 trajectories, each with 200 timesteps. The two-dimensional Glycolytic Oscillator system [73] models a biochemical process governed by pair of coupled ODEs with complex cubic nonlinear interactions. We adopt the same parameter settings as used in 137. The governing equations are given by:
ij = 0.75 - 0.1æ - x1
(18)
i2 = 00..11 x2 + 21
Lotka-Volterra System. We generate 100 trajectories, each with 200 timesteps. The two-dimensional Lotka- Volterra System is classical model in population dynamics that describes the nonlinear interactions between predator and prey species. It is governed by the following coupled ODEs:
i = 0.1x - 0.02x1
(19)
(d22 0.01.2 0.3x2
B.3 Power system datasets
Electricity consumption. We gather load consumption of Flores, Azores Islands in the year of 2008 [5]] The profile contains 366 days' load data with sampling interval of 10min. We treat each day's data (144 points) as a trajectory and gather 100 trajectories.
Solar energy generation. We use publicly available photovoltaic (PV) dataset (54 that records sequential solar power generation measurements. The data was collected in 2017 from a weather station located on the Gaithersburg, Maryland campus of the National Institute of Standards and Technology (NIST), with a sampling interval of 1 minute. We select data from four different locations and treat each day at each location as individual trajectory, resulting in total of 124 trajectories for a one-ma me-montth period. To focus on periods with active solar generation, we remove nighttime measurements with zero output and extract 100 valid time steps per trajectory.
Event measurements. Following 74 75 we simulate power system events using the commercial- grade Positive Sequence Load Flow (PSLF) software (76) developed by General Electric (GE). The simulations are based on the publicly available Illinois 200-node system (77). Each simulation spans 4 seconds with sampling interval of 33.33 milliscconds, capturing high-resolution system dynamics.
26

---

We extract measurements from 100 selected nodes, treating each node's time series as a separate trajectory. This results 100 trajectories, each containing 100 time steps. Each trajectory contains 10-dimensional data: voltage magnitude, voltage angle, current magnitude, current angle, frequency, rate of change of frequency, rotor speed, rotor angle, active power, and reactive power.
B.4 Weather system dataset
Air quality. We consider air quality data from the UCI repository 78 which contains the hourly responses of gas multisensor device deployed on the field in an Italian city. Specifically, we use the C6H6 (GT) feature, which records the hourly averaged benzene concentration (in /g//²)) We select the period from October 1, 2004 to December 13, 2004, containing 73 days of measurements without outliers. We treat each day's data (24 points) as a trajectory and gather 92 trajectories.
B.5 Biomedical system dataset
The signals correspond to electrocardiogram (ECG) shapes of heartbeats for the normal case and the cases affected by different arrhythmias and myocardial infarctions. We employ the ECG200 in [59]. The training data contains 100 trajectories, each with 97 timesteps.
C Introduction of Baseline Methods in Experiments
The following baseline methods are used. For interpolation, we have: (1) EqSINDY (37] EqSINDy adds equivariance regularization to SINDy method to discover the symbolic equations for the ODE data. The equivariance relation is discovered through GAN-based framework [40]. (2) EqGP
37 Similar to EqSINDy, the equivariance relation is added to Generic programming discover (3) RNN-A, [79]. The time difference between every two observations, i.e., A, is introduced a classic RNN model. (4) ODE-RNN [18] ODE-RNN can be directly utilized for dynamic learning in encoder-only framework. (5) Neural CDE (NCDE) 117. Neural CDE creates a continuous data path to control the evolution of the state's ODE flow. (6) ContiFormer 80)
symbolic ODE equations.
ContiFormer is Transformer-based with continuous attention mechanism to extract feature flows at arbitrary times. (7) Latent ODE [8]. The model is illustrated in Section 2.3 For extrapolation, in addition the above methods, introduce the state-ofllue the-art Transformer-bes based methods for time-series forecasting: (8) Informer [8I) Informer is a transformer variant tailored for time-series forecasting, using probabilistic sparse self-attention mechanism to reduce computation and improve efficiency. (9) Autoformer 82), Autoformer enhances time-series prediction by introducing series decomposition and autocorrelation-baat based attention, enabling it to better model trend and seasonal patterns. As Informer and Autoformer can't 't process irreuularly-sampled data, we conduct spline interpolation pre-process input data for extrapolation tasks. For fair comparisons, we maintain roughly the same size of the hidden state, number of layers, and units among different methods. Architecture details and training time are reported in Appendix and Appendix [.3] respectively.
D Model Architecture and
Computing infrastructure. All experiments were run in Python 3.12 on a Mac machine equipped with an Intel Core i5 processor (3.1 GHz) and 8 GB of RAM.
For fair comparisons, we maintain roughly the same size of the hidden state, number of layers, and units. In particular, we make the dimension of the latent space Z, i.c., to be the same for Latent ODE and latent MoS. We also restrict them to having the same ODE-RNN encoder. The specific architectures and hyper-paramette
are described as follows.
Encoder. Similar to Latent ODE 18), we employ an ODE-RNN as the encoder, as illustrated in Equation 4 In this ODE-RNN, we set the GRU module the same as the commonly used GRU baseline model, where the update gate, reset gate, and state transition module have two hidden layers with the hidden unit number to be m (specific values are shown in the following parts). The ODE function module also has two hidden layers with the hidden unit number to be m. The ODE solver used to solve the ODE-RNI is the fourth-order Runge- Kutta method ("rk4").
27

---

The decoder is composed of the parameterized Lie group representation and the gating neural network. For each of the network, we still consider two hidden layers with the number of neuron to be m. Their output dimensions are directly related to m and K, which have been illustrated in the main section.
Decoder.
Multi-level MoS. In our decoder, we consider S = 2 levels for multi-level MoS (see Section 3.3) Moreover, set L 2 and L 5 to extract multi timescale equivariances.
Output layers. There are two hidden layers with number of units to be m.
Activation function and learning rate. We utilize Tanh as the activation function, similar to Latent ODEs. We set the learning rate be 0.001.
Restrictions on each of the Lie group actions. To enable smooth transitions, we enforce certain restrictions the Lie group transition during At. Specifically, for rotational symmetry, assume in Rg to be within [-0.6, 0.6]. For translation symmetry, we assume the maximum norm of c(2(t), (t), t) be less than 0.001. For scaling symmetry, we assume 1.5 < (=2(t), t) S 1.5. These restrictions can be easily achieved by using Tanh activation. Further, to avoid scaling and translation dynamics leading to vanishing exploding latent vectors, we rescale the norm of (t4) into (0.5, 1.5).
Gating mechanism. We encourage the exploration in the gating mechanism by assuming warmup period where we don't 't restrict any sparsity the gating values. After 10 epoch for the warmup, select the top Ko I K experts to build the mixture of latent flows
Hyper-parameters We establish two important hyper- parameters for different datasets: the dimension of the latent space m and the number of experts Kg for non-zero outputs. Specifically, they are shown in the following table.
Table 2: Hyper-parameter settings for different datasets.
System
Spiral
Glycolytic
Lotka
Load
Solar
Power event
Air quality
ECG
Latent dimension
15
15
15
15
15
30
15
15
Non-zero gate numbers Ko
2
4
4
6
28

---

Table Test Mean Squared Error (MSE) (x10- for interpolation tasks under varying drop rates.
Data
Drop
EqSINDy
EqGP
RNN-A,
ODE-RNN
NCDE
Contiformer
Latent ODE
Latent MoS
90%
11.21
13.88
9.39
11.47
16.30
3.23
2.30
1.93
Spiral
60%
5.67
6.97
4.02
11.01
11.39
1.66
1.23
0.97
1.14
90%
14.45
23.09
14.28
14.08
31.90
4.66
11.24
Glycolytic
60%
6.38
10.58
7.76
13.54
22.82
2.89
2.68
0.21
30%
2.74
4.26
2.34
13:27
17:44
0.43
1.64
0.18
90%
13.45
10.98
11.22
3.75
38.13
2.21
2.63
0.53
Lotka
60%
7,88
4.77
6.26
3.34
28:21
0.92
0.22
0.30
30%
1.61
2,43
3.26
20.55
0.91
0.09
90%
18.98
16.79
8.53
3.51
14.88
6.61
2.76
2.30
Load
60%
8.45
7.74
3.13
2.37
18.25
4.50
0.97
1.10
30%
5.21
4.37
1.37
2.23
20.78
4.45
0.89
0.90
90%
15.89
13.79
40.21
5.69
32.07
11.96
4.01
3.10
Solar
60%
13.57
14.23
16.09
3.94
24.39
12.06
2.65
30%
10.11
10.87
6.43
3.78
14.37
12.21
1.73
5.09
Power
60%
6,58
4.53
8.82
5.22
14.52
4.34
4.17
3.65
30%
3.28
3.75
4.34
5.13
12.02
3.40
4.01
3.14
90%
15.89
18.70
17.38
4,76
26.70
7.53
4.36
3.30
Air quality
60%
15.23
12.18
3.27
14.61
8.12
3.16
30%
10.98
14.03
6.23
2.88
8.78
6.73
2.43
1.53
BCG
60%
5.62
3.21
4.99
5.52
11.03
3.17
1.42
0.56
30%
3.08
1.05
2.28
5.41
9.92
3.07
1.35
E Supplementary Experimental Results
E.1 Table of Interpolation Results
We present the tabular results for Fig. as follows. Contiformer, Latent ODE, and Latent MoS learn the continuous latent ODE in the decoding process, thus achieving the best performances. Among them, Latent MoS leverages geometric priors to structure the latent trajectories. Across all datasets, Latent MoS achieves relative improvements ranging from 15% to 97%.
29

---

E.2 Predicted Curve Visualization
We visualize the predicted trajectories of three complex ODE systems, including the Spiral system, the Glycolytic Oscillator, and the Lotka- Lotka-Volterra system, under low-resolution data conditions with drop rate of 90%. The ground truth trajectories are shown black curves, the observed low-resooe lowrresolution data points are plotted as blue dots, and the predicted trajectories from both the Latent ODE model and our proposed I Latent MoS model are illustrated as orange dashed curves. shown in Fig. 6 the Latent MoS model consistently provides more accurate reconstructions of the underlying dynamics, which we attribute its enhanced capability to preserve hidden symmetries in the data.
Dimension
Dimansion
Dimension
Dimansion
1.0
0.5
Spiral 3.0-
0.0-
0.0
-05-
-0.2
-0.2
Glycolytic Oscillator
-0.--
-0.5
-0.8
-0,00-
Voltera
-075
-100-
(oomalizzd)
(tomnaliad))
(somnellaedd)
Ground
Observed
Prediction
Figure 6: Comparison of ground truth trajectory and predicted trajectory for three complex ODE systems (Spiral, Glycolytic Oscillator, and Lotka-Voltenn Volterra) under low-resolution data with a 90% drop rate.
E.3
Training Time
We report the training time for different models in the following table. The results show that methods can achieve relatively moderate training time, comparable to Latent ODE and ODE-RNN, We eliminate the results EqSINDy and EqGP methods because they require separate symmetry discovery and symbolic regression processes.
Table 4: Training time (minutes) for different systems and models.
Data
RNN-A,
ODE-RNN
NCDE
Contiformer
Informer
Autoformer
Latent ODE
Latent MoS
Spiral
4.9
6.5
46.4
111.2
70.7
58.5
8.4
Glycolytic
7.7
10.4
81.2
162.5
121.9
91.7
12.2
15.5
Lotka
9.4
10.2
74.5
193.3
125.9
90.3
12.6
15.4
Load
7.5
10.1
80.4
168.8
120.6
98.5
12.0
13.4
Solar
11.4
13.4
83.6
204.1
150.6
112.5
18.1
Power event
19.4
118.8
317.4
211.6
165.6
21.2
25.4
Air guality
6.0
6.7
$3.0
116.9
85.8
63.2
8.4
10.0
ECG
6.8
9.2
59.7
153.3
103.9
76.4
10.4
13.3
30


Reviews:
[{"review_id": "tgUI5Wl79M", "reviewer": "Reviewer_C3FL", "summary": "The authors propose the Latent Mixture of Symmetries (Latent MoS), a model for learning dynamics by leveraging symmetry as an inductive bias. Unlike traditional approaches, which either impose a priori available symmetries or discover global ones, Latent MoS learns a mixture of local latent symmetries through Lie group transformations. As a result, it preserves local equivariance and allows for time-dependent adaptations. To enable long-term equivariance, the authors propose hierarchically stacking MoS modules. The authors provide a theoretical justification for the model's ability to preserve equivariance. Empirical experiments in physical systems demonstrate improved performance over baselines in terms of interpolation, extrapolation, and sample efficiency.", "questions": "- Can you provide qualitative evidence to support that latent symmetries are interpretable?", "limitations": "Yes.", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- Quality:\n  - The authors propose a mathematically rigorous model with formal guarantees of equivariance preservation.\n  - Experimental results, with thoroughly studied ablations, on a diverse set of physical systems demonstrate strong performance in low-data settings.\n- Clarity:\n  - The flow from single-group equivariance to mixtures and temporal hierarchy clarifies the fundamental ideas.\n  - In general, the paper is well-written.\n- Significance:\n  - The MoS formulation provides structured latent dynamics in various settings, including model-based RL and scientific computing.\n- Originality:\n  - Time-varying mixture of experts for modeling latent symmetry transformations seems like a novel idea.\n\nWeaknesses:\n- Quality:\n  - The ablation study does not address imperfect gating.\n- Clarity:\n  - Derivations in section 3, especially 3.2, may benefit from a more intuitive explanation to non-expert readers.\n- Significance:\n  - Applicability to closed-loop control, planning, RL, etc, has not been empirically tested.\n- Originality:\n  - The novelty mainly lies in the combination of well-known components.", "was_revised": true, "final_justification": "I have decided to increase my score from borderline accept to accept based on the detailed response of the authors.", "rebuttal": "**Q1. Imperfect gating**. Our ablation study in Section 4.4 indeed reveals how expert selection influences performance. Specifically, including unnecessary symmetry experts can lead to a little performance degradation, while removing essential ones significantly harms results. \n\nHowever, we emphasize that our MoS gating mechanism can effectively select the true experts. For example, in Section 4.4 and for the Glycolytic system, when we include all symmetry experts, the model still achieves a low MSE of 0.0033. Importantly, the learned gating weights are highly concentrated: the dominant expert, a second-order transformation (rotation × scaling), receives a gating weight of 0.97, while other gating weights are negligible (e.g., 0.025 for scaling × rotation and <0.005 for the rest). This indicates that the gating mechanism correctly identifies and prioritizes the relevant symmetry flow despite the presence of extraneous experts.\n\nMoreover, we observe that removing unnecessary experts can further improve performance. When the translational expert is removed (which is unnecessary for Glycolytic dynamics), the MSE improves to 0.001 and the gating weight for rotation × scaling increases to 0.99. This demonstrates that while the gating works well, a simplified expert set can enhance efficiency and robustness. To formalize this process, our ablation study suggests that one can perform a validation-based procedure to progressively remove unnecessary experts and compare the resulting MSE. Importantly, the gating mechanism remains valuable in this process: it helps identify the dominant expert(s) within a candidate subset, effectively narrowing down the search space. This makes the validation process more efficient and scalable. Overall, this approach provides a practical and systematic method for expert selection, particularly when symmetry priors are uncertain or only partially known.\n\nLooking forward, this validation-based insight opens the door to automated expert selection. One promising direction is to leverage Neural Architecture Search techniques, such as RL [zoph et al. 2017] or differentiable methods [zhang et al. 2022]. NAS can search over expert combinations and minimize validation MSE. Alternatively, foundation models and LLMs [tsai et al. 2023] can be used to suggest likely symmetry groups based on contextual or domain-specific descriptions of the system. We also envision incorporating Bayesian optimization or meta-learning strategies [shaw et al. 2019] to efficiently guide gating under limited data. While we leave full automation to future work, our findings suggest that expert selection can be both principled and learnable.\n\n**Q2. Readability**. The old version aims to give rigorous definitions in m-dimensional latent space, which makes it hard for readers without Lie group knowledge. We are happy to improve the readability by\n\n**(1) Architectural overview.** We will add a dedicated paragraph that walks the reader through how the mixture-of-symmetries (MoS) module is deployed.\n\n**(2) Expert interpretation and parameterization.** We will explain each expert with concrete examples (e.g., 3D rotation, translation, etc.). For each, we will describe the Lie-based parameterization and provide references. \n\n**(3) Visual clarification in Fig. 1.** We will revise Figure 1 to include 3D visualizations of symmetry groups. This will allow readers to immediately grasp the role of each expert.\n\n**Q3. Control tests**. As the topic is dynamic learning, we focused on learning tasks. We are glad to provide a control test from our draft of latent MoS for power system frequency control. The task is to stabilize the generator's frequency. Past methods [Cui et al. 2023] assume a second-order ODE model, i.e., swing equations, to model frequency dynamics. However, real-world dynamics are higher-order. Hence, we utilize latent MoS to learn it. In general, we compare model-free proximal policy optimization (PPO), model-based policy optimization (MBPO) with a Gaussian process (MBPO-GP), MBPO with Latent MoS (MBPO-MoS), and MBPO with the swing equation (MBPO-SE). We track the frequency deviation over time. At time $t = 0.5$, our MBPO-MoS reaches $-0.287$, compared to $-0.975$ for PPO, $0.100$ for MBPO-GP, and $-0.499$ for the MBPO-SE. By the final time point $t = 1.5$, our MBPO-MoS stabilizes near zero with a deviation $-0.024$, the lowest among all methods. In contrast, PPO remains highly unstable ($-0.928$), MBPO-GP yields $0.100$, and  MBPO-SE yields $-0.115$. The general cost (integral of frequency deviations) in the control process is $0.081$ (MBPO-MoS), $0.163$ (MBPO-SE), $0.257$ (MBPO-GP), and $0.51$ (PPO). Our learned dynamics in Latent MoS can much better predict the future frequency, largely improving control performance.\n\n**Q4. Novelty**. The paper is not a simple combination of existing methods, as explained below.\n\n**1. Symmetry-preserving experts**:\nTraditional MoE employs generic neural networks as experts, which can't fit the goal of symmetry preservation. We provide formal proofs showing how each expert guarantees equivariance with specifically parameterized experts. This Lie expert has never been discussed before.\n\n**2. Mixture of latent symmetry flows**:\nPrior methods often assume a single symmetry group. In contrast, Latent MoS introduces a mixture model over multiple symmetry groups, significantly expanding the expressiveness of latent dynamics and enabling the model to adapt to complex symmetries.\n\n**3. Multi-level MoS architecture**:\nOur design supports stacked MoS layers across different temporal resolutions, allowing the model to extract both short- and long-term symmetry patterns. This stands in contrast to prior work, which typically assumes fixed or temporally uniform symmetries, and makes our model more suitable for complex and evolving environments.\n\n**4. Connections to safety guarantees**:\nBecause each symmetry expert corresponds to a structured Lie group matrix, the latent dynamics are linearly interpretable. This leads to deep connections with control theory and safe reinforcement learning. For instance, in safe RL, forward-invariant safety sets [Emam, et al. 2022] link symmetry and equivariance to safety, which can be efficiently estimated using our latent linear symmetry flows. This potentially bypasses expensive formal verification tools like SMT solvers [Chang et al. 2019]. We are actively pursuing this as part of ongoing research in safe RL.\n\n**5. Significant empirical advances**:\nAcross multiple benchmarks, Latent MoS outperforms strong baselines including ODE-based models (Latent ODE, NCDE), sequence models (RNN, ODE-RNN), and transformer-based architectures (Informer, Autoformer). These results validate that symmetry-aware modeling leads to substantial improvements.\n\n**Q5. Interpretability**.\n\nWhile interpretability is not the primary motivation of our model, it emerges naturally as a byproduct of the linear, Lie group-structured latent dynamics. To support this, we provide both qualitative and quantitative evidence:\n\n\n**1. Qualitative evidence**. As shown in Fig. 2, we visualize the latent trajectories in a 2D setting for three different systems, each dominated by rotation and scaling symmetries (the left column of Fig. 2). These geometric structures are clearly expressed in the latent space learned by Latent MoS. Specifically, our latent trajectories (the right column of Fig. 2) exhibit smooth rotational and scaling motions that align with the known system dynamics. In contrast, the trajectories learned by Latent ODE (the middle column of Fig. 2) lack such structure and are difficult to interpret, highlighting the advantage of our symmetry-preserving formulation. \n\nIt is worth noting that the Glycolytic and Lotka-Volterra systems introduce nonlinear distortions due to their inherent dynamics (see Equations (18) and (19) in the Appendix). Despite this, Latent MoS effectively captures the linear symmetry structure in the latent space, while leaving the nonlinearity to the encoder and decoder networks. This result is consistent with the theoretical decomposition described in Section 2.2.\n\n\n**2. Quantitative evidence**. We also provide quantitative support that Latent MoS learns interpretable and symmetry-consistent dynamics in the latent space. In the Glycolytic system, for example, even when all symmetry experts are included, the model achieves a low MSE of 0.0033. Importantly, the gating weights are highly concentrated: the dominant expert, i.e., a second-order transformation (rotation × scaling), receives a weight of 0.97, while the next highest (scaling × rotation) receives 0.025, and the remaining experts collectively contribute less than 0.005. This concentration indicates that the model identifies and relies on the correct symmetry transformation.\n\nTo further support this, we conducted an additional experiment motivated by the PCA visualization in Fig. 2, which shows that a 2D latent space suffices to capture the dynamics of the tested systems. We set the latent dimension $m=2$ and fixed both the encoder and decoder to identity mappings. This avoids the encoder and decoder jointly transforming the latent dynamics into an equivalent—but potentially different from the true governing equation.\n\nThen, for the spiral dataset, Latent MoS recovers a scaling factor of 0.98 and a rotation matrix R = [0.968, -0.248; 0.248, 0.968], which closely matches the true dynamics used in data generation. The average error compared to the ground truth symmetry is less than 0.05, and the gating weight for this expert (rotation × scaling) is 1. This experiment demonstrates that Latent MoS can faithfully recover interpretable symmetry structures (or their equivalent forms) directly from data."}, {"review_id": "MIVNhoXATe", "reviewer": "Reviewer_pEQx", "summary": "This paper introduces Latent Mixture of Symmetries (MoS), a continuous-time dynamics model that keeps physical equivariance by treating each latent time-step as a learnable Lie-group action and then blending several such actions with a Mixture-of-Experts gate. The theoretical analysis gives a simple commutation criterion that guarantees any selected action remains equivariant with respect to the known symmetry in the observation space. Experiments on both synthetic ODEs and real-world datasets show improved accuracy and qualitatively symmetry-consistent latents compared with baselines.", "questions": "1. It would be helpful to state explicitly which symmetry (or combination) exists in each experimental dataset and provide quantitative evidence that MoS captures it (the PCA in Fig. 2 is a good start, but more discussion would be beneficial).\n2. Why is Corollary 1 limited to planar rotation? Did any experiment involve true 3D rotational symmetry, and if so how was it handled?\n3. Could the authors clarify how MoS might extend beyond rotation/translation/scaling?\n4. How sensitive is performance to the size of the symmetry candidate library? Will more irrelevant experts hinder the performance?", "limitations": "Although the authors list some limitations, the paper would benefit from a failure-case analysis, e.g., examples where the gating network consistently picks the wrong expert and how that might be resolved.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength\n1. Using MoS to learn a data-driven mixture of symmetry experts is a neat way to model symmetry in dynamical systems with unknown and/or shifting symmetries.\n2. The commutation-based design yields interpretable latent operators while still providing provable equivariance guarantees, which helps both sample efficiency and interpretability.\n\nWeakness\n1. The paper does not compare with symmetry-discovery baselines that explicitly discover symmetries from data. \n2. The paper does not compare itself against the symmetry-breaking literature (methods that gradually relax or modulate equivariance). Can one view the proposed method like breaking the symmetry from the affine group? A discussion here would be helpful.", "was_revised": true, "final_justification": "The authors addressed most of my concerns, so I increased my score accordingly.", "rebuttal": "**Q1. Symmetry-discovery baseline**. The reason we didn't include this baseline is that this type of method has strong requirements for data quality, but we mainly test realistic sample-inefficient cases. For example, GAN and SINDy in [Yang et al., 2024] both require many samples. Naturally, these baselines perform worse than other ODE-structured models like Latent ODE. As the reviewer asked, we conducted the experiments using the official code from [Yang et al., 2024] and tested two recent symmetry-discovery methods: EquivSINDy-r (SINDy-based) and EquivGP-r (Genetic Programming-based). They all require LaLiGAN to discover symmetry, which is further used in SINDy or GP as regularization. We report interpolation errors under a 30\\% data drop rate. For the Spiral, Glycolytic, and Lotka systems, the MSEs ($\\times 10^{-2}$) are as follows: EquivSINDy-r: 2.12, 2.74, 2.42; EquivGP-r: 2.13, 4.26, 1.61, all of which are around 4× to 10× larger than the results achieved by our Latent MoS. Similar trends hold across other datasets.  These findings highlight that existing symmetry-discovery methods based on GAN priors and symbolic regressors (SINDy/GP) are highly sensitive to data quality and quantity. In contrast, our method remains robust under sparse and limited trajectories.\n\n\n**Q2. Symmetry-breaking literature**. Our work identifies symmetry breaking in two orthogonal axes: ** a mixture of groups and a short- or long-time dependence** (see Section 2.1). This breaking changes the restrictive assumption of a fixed affine group. The mixture of groups appears in literature [Kim et al., 2023], and the locally adaptable symmetry breaking is discussed in [Wang et al., 2022; Wang et al., 2023; Ouderaa et al., 2022]. They are independently discussed, but our method addresses both in a unified framework. \n\nTo model symmetry breaking, past methods (1) utilize learnable or non-stationary weights [Wang et al., 2022], (2) add soft regularization  [Kim et al., 2023; Finzi et al., 2021], or (3) construct a symmetry-breaking set as auxiliary inputs [Xie \\& Smidt, 2024]. Our model adopts method (1) without the need to construct additional terms or sets. \n\n**Q3. Identified symmetries**. \n\nOur paper states that in Spiral, Glycolytic, and Lotka-Volterra systems, the dominant symmetry is rotation × scaling. It is worth noting that the Glycolytic and Lotka-Volterra systems introduce nonlinear distortions due to their inherent dynamics (see Equations (18) and (19) in the Appendix). Fig. 2 shows that latent MoS recovers this symmetry, but latent ODE can't. For other realistic systems, they don't have ground-truth symmetries.\n\nWe also have quantitative results. In the Glycolytic system, for example, even when all symmetry experts are included, the model achieves a low MSE of 0.0033. Importantly, the gating weights are highly concentrated: the dominant expert, i.e., a second-order transformation (rotation × scaling), receives a weight of 0.97, while the next highest (scaling × rotation) receives 0.025, and the remaining experts collectively contribute less than 0.005. This concentration indicates that the model identifies the correct symmetry.\n\nWe also conducted an additional experiment. We set the latent dimension m=2 and fixed both the encoder and decoder to identity mappings. This avoids the encoder and decoder jointly transforming the latent dynamics into an equivalent—but potentially different from the true governing equation.\n\nThen, for the spiral dataset, Latent MoS recovers a scaling factor of 0.98 and a rotation matrix R = [0.968, -0.248; 0.248, 0.968], which closely matches the true dynamics used in data generation. The average error compared to the ground truth symmetry is less than 0.05, and the gating weight for this expert (rotation × scaling) is 1. This experiment demonstrates that Latent MoS can faithfully recover interpretable symmetry structures (or their equivalent forms) directly from data.\n\n**Q4. Corollary 1 issue**. The restriction to planar rotation arises because general rotations are non-commutative. E.g., a die rotates along the x-axis and then z-axis, or along the z-axis and then x-axis, will cause different results. In contrast, planar rotations (fixed rotation axis) are commutative. The result is rigorously proved in Appendix A.3, meaning that the state should rotate along one axis at least locally to preserve rotation symmetry. If a rotation continuously changes the axis, symmetry is completely removed.\n\nHowever, our model is not restricted to 2D but focuses on dynamics in higher dimensions along an axis at a period. As shown in Corollary 1, this is achieved by parameterized planar rotation $R_{\\theta}$ (define the rotation axis) and a projection matrix $\\hat{P}$ (lift the dimension). For example, we include a 10D power event dataset (results in Table 1) where Latent MoS outperforms all baselines and angle rotations are almost synchronized due to physical constraints. \n\n**Q5. Symmetry extension**. While the current implementation focuses on rotation, translation, and scaling, which already form a rich and compositional symmetry basis, we outline multiple ways in which MoS can be extended:\n\n**1. Higher-order compositions of existing symmetries**. Our model already supports second-order multiplicative compositions in Corollary 3 (e.g., scaling × rotation). This formulation naturally extends to higher-order group actions by multiplying additional Lie group matrices, which captures very complicated interactions among different symmetries. \n\n**2. Generalized symmetry via nonlinear encoder–decoder structure**. As shown in Equation (2) and (3), MoS leverages a decomposed symmetry formulation. By the universal approximation theorem (e.g., [38] in the reference), this setup can represent any continuous nonlinear group action. This means MoS can approximate nonlinear symmetry transformations.\n\n**3. Integration of new symmetry types**. In our paper, we have shown a standard procedure to integrate other symmetry transformations into our MoS. \n\n**(1) Parameterize new group actions** (e.g., discrete symmetries, time-warping, reflection, permutation-equivariant transformations) using constrained neural networks or algebraic constructions.\n\n**(2) Add these as new experts in the MoS module**.  Since MoS is structured as a Mixture-of-Experts, each expert can encode a different symmetry family and is trained jointly through the gating mechanism.\n\nThis modularity makes the MoS architecture flexible and extensible.\n\n**Q6. Sensitivity analysis**. We performed sensitivity analysis with respect to noise levels, latent dimensions, and the number of open gates. However, we believe it's good to do an analysis with respect to the size of the candidate library. Using the Glycolytic system with a 90\\% drop rate as an example, we vary the number of symmetry experts from 1 to 9 and ensure that the correct symmetry (rotation × scaling) is always included. This helps to evaluate the impact of irrelevant experts. The resulting MSEs ($\\times 10^{-2}$) were: 0.19, 0.20, 0.20, 0.25, 0.27, 0.28, 0.31, 0.33, and 0.33, respectively. These results show that as more irrelevant experts are added, performance degrades slightly but remains stable. Even with all 9 experts, the MSE is only 0.0033, illustrated in Q3. This demonstrates that although more irrelevant experts may introduce minor degradation, the model is not highly sensitive to the library size due to the gating mechanism.\n\n**Q7. Limitation**. Our ablation study in Section 4.4 indeed reveals how expert selection influences performance. Specifically, including unnecessary experts (e.g., those not aligned with the system’s underlying symmetries) can lead to a little performance degradation, while removing essential ones significantly harms results. The former observation is illustrated using examples in answers to Q3 and Q6. \n\nFor the latter observation, for example, in the Glycolytic system, if we remove the rotation expert, the resulting MSE is 0.014, compared to 0.0033 with all experts included (Section 4.4). The gating weights shift toward translation × scaling (0.82) and scaling (0.15), with the remainder distributed across other experts (0.03). This demonstrates a clear failure case: the model attempts to approximate rotational dynamics using incorrect symmetry compositions, but performance suffers. Thus, including the correct symmetry group is critical.\n\nTo address such issues, our ablation study motivates that one can perform a validation-based procedure to progressively remove unnecessary experts and compare the resulting MSE. Importantly, the gating mechanism remains valuable in this process: it helps identify the dominant experts within a candidate subset, effectively narrowing down the search space. This makes the validation process more efficient and scalable. Overall, this approach provides a practical and systematic method for expert selection, particularly when symmetry priors are uncertain or only partially known.\n\nLooking forward, this validation-based insight opens the door to automated expert selection. One promising direction is to leverage Neural Architecture Search techniques, such as RL [zoph et al. 2017] or differentiable methods [zhang et al. 2022]. NAS can search over expert combinations and minimize validation MSE. Alternatively, foundation models and LLMs [tsai et al. 2023] can be used to suggest likely symmetry groups based on contextual or domain-specific descriptions of the system. We also envision incorporating Bayesian optimization or meta-learning strategies [shaw et al. 2019] to efficiently guide gating under limited data. While we leave full automation to future work, our findings suggest that expert selection can be both principled and learnable."}, {"review_id": "oMAPAEA2sq", "reviewer": "Reviewer_KeLt", "summary": "This paper addresses the problem of dynamic learning under data scarcity. The authors introduce the Latent Mixture of Symmetries (MoS) framework based on the Mixture-of-Experts (MoE) approach, efficiently preserving local equivariance through structured transformations. Utilizing a gating mechanism derived from MoE, MoS adaptively selects the most appropriate local Lie group symmetries. Extensive evaluations on a range of synthetic and real-world dynamical systems demonstrate promising improvements compared to several baseline methods.", "questions": "1. How sensitive is Latent MoS to the accurate identification of underlying symmetry groups? As indicated by ablation studies, including unnecessary symmetries can negatively impact performance. Could the authors comment on any conceptual connections between this scenario and overcomplete dictionary learning, considering symmetries analogous to dictionary items?\n\n2. Could the authors clarify how the interpolation and extrapolation tasks are precisely defined?\n\n3. In Figure 3 (Lotka System - Drop rate 30%), the blue bar appears to be missing. Could the authors provide clarification or correct this oversight?", "limitations": "yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths\n1. The concept of modeling different local symmetries within a single trajectory is appealing, clearly motivated, and innovative.\n2. The paper is well-structured, presenting detailed theoretical foundations supported by extensive empirical evaluations.\n3. The experimental results consistently indicate superior performance of the proposed approach compared to existing baselines.\n\n### Weaknesses\nSee Questions below.", "was_revised": true, "final_justification": "I am happy with the authors' rebuttal and would keep my rating of borderline accept.", "rebuttal": "**Q1. Sensitivity and relations to dictionary learning**. \n\nWe provide an additional sensitivity analysis to assess the impact of including irrelevant symmetry groups. Specifically, we consider the Glycolytic system with a 90\\% drop rate and vary the number of available symmetry experts from 1 to 9, while always including the necessary one (rotation × scaling). The resulting MSEs ($\\times 10^{-2}$) for 1 to 9 experts are: 0.19, 0.20, 0.20, 0.25, 0.27, 0.28, 0.31, 0.33, and 0.33, respectively. These results show that introducing irrelevant experts can slightly degrade performance; however, even with all experts, the model still achieves a low MSE of 0.0033. In this setting, the learned gating weights remain concentrated: the dominant expert (rotation × scaling) receives a gating weight of 0.97, while others are negligible (e.g., 0.025 for scaling × rotation and $<0.005$ for the remaining). This demonstrates that the gating mechanism effectively identifies the correct symmetry. Nonetheless, we appreciate the reviewer’s suggestion regarding overcomplete dictionary learning and recognize that advanced techniques from that literature could further improve MoE selection in the presence of redundant experts.\n\n\nOvercomplete dictionary learning is a long-standing topic that seeks to represent data as sparse linear combinations of basis atoms, enabling compact and interpretable representations. Recent theoretical advances have introduced provable algorithms for learning incoherent dictionaries [Arora et al., 2014; Sulam et al., 2022] and regularization strategies for identifiability and convergence guarantees [Sun et al., 2025]. Moreover, recent works [Wang et al., 2022; Mudide et al., 2025] have uncovered deep connections between dictionary learning and Mixture-of-Experts (MoE) models, including our Mixture-of-Symmetries (MoS): each expert can be viewed as a dictionary atom, and the gating network plays an analogous role to sparse coding, selecting a minimal set of active atoms. This conceptual link suggests that advances in dictionary learning, such as novel regularization and pruning, could inspire improved training and regularization techniques for MoS. We leave this direction as a promising avenue for future work.\n\n**Q2. Interpolation and Extrapolation clarification**. \n\nWe adopt a standard formulation consistent with prior work on continuous-time and irregularly sampled time series, such as Latent ODE, Contiformer, RNN-$\\Delta\\_t$, etc. For example, Appendix 1 in Laten ODE [Rubanova et al. 2019] gives a good description. The setup is as follows:\n\n(1) Interpolation. Given a time series with time points $(t\\_0,\\cdots,t\\_N)$, we condition on the subset of points from $(t\\_0,\\cdots,t\\_N)$ with a data drop rate (30\\%, 60\\%, 90\\% in Experiments) and reconstruct the full set of points in the same time interval.\n\n(2) Extrapolation. We split the time series into two parts $(t_0,\\cdots,t_{N/2})$ and $(t_{N/2},\\cdots,t_N)$. We input the first half of the time series and predict the second half. We apply the same random drop procedure only on the input half, and the model is tasked with predicting the entire future segment beyond the observed range.  \n\nAdditional details regarding dataset sizes, number of trajectories, and time resolution for each system are provided in Appendix B in our paper.\n\n**Q3. Figure issue**. \n\nWe thank the reviewer for pointing out the apparent omission of the blue bar in Figure 3 (Lotka system – Drop rate 30%). This issue was caused by an unintentional use of the plotting command \"ax.set_ylim(0.001, 0.4)\" during figure generation for better visual look, which truncated values below 0.001. As a result, both Latent ODE (0.0009) and our proposed Latent MoS (0.0006), which achieve significantly lower MSEs compared to other methods (e.g., NCDE: 0.2055), were visually suppressed in the plot. These MSE values are explicitly reported in Table 3 and Appendix E.1. We appreciate the reviewer’s careful observation and have corrected the figure to reflect the full MSE range. **We have also reviewed and refined all other figures to ensure consistent and accurate presentation**.\n\nImportantly, this result underscores a central contribution of our work. As shown in Fig. 2 in Section 4.1 and Fig. 6 in Appendix E.2, the Lotka and Glycolytic systems exhibit nonlinear oscillatory dynamics and are particularly challenging under partial observability. As described in Section 4.2, our Latent MoS model is uniquely designed to capture such structure by learning a mixture of symmetry-preserving latent dynamics, enabling both flexible interpolation and inductive bias. While conventional models (e.g., latent ODE) learn smooth latent flows without exploiting geometric invariances, our approach successfully learns structured latent trajectories that generalize well."}, {"review_id": "746v3n5WLq", "reviewer": "Reviewer_UuCF", "summary": "This paper proposes combining MoEs with Neural ODEs to learn latent dynamics which respect symmetry. The objective is similar to symmetry discovery, but avoids common limitations such as error propagation and limited expressiveness of multiple symmetry groups. The authors use a mixture of latent flows where each flow is governed by a different Lie group and use a encoder-decoder architecture for nonlinear group actions on the data. Experiments on many different domains with ODE dynamics and real-world data show that their method outperforms baselines.", "questions": "- Are the symmetry groups known a priori? Or can the method learn (perhaps through the gating mechanism) subgroups of the general linear/affine group.\n- How are learned symmetries preserved across time steps? Would this method correct learn the Lie point symmetries of the governing equations?", "limitations": "See weaknesses and questions.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 4, "significance": 2, "originality": 3, "strengths_and_weaknesses": "## Strengths\n- The authors provide various theoretical analyses on how Latent MoS can learn several different symmetry groups.\n- The authors experiment on several datasets, including real-world domains such as residential electricity consumption, solar energy, etc.\n\n## Weaknesses\n- I feel like the core flaw of the proposed approach is the linear mixing/weighted average (Equation 8). This seems to go against how symmetries should be composed together (composition/matrix multiplication). It's not clear to me how this method really does learn both symmetries with this mixing technique and it seems to in fact break the group structure, since the mixture isn't a valid group element. Can the authors clarify?\n- Furthermore, the ODE-RNN structure doesn't seem to use the same mixture or symmetry groups for the latent dynamics at all timesteps. Why would a symmetry be present at a certain timestep but not present at all other timesteps? The proposed method doesn't seem to preserve the mixing weights from previous timesteps.\n- LaLiGAN (Reference [38] Yang et al. 2023) and perhaps a downstream dynamics model should be included as baseline. It seems to have the same objective (learning latent symmetries) as this paper.", "was_revised": true, "final_justification": "The author's response has changed my view of the paper, in particular the motivation behind using a mixture of symmetry groups (which produces a nonvalid symmetry) for specific dynamics systems and real-world datasets.", "rebuttal": "**Q1. Flaws due to the mixture model**.\n\nIt is correct that the mixture in Equation (8) does not define a valid Lie group action. This is our deliberate and innovative design to **model the system dynamics**. \n\nOur design only requires that the **components of the latent vector** have symmetry transformations. This is because in Introduction and Section 2.1, we discussed that many real-world systems are influenced by multiple latent factors acting simultaneously, each evolving under a distinct Lie group action. For example, damping (scaling) and oscillation (rotation) forces co-exist in many mechanical and electrical systems. These effects operate concurrently and cannot be captured by a single group action or composition. Instead, they give rise to a superposition of latent flows, each governed by its own symmetry structure. Our model captures this behavior by treating each expert as a valid Lie group action and using a gating network to assign weights that combine their effects. While the resulting update (Equation 8) is not itself a group action, it provides a principled and expressive way to model complex system dynamics. **In general, Equation (8) defines the system's latent dynamics, but doesn't need to be a valid Lie transformation**.\n\nFor example, in our experiments with power event datasets, the results show that the gating weights for rotation and scaling are 0.83 and 0.17, respectively. This indicates that both **intrinsic oscillations and the extrinsic damping force** from devices like a power grid stabilizer (PSS) are simultaneously captured. Finally, thanks to our highly expressive yet symmetry-preserving design, Latent MoS performs strongly on complex real-world dynamical systems such as power events, ECG, and air quality. It achieves 30–60\\% reductions in MSE compared to state-of-the-art methods like Latent ODE.\n\nWhen the gating network concentrates on a single expert, **our model reduces to a special case** where the dynamics are governed by a single (nonlinear) Lie group action.  For example, our answers in Q5 show how the synthetic ODE datasets lead to single rotation × scaling symmetry.\n\n**Q2. Time range of the symmetry.**\n\nThis question directly echoes another key contribution we discuss in Section 2.1: short- and long-term equivariance in dynamical systems. Unlike classical equivariant models that enforce fixed symmetry globally, our method allows the effective symmetry structure to vary and persist over flexible time horizons.\n\nIn our model, **the mixing weights and symmetry transformations are held fixed within predefined temporal intervals**, which ensures consistent symmetry behavior over that range. Specifically, as shown in Equation (9) in Section 3.2, for each interval indexed by $l$, we define a start time $t^{(l)}$ and use the tuple $(t^{(l)},\\boldsymbol{z}(t^{(l)}))$\nas input to neural networks that generate both the gating weights and the symmetry matrices. This construction ensures that within a given interval, **the same mixture of symmetry-preserving flows** is applied. Moreover, as described in Algorithm 1 (Section 3.3), our multi-level Latent MoS architecture introduces variable interval lengths across different temporal resolutions. This enables the model to **flexibly capture short-term or long-term equivariance**, depending on the system's structure and data complexity. \n\nFor example, in our load forecasting dataset with 144 time points per day (i.e., 10-minute resolution), we define two temporal levels: a short-term level spanning 3 hours (18 points) and a long-term level covering the full day. The results show that the 3-hour interval exhibits a dominant scaling symmetry (gating weight = 0.96), reflecting local growth or decay. In contrast, the full-day interval reveals a dominant rotation symmetry (gating weight = 0.91), indicating a cyclic structure in the dynamics, such as daily progression through peak and off-peak periods.\n\nFinally, in our ODE-based synthetic datasets, even when we define both short- and long-term levels, the gating weights and learned symmetries remain nearly the same values across levels, demonstrating that the system exhibits a global symmetry structure. Specific results are illustrated in our response to Q5.\n\n**Remark: symmetry breaking**. As discussed in our responses to Q1 and Q2, one of the key contributions of Latent MoS is its ability to model not only systems with global symmetries but also those that exhibit symmetry breaking, i.e., the assumption of strict, global invariance does not hold uniformly over time. Our work identifies symmetry breaking in two orthogonal axes: **a mixture of groups and a short- or long-time dependence** (see Section 2.1). This breaking changes the restrictive assumption of a fixed affine group. The mixture of groups appears in literature [Kim et al., 2023], and the locally adaptable symmetry breaking is discussed in [Wang et al., 2022; Wang et al., 2023; Ouderaa et al., 2022]. They are independently discussed, but our method addresses both in a unified framework.\n\nTo model symmetry breaking, past methods (1) utilize learnable or non-stationary weights [Wang et al., 2022], (2) add soft regularization [Kim et al., 2023; Finzi et al., 2021], or (3) construct a symmetry-breaking set as auxiliary inputs [Xie & Smidt, 2024]. Our model adopts method (1) without the need to construct additional terms or sets.\n\n**Q3. Baseline for symmetry discovery and dynamic learning**. \n\nThe reason we didn't include this baseline is that this type of method has strong requirements for data quality, but we mainly evaluate realistic sample-inefficient cases. For example, GAN and SINDy in  [Yang et al., 2024] both require significant samples. Naturally, these baselines perform worse than other ODE-structured models like Latent ODE. As the reviewer asked, we conducted the experiments using the official code from [Yang et al., 2024] and tested two recent symmetry-discovery methods: EquivSINDy-r (SINDy-based) and EquivGP-r (Genetic Programming-based). They all require LaLiGAN to discover symmetry, which is further used in SINDy or GP as regularization. We report interpolation errors under a 30\\% data drop rate. For the Spiral, Glycolytic, and Lotka systems, the MSEs ($\\times 10^{-2}$) are as follows: EquivSINDy-r: 2.12, 2.74, 2.42; EquivGP-r: 2.13, 4.26, 1.61, all of which are around 4× to 10× larger than the results achieved by our Latent MoS. Similar trends hold across other datasets.  These findings highlight that existing symmetry-discovery methods based on GAN priors and symbolic regressors (SINDy/GP) are highly sensitive to data quality and quantity. In contrast, our method remains robust under sparse and limited trajectories, which are typical in dynamical systems.\n\n**Q4. Symmetry group as a priori**. \n\nUnlike many past methods [Weiler \\& Cesa, 2019;Cohen et al., 2019a] that require groups as a prior, latent MoS learns symmetry from data via the gating mechanism. Each expert in our model corresponds to a subgroup of the affine group (e.g., rotation, translation, etc.), and the gating network assigns weights to form a data-driven mixture of these symmetry-preserving flows. This design allows the model to adaptively select and combine latent symmetry structures at different times and regions, offering high expressive power for modeling complex and potentially symmetry-breaking systems.\n\n**Q5. Learn Lie point symmetry**.\n\nOur method can learn the Lie point symmetry. In our experiment, the physical systems in Fig. 2 contain global Lie point symmetries. The visualization of the latent trajectory indicates that one of our latent components accurately captures the dominant symmetry. For example, the Glycolytic system is governed by rotation × scaling symmetry.  In our result, the gating weights are highly concentrated: the dominant expert, i.e., a second-order transformation (rotation × scaling), receives a weight of 0.97, while the next highest (scaling × rotation) receives 0.025, and the remaining experts collectively contribute less than 0.005. Thanks to this accurate selection, the model achieves an extremely low MSE of 0.0033. The low interpolation/extrapolation error indicates the true symmetry is approximately captured. \n\nWe feel that it's good to test if the model can directly estimate the ground-truth symmetry transformation. Hence, we **conducted an additional experiment**. We set the latent dimension $m=2$ and fixed both the encoder and decoder to identity mappings. This eliminates the possibility of the encoder and decoder jointly transforming the latent dynamics into an equivalent—but potentially different from the true governing equation. \n\nThen, for the spiral dataset, Latent MoS recovers a scaling factor of 0.98 and a rotation matrix R = [0.968, -0.248; 0.248, 0.968], which closely matches the true dynamics used in data generation. **The average error compared to the ground truth symmetry is less than 0.05, and the gating weight for this expert (rotation × scaling) is 1**. This experiment demonstrates that Latent MoS can faithfully recover interpretable symmetry structures (or their equivalent forms) directly from data.\n\nFinally, we note that restricting the model capacity to learn exact the true symmetry transformation is uncessary because (1) real-world systems can have changing symmetries and (2) the true transformation matrix can be represented by many equivalent forms in our encoder-decoder structure. The test MSE in interpolation/extrapolation is a more appropriate mearue for our dynamic learning problem."}]

Author rebuttal:

Meta-review: This paper introduces Latent Mixture of Symmetries (MoS), a continuous-time dynamics model that preserves physical equivariance by treating each latent time step as a learnable Lie group action and blending multiple such actions through a Mixture-of-Experts gating mechanism. The strengths of the paper lie in its significance, clarity, and originality, although the evaluations could be further strengthened. The authors have adequately addressed most of the reviewers’ comments, and the work makes a valuable contribution to the field.
Decision: Accept (poster)