Rethink GraphODE Generalization within Coupled Dynamical System
Authors: Guancheng Wan
Zijie Huang
Wanjia Zhao
Xiao Luo
Yizhou Sun
Wei Wang
Venue: ICML 2025 spotlightposter (2025)
Keywords: Graph Machine Learning
GraphODE
Dynamical System
Primary area: applications->chemistry_physics_and_earth_sciences
TL;DR: We propose GREAT, a GraphODE framework that disentangles static and dynamic states, regularizes coupling dynamics, and significantly improves generalization across diverse dynamical systems.

Abstract: Coupled dynamical systems govern essential phenomena across physics, biology, and engineering, where components interact through complex dependencies. While Graph Ordinary Differential Equations (GraphODE) offer a powerful framework to model these systems, their **generalization** capabilities degrade severely under limited observational training data due to two fundamental flaws: (i) the entanglement of static attributes and dynamic states in the initialization process, and (ii) the reliance on context-specific coupling patterns during training, which hinders performance in unseen scenarios. 
In this paper, we propose a Generalizable GraphODE with disentanglement and regularization (GREAT) to address these challenges. Through systematic analysis via the Structural Causal Model, we identify backdoor paths that undermine generalization and design two key modules to mitigate their effects. The *Dynamic-Static Equilibrium Decoupler (DyStaED)* disentangles static and dynamic states via orthogonal subspace projections, ensuring robust initialization. Furthermore, the *Causal Mediation for Coupled Dynamics (CMCD)* employs variational inference to estimate latent causal factors, reducing spurious correlations and enhancing universal coupling dynamics. Extensive experiments across diverse dynamical systems demonstrate that ours outperforms state-of-the-art methods within both in-distribution and out-of-distribution.

Rethink GraphODE Generalization within Coupled Dynamical System
Guancheng Wan
Zijie Huang' Wanjia Zhao
Xiao Luo'
Yizhou Sun
Wei Wang'
Abstract
(I) Static Property and Dynamic State Entangled
Coupled dynamical systems govern essential phe- nomena across physics, biology, and engineering. where components interact through complex de- pendencies. While Graph Ordinary Differential Equations (GraphODE) offer a powerful frame- work to model these systems, their generalization capabilities degrade severely under limited ob- servational training data due to two fundamental flaws: (i) the entanglement of static attributes and dynamic states the initialization process, and (ii) the reliance on context-specific coupling pat- terns during training, which hinders performance in unseen scenarios. In this paper, propose Generalizable GraphODE with disentanglement and regularization (GREAT) to address these chal- lenges. Through systematic analysis via the Struc- tural Causal Model, we identify backdoor paths that undermine generalization and design two key modules to mitigate their effects. The Dynamic- Static Equilibrium Decoupler (DyStaED) disen- tangles static and dynamic states via orthogonal subspace projections, ensuring robust initializa- tion. Furthermore, the Causal Mediation for Cou- pled Dynamics (CMCD) employs variational in- ference to estimate latent causal factors, reduc- ing spurious correlations and enhancing universal coupling dynamics. Extensive experiments across diverse dynamical systems demonstrate that ours outperforms statc-of-the-a methods within both in-distribution and out-of nut-of-distribution
Eniiyy
Materials
Local
Temperatune
Esternal
Wrong Entanglement
Presuue
Dynamic State Evalation
(II) Learn Coupling Dynamics with Bias
limooni
Context-
Limited
Specifie
Training
Coupling
Unknown Condition
Duta
Degruded
Generalization
Figure I. Problem Illustration. In coupled dynamical systems, GraphODE models rely on partially observed trajectorics, which can result in bias towards the training data distribution. Specifi- cally. D) static attributes such material properties environ- mental conditions influence the initialization of system states but should affect the dynamic evolution governed by the ODE function. II) Context-specifie coupling during training often leads biased interaction pattems, which hinders the ability of the model to generalize across different distributions and conditions.
coupled dynamical systems (Bahsoun & Liverani, 2024; Börner et al., 2024; Sun Yu, 2017), in which multiple interconnected components interact to produce complex collective behaviors. Such systems ubiquitous across various fields, including physics, biology, and engineer- ing (Gambuzza et al., 2020; Wu et al., 2024b). For example, in electrical engineering, coupled oscillators are employed to model circuits where components like inductors and ca- pacitors interact, giving rise to phenomena such as synchro- nization and resonance (Galias, 2019). Accurately modeling these interactions crucial for understanding and predict- ing the behavior of complex systems. Traditional numerical methods often necessitate substantial -emaii--pecific ex- pertise and computational resources, which can limit their scalability to more intricate systems (de Arruda & Moreno, 2024). As a result, data-driven approaches have emerged as effective alternatives for capturing the dynamics of these
1. Introduction
Dynamical systems describe the evolution of states over time and are fundamental to wide range of scientific and engineering disciplines (Kipf et al., 2018; Huang et al., 2021; Khan, 2013). prominent subset of these systems is 'University of Califomia, Los Angeles "Stanford University. Correspondence to: Guancheng Wan <gewan03@ucla.c
Proceedings of the 42nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Rethink GraphODE Generalization within Coupled Dynamical System
systems (Rubanova al., 2019; Greydanus et al., 2019).
training distribution, potentially compromising the model's s performance in novel scenarios under different initial conditions (e.g., velocity or location). This raises another question: II) How can we design a processor that learns universal coupling dynamics without bias?
Among these approaches, graphs provide an effective means to model the intricate interactions by representing entities as nodes and their interactions as edges. Graph Neural Net- works (GNNs) (Kipf& & Welling, 2017: Veličković et al., 2017) leverage this structure to learn complex dependencies among components, capturing both individual behaviors and their interrelations. To further model the continuous evolution of such systems, the Graph Ordinary Differen- tial Equation (GraphODE) has been introduced (Luo et al., 2023; Qin et al., 2024), integrating GNNs with differential equation solvers to seamlessly capture temporal dynamics. However, these models often rely on partially observed tra- jectories during training, which can introduce bias toward the training data distribution (Huang et al., 2024a; 2023b). Consequently, when deployed in unseen environments, these models may struggle to capture universal physical laws, leading to reduced generalization ability.
To simultaneously address the challenges mentioned above, in this paper, we propose Generalizable GRaphODE with dis Entggntagll And regulariza (GREAT) framework. Through the lens of Structural Causal Models (SCM), as de- tailed in Sec. 2.2, we identify two potential backdoor paths that hinder the generalization abilities of GraphODE, corre- sponding to the issues outlined earlier. Therefore, based on the SCM, to answer question 1), we propose the Dynamic- Static Equilibrium Decoupler (DyStaFD). At the encoder level, this mechanism disentangles static and dynamic states through two distinct subspace projections constrained by orthogonality. This separation ensures that static attributes, such as material properties or ambient conditions, do not in- terfere with the dynamic evolution process. Furthermore, we leverage the self-exciting nature of the Hawkes process to augment dynamic states, thereby enhancing temporal depen- dencies and preparing robust initializations for dynamic evo- lution. To address II), we introduce Causal Mediation for Coupled Dynamics (CMCD), which mitigates spurious cor- relations between the coupling factor and contextual states. By utilizing variational inference, we estimate unobserved factors that influence coupling dynamics. This estimation enables us to regularize causal relationships within the state evolution, ensuring that the learned dynamics are not bi- ased by specific training data distributions. Our principal contributions are summarized as follows:
Building on this discussion, the development of a General- izable GraphODE, capable of modeling complex system dynamics across diverse environments, becomes critical. Existing GraphODE architectures generally consist of three components: an encoder, processor, and decoder (Huang et al., 2020). The encoder initializes the system's 's state, which is subsequently processed to model its temporal evo- lution. However, previous work neglects that this initial state often encapsulates various factors, including certain static attributes that are unrelated to the dynamic evolu- tion governed by the ODE function. For example, in a multi-pendulum system, attributes such material com- position or local temperature represent "static states" with minimal influence on the pendulum's 's dynamic motion. Sup- pose the differential process inadvertently incorporates the effects of these static factors into the dynamic evolution. In that case, it can negatively impact the model's generalization capabilities when applied to new environments or varying static properties. This observation raises critical question: 1) How can we disentangle static attributes from dynamic states in the initialization process?
Problem Identification. We are the first to apply Struc- tural Causal Models to systematically investigate Gener- alizable GraphODE. We identify two potential backdoor paths that hinder generalization and propose correspond- ing design principles to address these challenges. Practical Solution. We introduce the Dynamic- Static Equilibrium Decoupler, which disentangles static at- tributes from dynamic states through orthogonal subspace projections. Additionally, propose Causal Mediation, which uses variational inference to estimate latent causal factors, mitigating spurious correlations and enhancing the universal coupling dynamics.
Prior research has predominantly focused on refining the encoder to improve the initialization of system states. How- ever, it has largely overlooked the crucial role of the pro- cessor, which is responsible for learning the GraphODE function that governs the system's 's evolution. In the context of coupled interactions, we introduce the coupling factor, a control variable that influences the interaction between two entities. For example, in a multi-pendulum system, this factor quantifies the strength of the coupling force ex- erted by a spring on each pendulum. During the training phase, however, this coupling factor often captures context- specific interaction patterns present in the training data. This leads to a dynamic process that is overly dependent on the
Experimental Validation. We conduct comprehensive experiments on multiple benchmarks involving diverse coupled dynamical systems, demonstrating that GREAT significantly outperforms of-h--oft methods in both in-distribution and out-of-distribution settings.
2. Motivation
2.1. Preliminaries
Notations. A coupled dynamical system is represented as a temporal graph g = &V,E, X)), where V is the set of N
2

---

Rethink GraphODE Generalization within Coupled Dynamical System
entities (nodes), & S V x V denotes the edges deseribing interactions between entities, and X' is the feature matrix that records the entity states over time, Each entity i at timestamp t has a feature vector (t) E IR where d is the feature dimension. The adjacency matrix A E R en- codes the pairwise relationships, where Auv = 1 an edge Euv E & exists at time i, otherwise Aun = 0. The diagonal degree matrix D is defined as D vu Σ. Aur. and the nor- malized adjacency matrix is given by = D-I/A 1/2. The system's s dynamies can further described using the Laplacian matrix L D A or its symmetric normalized form L = I A, where I is the identity matrix.
(to
h(tt)
Generate
Spurtene Reiatiombll
aigte ot
Spartons
17(t)
Reisttenshig
Alt L
Effect
(T
(T
M
a: Eatity Feature
h(t) Lateat State
Static State
MV1 ODE Function
k(ta): Initial Dynamic
7 Coupling Factor
x
Neural ODEs for Dynamical Systems. The continuous evolution of entity states in a coupled dynamical system can be modeled using Neural Ordinary Differential Equations (Neural ODEs) (Chen et al., 2018: Alvarez et al., 2020). Each entity i E V has a latent state ht) E RA, where h represents the latent dimension. The state evolution over time is governed by the following ODE:
h(t)
S
Me
(T)
(t) go (f), (t) : A),
Figure 2. Structural Causal Model (SCM) for Generalizable GraphODE within Coupled Systems.
dt
(to) = fen: (X(f-Mr f-1).9),
where go is learnable neural network parameterized by 0, which models the dynamics of entity based on its own la- tent state (t) and the aggregated influence of its neighbors (i) using the graph structure A. The encoder fene ini- tializes the latent states (4)4 by mapping historical states (t) and graph information into the latent space. The latent state (t) at any future time t obtained by integrating the ODE with respect time I, typically solved using numerical solvers such Euler's 's method or Runge- Kutta methods:
spring a canonical example of coupled dynamics. Their motion can be described by: mä = -mg k(x v), mÿ = m -xx-r), where the spring constant embodies the coupling mechanism. Similar principles gov- ern interactions in molecular systems (harmonic oscillators) and electrical circuits (coupled RLC networks), demonstrat- ing the universality of coupling phenomena across scales, Effectively modeling these systems demands frameworks that reconcile physics-based interactions with nonlinear het- erogeneous dynamics in complex environments, motivating our generalizable GraphODE:
(t) = (t()) + [" (h, 1 e (N))), dr.
(2)
Definition 2.2 (Generalizable GraphODE for Coupled Sys- tems). To describe the dynamics of coupled systems, we define a Generalizable GraphODE as:
The predicted entity states (t, are reconstructed from the latent states (t, through a decoder face: (1) = faee (t)( The objective is minimize the difference between the predicted trajectories (t) and the ground truth (t) over the prediction horizon [fo, T]. The loss func- tion is expressed as:
Ola(l) 0t
Σ (t) (f), O JEN()
(4)
where h₂(t) R represents the latent state of entity at time and (t)(t) is the coupling factor quantifying the interaction strength between entities i and j. The interac- tion function (h,(1)) ha(l); m models nonlinear, lime- varying relationships parameterized by learnable param- eters Oode- This framework unifies interpretable, physies- based principles with flexible data-driven learning to ro- bustly capture complex coupled interactions.
Cpredict
-
- (((t
(3)
where 4y represents the timestamp.
2.2. A Causal View on Generalizable GraphODE Definition 2.1 (Coupled Dynamical System). Coupled dy- namical system refers system where the state evolution of one entity is inherently influenced by its interactions with others, often leading to energy transfer or synchronization. The interactions are governed by physics-base coupling mechanisms that determine energyfinformation exchange.
To design Generalizable GraphODE that effectively captures graph rationalization and facilitates generalization, we analyze its underlying mechanism using a Structural Causal Model (SCM) (Pearl et al., 2000; Pearl, 2016). The SCM framework enables us to systematically understand the causal relationships within coupled dynamical systems.
For instance consider two pendulums connected by
3

---

Rethink GraphODE Generalization within Coupled Dynamical System
Below, we detail the key variables and their interactions (subscript omitted for brevity):
menting this, the Causal Mediation for Coupled Dynamies addresses context-specific coupling biases through varia- tional estimation of latent causal mediators, enabling the GraphODE to learn interaction dynamics governed by uni- versal physical laws rather than observational artifacts. The illustration of the overall framework is detailed in Figure 3.
s - x + h((o): Initial states x consist of two disjoint components: the static variable s, which encapsulates time-invariant properties (e.g., material properties, envi- ronmental conditions), and the dynamic variable h(to), representing the system's 's initial dynamic state.
h(t) oh(t)/0r, htt + 1): The state evolution process de- scribes how the dynamic variable h(()) evolves over time, governed by a differential equation that captures temporal dependencies and interactions between entities.
3.2. Dynamic-Static Equilibrium Decoupler
Dynamic-Static Bilinear Orthogonal Projections. Given the historical observation input x x;(t-M : f-1). we first encode it into a latent state 2; = (xx;) ) E RB where the encoder fe implemented as two-layer multilayer perceptron (MLP). The latent state 2; captures both the temporal evolution (dynamic) and invariant contextual prop- erties (static) of the input. To disentangle these components, we decompose 2; into two orthogonal parts:
From the SCM, two backdoor paths are identified that intro- duce spurious correlations, hindering generalization:
((4) + r + + A(1) + h(t + 1) Static states s act as confounder between the initial state h(fo) and the evolv- ing state h(t). Static variables such as material properties should only influence initialization but not directly affect dynamic evolution. If improperly incorporated into the dif- ferential process, these variables can reduce generalization ability in scenarios with different static conditions.
(5) This decomposition separates 2; into a dynamic compo- nent (o;) and a static component (s;), corresponding to the system's 's time-varying and invariant properties, respec- tively. To ensure trainability and adaptability, we introduce two learnable subspaces: the static subspace Static and the dy dynamic subspace Sdynamic. parameterized as:
o, + 8i, 1
h(1) - n((1) + h(t + 1)- The coupling factor vj((), de- rived from h(t), acts as a confounder between the current state h(tt) and the next state h(t + 1). If ntt) captures domain-specific contextual patterns (e.g., particular cou- pling strengths in training data), it can lead to biased dy- namics. Based on the discussion, we present the following principle for designing Generalizable GraphODE:
Suatie = EMBEDDING Gaturic (p. P),
(6)
Saynamic EMBEDDING (p.P).
where S
(Sstate, Satatic Satatic.p) and
S Sdynamic = {Stymamicc, 1. Stynamic, ..... Sdymamic.p) } Here, p denotes the number of basis vectors in each subspace. with Sstatica E R xp and Sdymamic.6 IR Xx representing the a-th and b-th basis vectors of the static and dynamic subspaces, respectively. These subspaces are optimized during training to learn disentangled representations of zi. We project 2 onto S static and Sdynamic to obtain 8; and o;:
Generalizable GraphODE Design Principle: Disen- tanglement: Effectively separate static variables from dynamic states, ensuring static properties influence only initialization without confounding dynamic evo- lution. Regularization: Address spurious correlations in the coupling relationship, ensuring context-invariant interactions aligned with universal physical rules. Per- formance: Achieve robust generalization both within the training distribution and out-of-distributian
Saatica
2,
Satatic,o
8
|| a ll
|Ssuatic ||
(7)
Sdynamic, b 2TT
Sdysamic,b
|Soymamicc alla |sdynamic.bll
Here, 8; E TR. x and o, E RRM Mxd denote the projections of 2; onto the static and dynamic subspaces, respectively. The operator || denotes concatenation, and ||- is the Euclidean norm. While s; and o, are computed separately, orthogonal- ity between the subspaces is not inherently guaranteed. To enforce this, we introduce an orthogonality loss La:
In the following sections, we will elaborate on how GREAT adheres to these principles, effectively severing spurious correlations through disentanglement and regularization.
3. Methodology
Sptatic,
Sdynamic &
L.
abs
(8)
3.1. Framework Overview
[|sttaicc
where abs(-) denotes the absolute value. Minimizing Co re- duces the cosine similarity between the subspaces, ensuring mutual orthogonality. This prevents interference between static and dynamic properties during prediction, enhancing both robustness and interpretability.
The proposed GREAT framework achieves generalization in coupled dynamical systems through two core innova- tions: The Dynamic-Statie Equilibrium Decoupler elimi- nates static-dynamic entanglement during state initialization via orthogonal subspace projections, decomposing system states into independent static attributes (e.g.. material prop- erties) and dynamic patterns (e.g., velocity fields). Comple-
Dynamic Hawkes Process Augmentation. To further en-
4

---

Rethink GraphODE Generalization within Coupled Dynamical System
Static Safregunn
Meultoa M
Dynamic-Slaid Bilinear Orthogonal Projections
Dynamic Hawkes Process Augmentation
P lt
I
19 R
(e +)
0,00) aitt) +
May (((
Dynaniic subepaac
I
C
9000
Spatial Temporal GNN Encoder
(e((t)) = dbd,s Ome)
Causal Modiation for Coupled Dynamics
Lxz
n
B0
Edtimatur
metiiaao ali-)
Static
En weetC2211
Latent
State
t
Pinal
Coupling Brolution
Dynamic
Project
wi"hao w," w/ihoo
Latent
Synergy Decoder
Predict predict
State
Figure 3. Architecture illustration of Generalizable GraphODE with disentanglement and regularization. Best viewed in color.
hance the disentangled dynamic representation 0,, pro- pose the Dynamic Hawkes Process Augmentation (DHPA), which leverages the secftixxiting nature of the Hawkes pro- cess (Okawa et al.; Han et al.; I.nn et al., 2021) to capture temporal dependencies in complex dynamical systems bet- ter. the context of dynamic systems, states at a given time are influenced by prior states, often in cascading manner. For example, forces propagate dynamically in physical systems, creating continuous inter- actions across time. While the disentangled representation o captures immediate temporal patterns, it does not fully account for the compounding influence of past states. DHPA addresses this limitation by explicitly modeling these tem- poral interactions, enriching the dynamic representation and enabling it to capture both short- and long-term dependen- cies inherent in physical systems. Formally, the augmented representation at time f, (t) E Rd:
representation with richer temporal dependencies:
à = {o,(-M), a-M + +).....,.." (-1)-
(11)
This augmentation offers several advantages: 1 It explicitly captures cascading effects and long-ten term dependencies in dynamic observations, which are essential for understand- ing physical systems governed by differential equations. @ By integrating temporal interactions, it smooths noisy fluctuations and enhances robustness, ensuring that the rep- resentation remains stable across varying system conditions.
are rarely independent-
The learned influence weights Wr,t provide interpretable insights into the temporal structure of the system, revealing how past states influence future behavior.
Dynamic-Static State Initialization. To initialize the dy- namic state for GraphODE and effectively learn spatial- temporal relationships among agents, we follow the method- ology in (Huang et al., 2021: 2024b; Luo et al., 2024). We adopt spatio-temporal GNN (()), parameterized by + to compress historical trajectories into latent representations: htt) = :(0;; enne). (For details about the spatio-temporal GNN <(-), see Appendix D.) For the static state, we directly apply a global pooling mechanism to summarize station- ary characteristics, capturing unchanging information over time in sj: 8; M EM st()). After obtaining the output (T) at time T through dynamic evolution (as described in Sec. 3.3), we propose synergy decoder to seamlessly combine these two components in the final output stage: (12)
a.()) = (1) + Σ W (()),
where is the historical window size determining the tempo- ral range of influence, and 8 is a learnable parameter control- ling the augmentation strength. The influence weight Wr,t. which quantifies the contribution of the past state ot -))
to the current state (t). is defined as:
((-))+1
(10) ensuring exponentially decaying influence from more distant past states, where more recent states exert stronger influence. By summing over this weighted contribution of past states, DHPA explicitly integrates the self-exciting and decaying temporal dynamics into o; augmenting the
W7.e exp
-t ( (t-
(:(T) = fase (h: (T); Qdec).
3.3. Causal Mediation for Coupled Dynamies
Analysis on State Evolution. After obtaining the separated dynamic state, we analyze how to design the generalized
5

---

Rethink GraphODE Generalization within Coupled Dynamical System
GraphODE and sever spurious relationships @. The distri- bution process be defined as p(h,(t++1) (t), ((t), 9), governed by the differential process in Equation (4). Con- sidering the impact of coupling factors, the generative form can be expressed as:
By incorporating context-independent prior p(%(()), (!), the scaling term prevents the model from overfitting context- specific correlations introduced by q(n (t)), enhanc- ing its ability to handle distributional shifts. Furthermore, the scaling term reduces the influence of frequently occur- ring coupling patterns and increases the importance of rarer ones, promoting the model's focus on invariant dynamics that hold across different environments. By approximating ((y,(t) do h2 (t)))) 9) through the scaling term, the model aligns with invariant principles governing coupled dynam- ics, such as conservation laws. This enables the ODE solver to learn stable dynamics from h:; (t) to ht 1), facilitating robust predictions and ensuring unbiased dynamics.
E (n.tooa..t)) [p(ha(t+ + ()) 9)).
(13)
While the generative distribution (t+,,tt))) (t)
provides a principled framework for modeling state transi- tions, (t) can act as a confounding variable, introducing biases. Specifically, the distribution p(n (t)((()))) serves as a prior for coupling dynamics. However, relying on it biases the model toward the training data distribution, entangling the mapping from (t) to (t + 1) with (t) and creat- ing spurious correlations, as illustrated in Figure 2. These context- -ontext-specific influences limit the model's 's ability to gen- eralize to unseen scenarios, as coupled systems generally adhere to fundamental physical laws, such as the conserva- tion of energy, which transcend specific coupling factors.
Factor Estimator. To tackle the confounding effect intro- duced by the coupling factor (t),), we introduce a mech- anism that explicitly estimates my (t) based on the disen- tangled dynamic state (x,(). We define (1) E R as a K -i-dimnnioona representation of the inferred coupling fac- tor for node i at time t. Instead of directly assigning 7) (t), it is modeled as a probabilistic variable conditioned on the node's dynamic state (t).). Formally, we parameterize the distribution of (t) as (t) Softmax (W,,hy ,,,,(()), where (1) E R represents the probability distribution over K possible coupling patterns, W,, R is learnable parameter matrix, and d is the dimension of (t).). The coupling factor v7:()) is then sampled using the Gumbel- Softmax approach (Jang et al., 2016; Shah et al., 2024) to ensure diffcrentiability during backpropagation:
Deconfounding Coupled Evolution. To address the con- founding influence of (t). we propose a causal regulariza- tion method. Inspired by deconfounded learning (Wu et al., 2024a), we aim to compute log ply, (T)) (o())(t))), 9). The causal intervention do(h;(to)) eliminates the spurious cor- relation between the coupling factor 1;(() and the state transition dynamics, enabling the model to focus on the invariant relationship between (t) and (t that holds across domains. However, due to the unobservable nature of y) (L), we cannot directly compute this quantity. Instead, we leverage variational distribution q(n. (t) (t)), allow- ing the model to approximate the interventional likelihood log p(y: (T) (1o))) 9) in a tractable manner:
exp ((Cs (1) + 9x))
m1,x = Σ exp ((G,s(t) + 9s)/r)' Oa Gumbel 1),
(15) where Ik is noise sampled from the Gumbel distribution, and T is the temperature parameter controlling the smoothness
Theorem 3.1. Consider the state evolution process + (1), (((),)) with the coupling factor ni(t)
Coupling-Aware State Evolution in GraphODE. To bet- ter model complex interactions in dynamical systems, we decompose the dynamic state (t) into sub-states hi, (t), each representing a specific coupling pathway ni,x(t), This allows the model to capture distinct periodic and oscillatory behaviors, such as synchronization and resonance. The state is represented using a trigonometric basis, where h (t) is expressed as sine and cosine transformations:
as a latent confounder. The regularization likelihood log (ddooh)dohha)))) 9) can be bounded as:
log (d))h)))tt,,(tt))) 9) 2 (erl((()) sg +
B (1))
log Σ
p(ha(t + 1)]ha(t), 9: @mw)
Σ KL (L((()())) p()()))))
h1.2m-1 (1) = sin(m--(.()). 11.2m((() = cos(m- (16) Each sub-state ht, interacts with its respective coupling factor ((), allowing independent modulation of state evolution for each mode. The inferred coupling factor ny (t()) influences state transitions, dynamically adjusting sub- state evolution. Unlike traditional static coupling rules, our method adapts each sub-state based on its associated cou- pling factor. The overall state evolution modeled as: ahu(l) Ot
(14)
where (t)) is a context-agnostic prior over the coupling factor, and ((n((t)),,(t) is the variational approximation of the posterior for (1).
The proof of this bound is provided in Appendix B. Here, (t)) is typically assumed to be uniform prior, (t) = 1/K, to ensure that ()()) remains agnostic to domain-specific contexts. The causal mediation objective
K
Σ (t)
Σ JEN(
w ky,(() + w (t)
introduces a scaling term (0](,,)) which adjusts the in- fluence of the coupling factor 7)s (t) on the learning process.
p((((t))
(17)
6

---

Rethink GraphODE Generalization within Coupled Dynamical System
SPRING
CHARGED
PENDULUM
Methods
In-Distribution
Ont-af-Distribttinn
In-Distribution
unn-af-Diitribution
Ir- In-Diisribution
Ont-ef-Distriuation
RMSE
MAPE
RMSE
MAPE
RMSE
MΑPE
RMSE
MAPE
RMSE
MAPE
RMSE
VAR
9.881 p3.210
24.55-18.00
16.11-6.220
25 18..18
11
30.20-18.00
184458.220
27.88
36.72 20
42.31 5.350
12.77
65.02,10.
LSTM
r..92
20.31
10.22
15.09
7.432yo.um
22.63-s..12
24.66
31.45
30.64
40.2221.250
37.08
60.74 820
LDeeODE
6.671
6.544
7,381
7.542
7.444
16.75
10.23
23.77
17.46
38,96
34.22
54,92
HODEN
5.029 11.
8.327
p2.49
5.892
p1.50
11.23
55.520
5.107 15.323
13.67
p10 10
27.68 22
p1.50
y9,45
60,22
TRS-ODEN
5.256
4,801
11,
7.120 0o 0.201
8.620 1.078
4.601
12
10:45
.0.308
6.452 (3.78)
18.22
16 000
38.92
46.89
7,990
48.74
67.33 41
LG-ODB
19.004
2,918
630
4.346.a
6.112
2,883
6,430
10
3.145,7.083
10.09
08
11,43 030
23,90,15.00
(4.87
33.70,a1.00
SoclalODE
4.452
3,520 ys. 09.
4.902.
5.221
(2.921
2,756 14.088
6.788_0
.9.922
3.001
8.452
(1.
13.72
26.33
24.04, 11018
37.72
PGODE
3.901 2,562
4.021
5.772
2.561
6,503
210.24
2,901
7.325
9.192
10.92
22.71
20.19
32,90
TREAT
(9,777
2,748
jo 706
4.213 p. 168
5,234
12.00e
2.910
664
7,234
222667.000
10:42
110 96
11.96
500
28.91
(14.05
22.34_
12:05
34.21 :00
GREAT
3.687 (2.984
2.348
196
3.619
a. 762
4.521
p3.003
2.337
15.107
6.042
10.71
2,690
77.540
8.322
11
.5
10.34
120
23:44
15.52
18.24 15.00
30.44 p4
Table 1. Comparison with the counterparts on three datasets with RMSE and MAPE (10-1). Best in bold and second with underline.
fruhh
GREAT
POODE
L0-20E
Figure 4. Visualization of predicted trajectories (Semi-transpp transparent) given observation trajectories (Solid) in the SPRING OOD scenario.
where M(i denotes node i's neighbors, and ((()) w-(+) are trainable weight matrices for the k-th coupling pathway. This trigonometric decomposition ensures that GraphODE can effectively capture periodic and interaction-specific dy- namics in a flexible and interpretable way.
4.1. Experimental Setup
Coupled Dynamical Systems Datasets. We evaluate GREAT using three coupled dynamical systems datasets: SPRING, CHARGED, and PENDULUM, which model the dynamics of physical systems with complex interdepen- dencies. For evaluation, we adopt two metrics: RMSE (Root Mean Square Error) and MAPE (Mean Absolute Percentage Error). We assess GREAT under both in-distribution (ID) and out-of-distribution (OOD) settings, where the OOD setting modifies the test dataset's initial conditions (e.g.. velocity, position) to test the model's ability to generalize to unseen scenarios. Further details are provided in Ap- pendix A. The code is available at https:/ https:///ithha /github. com/ Sonn/uancchngWaan GREAT.
3.4. Training Objective
The final training objective integrates prediction accuracy, disentanglement, and regularization into a unified loss:
L = Lpredict LKI. + Aoho,
(18)
where Sprediet ensures accurate predictions, Co enforces or- thogonality between static and dynamic subspaces, and CKL regularizes the coupling factor (t) estimation. Epredic and CKL together form the core of the evidence lower bound (ELBO), where Epredict corresponds to the reconstruction likelihood, and CKL imposes a regularization. Specifically, LKL, derived from Equation (14), can be calculated as:
Counterparts. We compare ours against traditional ap- proaches and several SOTA Neural ODE methods including VAR (Song et al., 2020), LSTM (Sesti et al., 2021), Laten- tODE (Rubanova et al., 2019), HODEN (Greydanus et al., 2019), TRS-ODEN (Huh et al., 2020), LG-ODE (Huang et al., 2020), SocialODE (Wen et al., 2022), PGODE (Luo et al., 2024). TREAT (Huang et al., 2024b).
N ΣΣ (L(((T((() | hc()) Il (()),))) (19)
CXL N
this ensures that (t) is disentangled from spurious correla- tions and aligns with context-invariant principles.
4.2. Superiority (Q1)
4. Experiment
To assess the performance of GREAT, we evaluate it in both ID and OOD settings, as shown in Tab. 1. Several obser- vations can be made (Obs.): Obs. 1 Traditional methods, such as LSTM and VAR, often perform worse than Neural ODE-based deep learning methods, highlighting the advan-
We comprchensively evaluate GREAT through four axes: Q1 (Superiority). Q2 (Resilience), Q3 (Effectiveness), and Q4 (Sensitivity). Q1-Q3 illustrated in Sec. 4.2-Sec. 4.4, and sensitivity analyses (Q4) can be found in the Appendix F.
7

---

Rethink GraphODE Generalization within Coupled Dynamical System
4.5 RASE
5. RMBE
4.
4.
43
47
4.
444
1400
242
1.
4.2
4.1
0.903
9992
2.00
0.
3.
3.8
0,00
2200
3.5
3/5
10
41
Esh Strek
(a) In- Distiiiiibutin
(b) Oft-of-Disse Distribbuinnn
Sour
Figure 5. Analysis on proposed components for both ID and OOD settings SPRING. Please see details in Sec. 4.4.
(a) In- In-Distribution Figure 7. Analysis on Substate K and dimension p for both ID and OOD settings on SPRING. Please see details Appendix
(b) Oft- Distribbtiin
7.0
7.5
TREAT
TREAT
0.5
6.0
POODE
SoCIa
66.0
6.5
SociaIDDE
POODE
fectively generalize and sustain performance, even under challenging conditions with varying prediction lengths.
255
5.0
4.5
4.0
4.0
4.4. Effectiveness (Q3)
20
Prediction Length
Prediction Length
We evaluate the effectiveness of two key components in GREAT:
(a) In-DDistribttinn
(b) Out-of-Distribution
DyStaED and CMCD. As shown in Figure 5, both components significantly improve model performance. GREAT enhances the disentangling of dynamic and static components, addressing spurious dependencies and en- abling better generalization under new conditions. DHPA further improves this disentanglement, encouraging the char- acteristics of dynamic representation and separation. Mean- while, CMCD contributes modeling coupling dynamics, allowing the model to capture system's 'ss robust interdepen- dencies. When both DyStaED and CMCD are combined, performance reaches its peak, as evidenced by the lowest RMSE across both ID and OOD settings. Notably, CMCD's impact is more pronounced in OOD scenarios (Figure 5 (b)). This indicates that learning universal coupled dynamics crucial for improving generalization unseen scenarios.
Figure 6. Analysis on prediction length for both ID and OOD settings on SPRING. Please see details in Sec. 4.3.
tages of incorporating dynamic system modeling for com- plex tasks. Obs. @ GREAT consistently achieves the best performance across both RMSE and MAPE metrics, outper- forming all baselines in both ID and OOD settings. This demonstrates its robustness and adaptability. Specifically, GREAT achieves an RMSE of 3.687 and a MAPE of 2.348 on the SPRING dataset in the ID setting, outperforming all other methods, including traditional approaches, well as -tate-of-the-art Neural ODE models such as LatentODE and TREAT. Obs. ? In OOD scenarios, GREAT exhibits even greater performance improvements compared to the ID setting, demonstrating that the correct decoupling and regu- larization techniques help stabilize the model and enhance its generalization ability.
5. Conclusion
Furthermore, visualized the model's predicted trajecto- ries (semm-transparent) alongside the given observation tra- jectories (solid) in the SPRING OOD scenario. As shown in Figure 4, GREAT more accurately fits the ground truth com- pared to other methods. The solid ground truth trajectory exhibits smooth, consistent behavior, with GREAT closely following the true dynamics.
In this paper, we propose a novel framework, GREAT, to en- hance the generalization capabilities of GraphODE models for coupled dynamical systems. Leveraging insights from Structural Causal Models, we identify two key challenges: the entanglement of static attributes with dynamic states during initialization and the reliance on context-specific coupling patterns that hinder performance in unseen sce- narios. To address these issues, we introduce the Dynamic- Static Equilibrium Decoupler (DyStaED), which disentan- gles static and dynamic states through orthogonal subspace projections, ensuring robust initialization. Additionally, we propose the Causal Mediation for Coupled Dynamics (CMCD), which uses variational inference to estimate latent causal factors, mitigating spurious correlations and promot- ing universal coupling dynamics. Extensive experiments on diverse coupled dynamical systems demonstrate that GREAT outperforms state-of-the methods in both in in-distribution and outtt--diitributiin settings. Our GREAT provides a ro-
4.3. Resilience (Q2)
As shown in Figure 6, GREAT consistently achieves the best performance across various prediction lengths, demonstrat- ing its resilience as the prediction horizon extends. In both ID and OOD scenarios, GREAT maintains a lower RMSE compared to other methods, underscoring its robustness in handling long-term predictions. Particularly in the OOD setting (as shown in Figure 6 (b)), GREAT's performance re- mains stable and competitive, outperforming methods such as TREAT and PGODE. This indicates that GREAT can ef-
8

---

Rethink GraphODE Generalization within Coupled Dynamical System
bust solution for modeling complex systems and highlights the importance of disentanglement and regularization for generalization across dynamic environments.
Chen, R. T., Rubanova, Y., Bettencourt, J., and Duvenaud,
D. K. Neural ordinary differential equations. NeurIPS, 31, 2018.
de Arruda, G. F. and Moreno, On the temporal resolu- tion limits of numerical simulations in complex systems. arXiv preprint arXiv:2403.0668 2024.
Acknowledgement
This work was partially supported by NSF 2200274, NSF 2106859, NSF 2312501, NSF 22115 NSF 2119643, NSF 2303037, DARPA HR001 12490370, NIH U54HG012517, NIH U24DK097771, NIH U540D036472, SRC JUMP 2.0 Center, Amazon Research Awards, Snapchat Gifts, NEC,
Fu, X., Zhang, B., Dong. Y. Chen, C., and Li, J. Federated graph machine learning: survey of con- cepts, techniques, and applications. arXiv preprint arXiv:2207. 0tn:22277.111122 11812, 2022.
and Optum.
Galias, Z. Nonlinear dynamics of coupled inductor- capacitor-memriisor oscillators. In 2019 26th IEEE Inter- national Conference on Electronics, Circuits and Systems (ICECS), pp. 354-557. IEEE, 2019.
Impact Statement
This paper presents work whose goal advance the field of Machine Learning. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here.
Gambuzza, L., Di Patti, F., Gallo, L., Lepri, S., Romance, M., Criado, R., Frasca, M., Latora, V., and Boccaletti, S. The master stability function for synchronization in simplicial complexes. arXiv preprint arXiv:2004.039113, 2020.
References
Alvarez, V. M. M., Rosca, R., and Fälcujescu, C. G. Dyn- ode: Neural ordinary differential equations for dynam- ics modeling in continuous control. arXiv preprint arXiv:2009.04277 042778,, 2020.
Greydanus. S., Dzamba, M., and Yosinski, J. Hamiltonian neural networks. Advances in neural information process- ing systems, 32, 2019.
Hamilton, W., Ying, Z., and Leskovec, J. Inductive repre- sentation learning large graphs. In NeurlPS, 2017.
Auzina, I. A., Y1 Idiz, C. g., Magliacane, S., Bethge, M., and Gavves, E. Modulated neural odes.
Han, R., Li, H., and Xie, B. Dhprep: Deep hawkes process based dynamic network representation.
Bahsoun, W. and Liverani, C. Mean field coupled dynam- ical systems: Bifurcations and phase transitions. arXiv preprint arXiv:2402. / 1076, 2024.
He, D., Liang, C., Liu, H., Wen, M., Jiao, P., and Feng, Z. Block modeling-guidern graph convolutional neural networks. In AAAI, volume 36, pp. 4022-4029, 2022a.
Börner, G., Schröder, M., Thümler, M., and Timme, M. Perturbation-response dynamics of coupled nonlinear sys- tems. Chaos: An Interdisciplinary Journal of Nonlinear Science, 34(10), 2024.
He, Q. Chen, J., Xu, H., and He, K. Structural robust label propagation on homogeneous graphs. In Proceedings of the IEEE International Conference on Data Mining, pp- 181-190. IEEE, 2022b.
Cai, X., Huang, C., Xia, L., and Ren, X. Lightgel: Simple yet effective graph contrastive learning for recommenda- tion. arXiv preprini arXiv:2302.0819) 2023.
Huang, W.. Wan, G., Ye, M., and Du, B. Federated graph semantic and structural learning. 2023a.
Chapfuwa, P., Rose, S., Carin, L., Meeds, E., and Henao, R. Capturing actionable dynamics with structured latent ordinary differential equations.
Huang, Z., Sun. Y., and Wang, W. Learning continuous system dynamics from irreuularly-sanpled partial ob- servations. Advances in Neural Information Processing Systems, 33:16177-16181 2020.
Chen, J.. Gao, K., Li, G., and He. K. Nagphormer: A tokenized graph transformer for node classification in large graphs. In Proceedings of the Eleventh International Conference on Learning Representations, 2023.
Huang, Z., Sun, Y., and Wang, W. Coupled graph ode for learning interacting system dynamics. In Proceedings of the 27th ACM SIGKDD conference on knowledge discov- ery & data mining, pp. 705-715, 2021.
Chen, J., Liu, H., Hopcroft, J. F.,, and He, K. Levergging contrastive learning for enhanced node representations in tokenized graph transformers. In Proceedings of the 38th Annual Conference on Neural Information Processing Systems, 2024.
Huang, Z., Sun, Y., and Wang, W. Generalizing graph ode for learning complex system dynamics across envi- ronments. In Proceedings of the 29th ACM SIGKDD

---

Rethink GraphODE Generalization within Coupled Dynamical System
Conference on Knowledge Discovery and Data Mining, pp. 798 809, 2023b.
Mei, W., Zheng, D., and Li, S. Controlsynth neural odes: Modeling dynamical systems with guaranteed conver- gence.
Huang, Z., Hwang, J., Zhang, J., Baik, J., Zhang, W., Wodarz, D., Sun, Y.. Gu, Q., and Wang, W. Causal graph ode: Continuous treatment effect modeling in multi-agent dynamical systems. In Proceedings of the ACM on Web Conference 2024, pp. 4607-4617, 2024a.
Okawa, M., Iwata, T., Tanaka, Y., Toda, H., Kurashima, T., and Kashima, H. Dynamic hawkes processes for discovering time-evolving communities' states behind diffusion processes. doi: 10.48555/ARX1V.2100.
Huang, Z., Zhao, W., Gao, J., Hu, Z. Luo, X., Cao, Y., Chen, Y., Sun, Y., and Wang, W. Physics-informed regulariza- tion for nomain-agnosttr dynamical system modeling. arXiv preprint urXiv:2410.06366 2024b.
Pearl, J. Causal inference in statistics: a primer. John Wiley & Sons, 2016.
Pearl, J. et al. Models, reasoning and inference. Cambridge, UK: CunbrrdgeUniiveeiiyyPeesss 19(2):3, 2000.
Huh, I., Yang, E., Hwang, S. J., and Shin, J. Time-reversal symmetric ode network. Advances in Neural Information Processing Systems, 33:19016-1990 2020.
Poli, M., Massaroli, S., Park, J., Yamashita, A., Asama, H., and Park, J. Graph neural ordinary differential equations. arXiv preprint arXiv: 191::9111100500 1.07532, 2019.
Jang, E., Gu, S., and Poole, B. Categorical repa- rameterization with gumbel-softmax. arXiv preprini arXiv:1611.01144 2016.
Qin, Y., Ju, W., Wu, H., Luo, X., and Zhang, M. Learn- ing Graph ODE for Continuous-Time Time Sequential Recom- mendation. IEEE Transactions Knowledge and Data Engineering, pp. 1-14, 2024.
Khan, Y. I. Applications of dynamical systems in engineer- ing. arXiv preprint arXiv: 10,,5.511,,000000
Rubanova, Y., Chen, R. T., and Duvenaud, D. K. Latent ordinary differential equations for irregulall-sampple time series. Advances in neural information processing systems, 32, 2019.
Kingma, D. and Ba, J. Adam: A method for stochastic optimization. arXiv preprint arXiv:
2014.
Kipf, T., Fetaya, E., Wang, K.-C., Welling, M., and Zemel, R. Neural relational inference for interacting systems. In International conference on machine learning, pp. 2688- 2697. PMLR, 2018.
Sesti, N., Garau-Luis, J.,., Crawley, E., and Cameron, B. In- tegrating LSTMs and GNNs for COVID-19 Forecasting, 2021.
Kipf. T. N. and Welling, M. Semi-supervised classification with graph convolutional networks. In ICLR, 2017.
Shah, R., Yan, M., Mozer, M. C., and Liu, D. Improv- ing discrete optimisation via decoupled straight-through gumbel-sofine softbax. arXiv preprint vrXiv:2410.113
2024.
Lin, X., Cao, J., Zhang, P., Zhou, C., Li, Z.. Wu, J., and Wang, B. Disentangled deep multivariate hawkes process for learning event sequences. In 2021 IEEE International Conference on Data Mining (ICDM), pp. 360-369, IEEE, 2021.
Song, C., Lin, Y., Guo, S., and Wan, H. Spatial-temporal synchronous graph convolutional networks: A new frame- work for ppatial-tempone network data forecasting. In Proceedings of the AAA/ conference on artificial intelli- gence, volume 34, pp.
2020.
Liu, Y., Jin, M., Pan, S., Zhou, C., Zheng, Y., Xia, F,, and Yu, P. Graph self-supervised learning: A survey. IEEE TKDE, 2022.
Sun, Z. and Yu, C. Bimensional-invarian principles in coupled dynamical systems. Submitted to IEEE Transac- tions on Automatic Control. Available ArXiv preprint ArXiv, 1703, 2017.
Liu, Y., Liang, K., Xia, J., Zhou, S., Yang, X., Liu, X., and Li, S. Z. Dink-net: Neural clustering on large graphs. In 2023.
Veličković, P., Cucurull, G., Casanova, A., Romero, A.. Lio, P., and Bengio, Y. Graph attention networks. arXiv 2017.
Luo, X., Yuan, J., Huang, Z., Jiang, H., Qin, Y., Ju, W., Zhang, M., and Sun, Y. HOPE: High-order Graph ODE
preprint 1eXiv:1710.100
For Modeling Interacting Dynamics. In Proceedings of the 40th International Conference on Machine Learning. pp. 23124-23139, PMLR, 2023.
Wan, G., Huang, W. and Ye, M. Federated graph learning under domain shift with generalizable prototypes. In AAAI, 2024a.
Luo, X., Gu, Y., Jiang, H., Zhou, H., Huang, J., Ju, W., Xiao, Z., Zhang, M., and Sun, Y. Pgode: Towards high-quality system dynamics modeling. In Forty-first International Conference on Machine Learning, 2024.
Wan, G., Tian, Y., Huang, W., Chawla, N. V., and Ye, M. S3gel: Spectral, swift, spatial graph contrastive learn- ing. In Forty- first International Conference on Machine Learning, 2024b.
10

---

Rethink GraphODE Generalization within Coupled Dynamical System
Wan, G., Liu, Z., Shan, X., Lau, M. S., Prakash, B. A., and Jin, W. Epidemiology-awarc neural ode with continuous disease transmission graph. In Forty-second International Conference on Machine Learning, 2025a.
Wan, G., Shi, Z., Huang, W., Zhang, G., Tao, D., and Ye, M. Energy-based backdoor defense against federated graph learning. In International Conference on Learning Representations, 2025b.
Wen, S., Wang, H., and Metaxas, D. Social ode: Multi-agent trajectory forecasting with neural ordinary differential equations. In European Conference Computer Vision, pp. 217-233. Springer, 2022.
Wu, Q.. Nie, F., Yang, C., and Yan, Learning divergence fields for shift-robust graph representations, 2024a. URL http:/ arxiv..... org/ abs / 2406. 04963.
Wu, Y., Ding, Q., Huang, W., Li, T., Yu, D., Jia, Y. Dy- namic learning synchronization coupled nonlinear systems. Nonlinear Dynamics, 12224))21945-2166, 2024b.
Xia, J., Wu, L., Wang, G., Chen, J., and Li, S. Z. Progel: Rethinking hard negative mining graph contrastive learning, arXiv preprint arXiv:2 10.00220 2021.
Xu, D., Cheng, W., Luo, D., Chen, H., and Zhang, X. In- fogel: Information-awin aware graph contrastive learning. NeurlPS, pp. 30414-30425, 2021.
You, Y., Chen, T., Sui, Y., Chen, T., Wang, Z., and Shen, Y. Graph contrastive learning with augmentations. In NeurIPS, pp. 5812-5823, 2020.
Zhang, G., Dong, H., Zhang, Y., Li, Z., Chen, D., Wang, K., Chen, T., Liang. Y., Cheng, D., and Wang, K. Gder: Safe- guarding efficiency, balancing, and robustness via proto- typical graph pruning. arXiv preprint arXiv.2410.13761, 2024a.
Zhang, G., Sun, X., Yue, Y., Wang, K., Chen, T., and Pan, S. Graph sparsification via mixture of graphs. arXiv preprint arXiv:2405. 14260, 2024b.
Zhang, G., Yue, Y., Sun, X., Wan, G., Yu, M., Fang, J., Wang, K., Chen, T., and Cheng, D. G-designer: Architect- ing multi-agent communication topologies via graph neu- ral networks. arXiv preprint arXiv:2410. 17722, 2024c.
Zhu, Y., Xu. Y., Liu, Q. and Wu, S. An empirical study of graph contrastive learning. arXiv preprint 17X1v22109.01116.000 2021.

---

Rethink GraphODE Generalization within Coupled Dynamical System
A. Datasets and Settings
The details of these datasets are described below: The SPRING dataset simulates a system of interconnected springs governed by Hooke's 's law. Each sample contains interacting springs with properties such as clasticity coefficients, initial positions, velocity, and acceleration. The system parameters include box size (a), initial velocity norm (3), interaction strength (7), and spring connection probability (6). The CHARGED dataset simulates electromagnetic phenomena, where particles interact through Coulomb forces. Similar to the SPRING dataset, each sample contains 5 particles with properties such as charge, mass, and initial velocity. The system parameters include box size (a), initial velocity norm (3), interaction strength (7), and charge probability (ő). Notably, particles in the CHARGED system attract or repel with equal probability, while the SPRING system has unequal probabilities for spring connections. The PENDULUM dataset simulates a system of connected pendulums, where each pendulum is governed by the laws of classical mechanics. Each sample contains 3 connected pendulums with properties such as stick length, mass, and initial angular positions. The system parameters include: .L: Length of the pendulum sticks (in meters), Mass of the pendulum sticks (in kilograms), onee Standard deviation of initial angular positions (in radians), Vnorm Norm of initial angular velocities (in radians per second).
The datasets are split into training, validation, and test sets, with additional out-off-dstribuun (OOD) test sets to evaluate model generalization. The parameter ranges for ID (in-distribution) and OOD -uttof--iistibbution) settings are as follows: The datasets are generated using a physics-based simulation framework, where the system parameters are sampled from
Parameter
SPRING [4.9, (ID)
CHARGED
PENDULUM
a (box size)
[4.9, 5.1] (ID)
3 (initial velocity norm)
[0.49, 0.51] (ID)
[0.49, 0.51] (ID)
? (interaction strength)
[0.09, 0.11] (ID)
[0.9, 1.1] (ID)
8 (spring/charge probability)
[0.49, 0.51] (ID)
[0.49, 0.51] (ID)
L (stick length)
[0.9, 1.1] (ID)
m (stick mass)
[0.9, [.]](ID)
OToc (position std)
[0.09, 0.11] (ID)
Uporm (velocity norm)
[0.49, 0.51J (ID)
a E [4.5,
a E [4.5, 5.5]
LE E 00.5,
8 E [0.45, 0.55)
3 E [0.45, 0.55]
m E [0.5, 1.5
OOD Test Set
7 E [0.05, 0.15]
Y E [0.5, 1.5]
Toe E [0.05, 0.15]
8 (0.45,
8 E (0.45, 0.55]
Unorm
[0.45, 0.55)
Number of Samples
Training Set
5000
5000
5000
Validation Set
1000
1000
1000
Test Set OOD Test Set
1000
1000
1000
1000
1000
1000
Table Datasets and distributions of system parameters. For the OOD test set, there is at least one the system parameters outside the range utilized for training.
the specified ranges. For the OOD test sets, at least one parameter is sampled outside the ID range to evaluate the model's generalization capability. The training, validation, and test sets are split as follows: 5000 samples for training, 1000 samples for validation, 1000 samples for testing, and 1000 samples for OOD testing. This setup ensures a rigorous evaluation the model's performance in both in-distribution and -iittii-ittiittion scenarios.
B. Proof of Equation (14)
We first introduce the fundamental rules of do-calculus, which are crucial for understanding the causal relationships between variables in a directed acyclic graph (DAG). These rules allow us to manipulate and simplify expressions involving interventions and observations in causal models. We will then apply these rules to derive the causal intervention for causal inference the context of a Structural Causal Model (SCM), as shown in Figure 2.
12

---

Rethink GraphODE Generalization within Coupled Dynamical System
B.1. Rules of Do-Calculus
Do-calculus is formal system introduced by Pearl that provides set of rules handle interventional distributions. Let G' be directed acyelic graph (DAG) representing the causal relationships between variables, and let Gao(s) be the graph obtained by intervening variable X. The graph Gáo(a) is identical to G' except for the removal of all arrows leading X from its parents. The nullified graph Gnat((a) is defined as the graph where all arrows from X have been removed. We can now introduce three primary rules of do-calculus (Pearl et al., 2000; Pearl, 2016):
Insertion/Deletion of Observations (Rule 1): This rule states that if Y and Z are independent given X in the interventional graph Gdo(z), then we can remove Z from the conditioning set:
Ply do(x), Pyy do(x)) if y 1 z x in Gdo(z))
2. Action/OOseervttion Exchange (Rule 2): This rule allows for exchanging an intervention on a variable with an observation, provided that is independent of other variables when conditioned on X the interventional graph Gdo(z) and the nullified graph Grult(s):
Ply do(a:), do(=)) Ply 2) if y 2 Gdo(z): Gnull(s)
3. Insertion/Deletinn of Actions (Rule 3): This rule asserts that if Y and Z are independent given X in the interventional graph Gao(2) and the graph where both X and Z are intervened, we can remove the intervention on Z from the conditioning set:
Ply do(z)) Ply do(x)) if y I x in Gao(z)) Gdo(z)-
These rules allow to manipulate interventional distributions and simplify expressions involving interventions in causal graphs. They are particularly useful for deriving causal effects from observational data when randomization or controlled experiments are not feasible.
B.2. Interventional Likelihood Derivation
We derive the interventional likelihood for the coupled dynamical system by combining causal calculus with neural ODEs. Let (T)) denote the observational target at terminal time T. (4)() the latent states governed by physical interaction dynamics, and (t) the latent coupling factors mediating state transitions. The derivation establishes a deconfounded learning objective through the following key transformations:
logp(y (o (ta))
loog [p(n: (T) he (T) (ac))))))) hhhpaathhhh.....oaaasahhhhhhhhaa
(20)
log
(lu:(T)) 85; O ((ss)
Σ ((b,,T))dd((((()))))
(21)
= log
((u,(T))),,T) 8: Ou (pss)
(22)
log
(I Oaw)pss)
Σ
Ep(h,(T))hatto) G: eode
(23)
The derivation progresses through three erucial causal manipulations. First, marginalize the latent coupling factors 7); while preserving the (o():(lo) intervention in both ODE dynamics and coupling factor distributions. Subsequently, we apply do-calculus Rule 2 to climinate the intervention from (To) (ado(h;(te) (to)), n: 9) by leveraging the d-separation property in (aock given the parent states (to),, }. the intervention becomes conditionally independent of subsequent state evolution. Finally, Rule 3 justifies replacing p(n) do h2 (to)) with p(n) since the coupling factors remain invariant to interventions on system states under our structural causal model.
13

---

Rethink GraphODE Generalization within Coupled Dynamical System
To address the temporal confounding effects in coupled dynamics, decompose the joint coupling factor distribution through Markovian factorization:
T-1
p(m(())..... ...0,))).............. " 1))
II p(n(1))
Combined with the state evolution process p(h;(t+++))//pp over all possible latent trajectories:
we expand Equation (23) into path-space integrals
log (T)()(T)), s43 dcc))((s))
Σ (. (T(T)))
II =
phult + ()),((t) O.)))((
(24)
si dcc))((s))
log
Σ m((t)
1) Oxx))(((())
(25)
As shown in Equation (24), the temporal factorization enables decomposing the joint likelihood into products of Markov transitions governed by the coupled dynamics. The logarithmic transformation in Equation (25) further separates the objective into additive components across time steps, revealing the cumulative impact of coupling factors on system evolution.
To resolve the intractability induced by latent coupling factors, we introduce variational posterior q(1 (1) (1)) that approximates the true confounding distribution. Through importance sampling and Jensen's 'ss inequality, derive the evidence lower bound:
log s;; (ccc)p(ss)
+ ()) O P0000)00000.00000000000000000 ((((0)))
log Σ
Σ
(26)
((,(1)444))
kc((++1) mi(t)
log plpp(u(( (T) (T)) 85; (cc))(()))
log Σ
(h,((++1) (1](++))))())))))))))))))) odde)
(2(++1)
KL (g()())))))))) (())
(27)
The variational formulation in Equation (27) achieves three critical objectives: 1) The reconstruction term preserves observational consistency through the decoder aeci 2) The expectation term enforces physical plausibility of state transitions under the neural ODE constraints; 3) The KL divergence regularizes the coupling factor posterior against domain-agnostic prior (t), effectively mitigating spurious correlations induced by context-specific coupling patterns. This regularization forces the model to disentangle invariant physical laws from transient coupling effects, as visualized in Figure 2.
By bridging causal intervention theory with variational inference, the derived bound enables robust learning of coupled dynamics that generalizes beyond observed coupling configurations. The complete implementation framework building upon this theoretical foundation detailed Sec. 3.3.
14

---

Rethink GraphODE Generalization within Coupled Dynamical System
C. Related Work
C.1. Graph Neural Networks
Graph Neural Networks (GNNs) (Hamilton et al., 2017; Veličković et al., 2017; Kipf & Welling, 2017; Fu et al., 2022: Chen al., 2024) are widely recognized for processing non-Euclidean data structures, including social network analysis and recommender systems (Xu et al., 2021; Cai al., 2023; Liu et al., 2023; Chen et al., 2023). They update node representations by aggregating information from neighbors via pessage-passing (He et al., 2022a; You et al., 2020; Liu et al., 2022; Zhu et al., 2021; He et al., 2022b; Wan et al., 2025b). This approach enables GNNs to capture the complex dependencies (Zhang et al., 2024b;a) and structures inherent in graph data (Xia et al., 2021; Wan et al., 2024b; Zhang et al., 2024c; Huang et al., 2023a; Wan et al., 2024a). However, due to their unique message-passing mechanism, GNNs often learn patterns specific to the context of neighboring nodes. In coupled dynamic systems, this can lead to models that overfit to certain biases present in the training data, resulting loss of generalization ability under new conditions.
C.2. Neural Ordinary Differential Equation
Neural Ordinary Differential Equation (ODE) is a class of deep learning models that treat the depth of a neural network as a continuous variable, enabling the modeling of complex, continuous-time dynamics. Introduced in (Chen et al., 2018), Neural ODEs parameterize the derivative of the hidden state using neural network, allowing for the modeling of both static and dynamic systems (Chapfuwa et al.; Auzina et al.; Mei et al.). Building upon this framework, Graph Neural Ordinary Differential Equation (GraphODE) extend the concept to graph-structured data (Huang et al., 2020; 2021; Luo et al., 2023; Huang et al., 2023b; Qin et al., 2024; Huang et al., 2024a; Wan et al., 2025a). In (Poli et al., 2019), researchers formalized GraphODE as the continuous counterpart to GNN, where the input-ouiput relationship is determined by a continuum of GNN layers. This approach blends discrete topological structures with differential equations, offering computational advantages and improved performance by leveraging the geometry of underlying dynamics (Wen et al., 2022; Huang et al., 2024b).
D. Spatio-temporal GNN
Following the spatio-temporal GNN backbone (Huang et al., 2021; 2024b; Luo et al., 2024), the process begins with the construction of a temporal graph Gtem, where nodes represent individual observations, and edges encode both temporal and spatial relationships. Temporal edges connect consecutive observations of the same entity, while spatial edges link observations of different entities at the same timestamp. The adjacency matrix A tem is defined as:
wis
L L'
Com (1 (1.
i=
(28)
otherwise.
where i denotes the observation of entity i at time f, and w/ represents weights derived from the spatial graph G.. To capture temporal and spatial dependencies, information is propagated through the temporal graph using message-passing mechanism. The representation of each node at layer / is denoted as o! (t), which is iteratively updated. Temporal patterns are incorporated by adding a temporal encoding TE) to each node, where
t
[ = sin (100002777)
TE())22 + +1] cos
100002k74
with d representing the feature dimension. The interaction between nodes is modeled using attention, computed as:
tem (s', (6',5') (w, (w,a("()) (1) T (W)of(())):
a 10',,, ((' =
(29)
Vd
where (t) = (t) + TE(t), and W q. Wa E Rddxd are learnable matrices. Using these interaction scores, the node representation is updated as:
(t) = of to
Σ a (f (w.of(())
(30)
where (it)) denotes the neighbors of 1 W. E Rdxd is a learnable value transformation matrix, and a represents a non-linear activation function. After propagating through layers, the observation representations are aggregated into
15

---

Rethink GraphODE Generalization within Coupled Dynamical System
entity-level latent states. The final initial dynamic state of the observation is given by:
1 M
q.(1) of (1) +TE()) h.(to)
Σ oWW.((()))
(31)
M
where M is the number of historical observations, and W. Rd is learnable transformation matrix. These latent states {to)) serve the initialized dynamic states for each agent. This approach transforms the enhanced dynamic representation o; into temporally enriched and spatially aware representations, effectively capturing dependencies essential for modeling complex systems.
E. Implement Details
The experiments are conducted using NVIDIA GeForee RTX 3090 GPUs the hardware platform, coupled with Intel(R) Xeon(R) Gold 6240 CPU @ 2.60GHz. The deep learning framework employed was Pytorch, version 1.11.0, 1.0, alongside CUDA version 11.3. We utilize the MLP for any projection models. The hidden layer size set to 32 for each dataset. For optimization, the Adam optimizer (Kingma & Ba, 2014) was chosen, with a learning rate of le - and weight decay of le - 3 during the training process.
F. Sensitivity (Q4)
To investigate the sensitivity of GREAT to the changes substate K and the dimension p, we analyze the performance across both ID and OOD settings. The analysis focuses on understanding how the model behaves when varying these key factors. As shown in Figure 7, we have inverted the RMSE values, higher bars indicate better performance. We observe the following trends: Obs. O In any scenario, higher dimensions generally lead to better performance. Lower dimensions constrain the model's 's expressiveness, reducing the separability of projections, and thus fail to properly disentangle the dynamics. While increasing the dimension (c.g., to 64) brings some improvement, the effect becomes limited, and the computational overhead increases. Therefore, the choice of dimension should balance performance and computational efficiency. Obs. @ In the ID setting, the number of substates has relatively small impact on model performance. Smaller substate numbers are sufficient for simpler tasks. However, in the OOD setting, higher number of substates contributes to greater diversity of coupling patterns, which enhances the model's generalization ability. This aligns with the design motivation of our model, as higher substate numbers help capture more complex coupling relations, thus improving adaptability to unseen scenarios.
16


Reviews:
[{"review_id": "HR5ExAE1bh", "reviewer": "Reviewer_LVHe", "summary": "This paper introduces a new framework named GREAT (Generalizable GraphODE with disEntanglement And regularizaTion) to enhance the generalization capabilities of Graph Ordinary Differential Equations (GraphODE) models for coupled dynamical systems. The key contributions and findings include identifying generalization challenges in GraphODE and devise corresponding modules to overcome. The results in three datasets show good performance in both in-distribution (ID) and out-of-distribution (OOD) settings.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Response to Reviewer LVHe\nThank you for your thorough review and constructive feedback. We address your concerns below and hope these clarifications will help you re-evaluate and update the score:\n\n> `Weakness 1`: Systems too simple with low dimensionality (≤5).\n\nThank you for this observation. We would like to offer several considerations that guided our experimental design:\n\n1) These systems exhibit complex nonlinear dynamics and chaotic behaviors. We calculated the Maximum Lyapunov Exponent (MLE) for each system, where positive values indicate chaotic dynamics:\n\n    |SPRING|CHARGED|PENDULUM|\n    |-|-|-|\n    |0.651|0.875|18.932|\n\n    As shown, all systems exhibit positive MLEs, with PENDULUM showing strong chaos (MLE≫0). The SPRING system captures elastic interactions with long-range dependencies, the CHARGED system models electromagnetic forces with phase transitions, and the PENDULUM system demonstrates extreme sensitivity to initial conditions—each representing distinct coupling mechanisms common in real-world applications.\n\n2) Our research addresses two challenges in GraphODE generalization: static-dynamic entanglement and coupling pattern bias. These challenges **are intrinsic to coupled systems regardless of dimensionality**.\n\n3) Following established practice [1,2], these systems serve as standard benchmarks enabling fair comparison with prior work.\n\n4) We conducted additional experiments on a higher-dimensional SPRING system (10 particles) and the MD17 molecular dynamics (please refer to response to Reviewer qXoi), where GREAT consistently outperformed baselines:\n\n    |RMSE|LatentODE|LG-ODE|PGODE|GREAT|\n    |-|-|-|-|-|\n    |ID|7.65|6.27|4.92|4.29|\n    |OOD|8.37|6.81|5.39|5.01|\n\n[1]: Neural Relational Inference for Interacting Systems. In ICML, 2018.\n[2]: Physics-informed regularization for domain-agnostic dynamical system modeling. In NeurIPS, 2024.\n\n> `Weakness 2`: Unclear motivation for Dynamic Hawkes Process Augmentation.\n\nDHPA addresses a fundamental challenge: coupled systems exhibit complex temporal dependencies that basic representations cannot capture. While our disentanglement separates static and dynamic features, the dynamic component needs enhancement to model multi-scale patterns. Following a \"divide and conquer\" strategy, DHPA strengthens the dynamic representation with weighted historical information: $ \\hat{o}\\_i(t) = o_i(t) + \\delta\\sum_{\\tau=1}^s w_{\\tau,t}\\cdot o_i(t-\\tau) $. This approach is **universal across systems** because its adaptive weights $ w_{\\tau,t} $ automatically adjust to each system's temporal characteristics, with its effectiveness confirmed by our ablation studies (Fig.5).\n\n> `Weakness 3`: Insufficient validation of claimed advantages.\n\nWe provided ablation studies in Figure 5 and Section 4.4 demonstrating each component's contribution. Additional evidence validates our key contributions:\n\nChallenge 1: Static-Dynamic Disentanglement: The orthogonality loss decreases during training, showing successful separation:\n\n|Epoch|1|100|200|400|\n|-|-|-|-|-|\n|SPRING|13.88|2.11|0.67|0.54|\n|CHARGED|18.94|5.63|2.15|1.62|\n|PENDULUM|8.76|1.58|0.63|0.64|\n\nFeature separation testing: Mutual information (MI) analysis between static and dynamic representations with permutation tests: https://anonymous.4open.science/r/Response_to_Reviewer_LVHe-F29E/table-MI.md. MI was estimated using the Kraskov estimator across 1000 time points, with permutation tests (10,000 shuffles). Decreasing p-values confirm statistically significant separation. The steady MI decrease demonstrates our model eliminates information leakage between representation spaces.\n\nTemporal consistency: Cosine similarity between representations at t=0 and later points confirms static representations maintain higher consistency: https://anonymous.4open.science/r/Response_to_Reviewer_LVHe-F29E/table-time-consistency.md\n\nChallenge 2: Coupling Pattern Capture: Our experiments and ablation studies demonstrate GREAT's superior performance in OOD scenarios. Our theoretical analysis in Section 3.3 provides rigorous justification by formulating the problem through a causal lens - modeling coupling factors as confounding variables and deriving an interventional likelihood $p(y_i(T)|do(h_i(t_0)), \\mathcal{G})$ rather than observational likelihood. Theorem 3.1 proves this formulation captures intrinsic physical dynamics invariant, going beyond empirical validation to provide theoretical guarantees.\n\nWe will include more detailed analyses in the revised version.\n\n> `Other Comment`: Excessive icons in Figure 3.\n\nWe agree and will simplify Figure 3 while maintaining the figure's informational content.", "claims_and_evidence": "The two main challenges claimed in this paper are appropriately tackled in the method design.\n\nHowever, I am not fully convinced by the presented results since only average performance metrics are included. For example, \n\n- For challenge 1, does the proposed method correctly decouple static and dynamic parts?\n\n- For challenge 2, does the proposed method correctly capture the coupling factor?", "methods_and_evaluation_criteria": "Most parts of the method design are reasonable. However,\n\n- The motivation behind Dynamic Hawkes Process Augmentation is unclear. Is it universal to all kinds of dynamical systems?", "theoretical_claims": "Checked.", "experimental_designs_or_analyses": "I think more ablation studies and in-depth analyses are needed to demonstrate that the proposed method successfully overcomes two challenges.", "relation_to_broader_scientific_literature": "Coupled dynamic systems are ubiquitous across domains. However, the conducted experiments only cover a small range of simplified systems.", "essential_references_not_discussed": "Enough.", "other_strengths_and_weaknesses": "Strength\n\n- Rigorous deriviation and method design.\n\n- Well-motivated in terms of two main challenges.\n\n- Good performance compared with several SOTA methods.\n\nWeakness (Detailed in above comments)\n\n- The covered systems seem too simple. For example, the dimensionality of these systems is no greater than 5.\n\n- Lack of motivation for Dynamic Hawkes Process Augmentation.\n\n- Performance evaluations cannot fully validate the claimed advantages.", "other_comments_or_suggestions": "Figure 3 is Beautiful, but I think the icons are too much, and some of them are not necessary.", "questions_for_authors": "Please see my comments above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vbohJgynSE", "reviewer": "Reviewer_qXoi", "summary": "The paper proposes a new graphODE-based method to process the compiled dynamical system. The trajectory is decomposed into dynamic and static parts in the latent space. The experiments demonstrate the effectiveness of the proposed method.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Response to Reviewer qXoi\n\nThank you for your constructive comments and positive assessment of our work. We address your concerns below and hope our responses will help update your score:\n\n> `Weakness 1`: Limited background on coupling dynamics.\n\nWe sincerely appreciate this valuable suggestion. Due to space limitations, we provided only essential background information. We will expand the background section in the revised version to include more examples and fundamental principles of coupled dynamical systems.\n\n> `Weakness 2 & Comment 2`: Small datasets and need molecular system testing.\n\nWe greatly appreciate these insightful suggestions. We respectfully would like to clarify several points regarding the systems in our experiments:\n\n1) While appearing simple, our systems exhibit strong interaction patterns: the CHARGED system demonstrates complex electromagnetic interactions with high coupling strengths, and the PENDULUM system captures chaotic behaviors with highly entangled dynamics.\n\n2) Following standard settings in previous works [1,2], these systems were deliberately chosen as they present the core challenges (static-dynamic entanglement and coupling bias) that our method addresses. These **challenges are fundamental** across coupled systems regardless of complexity level and have been widely recognized in the field.\n\n3) Our OOD experiments already demonstrate generalization across varying interaction strengths by testing with different coupling parameters (e.g., interaction strength γ∈[0.05,0.15] for SPRING and γ∈[0.5,1.5] for CHARGED in OOD settings).\n\n4) We appreciate your suggestion about molecular systems and have conducted preliminary experiments on MD17:\n\n| Method     | Aspirin | Benzene | Ethanol | Malonaldehyde | Naphthalene | Salicylic | Toluene | Uracil |\n|------------|---------|---------|---------|---------------|-------------|-----------|---------|--------|\n| LatentODE  | 0.667   | 0.809   | 0.272   | 0.698         | 0.132       | 0.186     | 0.351   | 0.154   |\n| LG-ODE     | 0.436   | 0.581   | 0.240   | 0.348         | **0.046**   | 0.092     | 0.157   | 0.091   |\n| PGODE      | 0.398   | 0.534   | 0.163   | 0.325         | 0.057       | 0.085     | 0.134   | 0.074   |\n| GREAT      | **0.252** | **0.123** | **0.156** | **0.317**  | 0.048     | **0.079**  | **0.127** | **0.064** |\n\nWe report RMSE values (Å) on MD17 dataset with irregular temporal sampling where only 60% of timesteps were randomly sampled and follow other stanard setting in our manuscript. GREAT also performs well on molecular systems because these systems naturally exhibit the same challenges our method addresses: varying interaction patterns. Our approach effectively captures both local and global dynamics, achieving the best performance on 7 molecules. \nWe acknowledge the value of testing on diverse systems and will include more comprehensive results including QM9 in the revised version.\n\n\n\n> `Comment 1`: Need visualization of orthogonality loss.\n\nThank you for this valuable suggestion. The orthogonality loss trajectory during training on SPRING:\n\n| Epoch: | 1 | 50 | 100 | 150 | 200 | 300 | 400 |\n|--------|---|----|----|----|----|----|----|\n| Loss (Lo): | 13.88 | 5.42 | 2.11 | 1.25 | 0.67 | 0.55 | 0.54 |\n\nAs shown, the orthogonality loss steadily decreases during training, indicating effective disentanglement of static and dynamic components. We have also demonstrated its effectiveness through ablation studies in Sec. 4.4 and Figure 5. We will include more detailed visualizations in the revised version.\n\n> `Concern 1 (Experimental Designs Or Analyses)`: OOD setting.\n\nOur OOD split based on system parameters follows established practices in prior work [1] and directly tests the primary challenge in coupled dynamical systems: generalization across varying initialization conditions. This design has strong physical meaning as parameter changes, fundamentally alter system dynamics.\n\n\n\n\n[1]: Pgode: Towards high-quality system dynamics modeling. In ICML, 2024. \n[2]: Physics-informed regularization for domain-agnostic dynamical system modeling. In NeurIPS, 2024.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "No", "experimental_designs_or_analyses": "I'm not sure the OOD split according to the system parameters makes sense.", "relation_to_broader_scientific_literature": "GraphODE is a previous work.", "essential_references_not_discussed": "None", "other_strengths_and_weaknesses": "Strengths:\n- The paper is well-written and well organized.\n- The idea of decomposing the trajectory to static and dynamic parts in the latent space is interesting and technique sound.\n- This paper also provides many useful insights, detailed discussions, and theoretical support. \n\nWeakness:\n- The introduction of background on coupling dynamic is limited.  A detailed background on coupled dynamical systems contributes to a better understanding.\n- The experiments only contain 3 small datasets. More experiments on other domains like molecule or protein will make the paper more convincing.", "other_comments_or_suggestions": "-  It is better to visualize the orthogonality loss L_o, which would be helpful to let the reader know if the decomposition is successful.\n- The test sets seem all about the small particle system; those particles don't have strong interaction with each other. Can the method test on molecular systems, such as MD17 and QM9 in [1].\n\n[1] Geometric Trajectory Diffusion Models, NIPS 2024", "questions_for_authors": "None", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CF3m6zPpB5", "reviewer": "Reviewer_XMqH", "summary": "This paper presents the GREAT framework to improve GraphODE models' generalization in coupled dynamical systems. It tackles two key issues: the entanglement of static and dynamic information during initialization and the reliance on environment-specific coupling patterns. The framework introduces two modules: the DyStaED for separating features via orthogonal projections, and CMCD, which uses variational inference to disentangle latent coupling factors and reduce spurious correlations. Validated through extensive experiments, the approach shows significant improvements in both in-distribution and out-of-distribution scenarios.", "questions": "", "limitations": "", "rating": 5, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Response to Reviewer XMqH\n\nThank you for your thorough review and encouraging feedback. We are grateful for your positive assessment about novelty and importance of our work. We address your questions below:\n\n> `Weakness`: Limited exploration of approach limitations.\n\nWe fully agree that a more thorough discussion of limitations would significantly strengthen the paper. In the revised version, we will add a dedicated limitations section that provides comprehensive analysis. One potential limitation is training stability: Similar to many deep learning approaches utilizing variational inference, our method requires careful tuning to balance multiple loss components (prediction loss, orthogonality loss, and KL divergence term). We will provide detailed discussion of these potential training challenges along with practical strategies for mitigating them through hyperparameter selection and optimization techniques.\n\n\n> `Comment`: Need clearer baseline implementation details.\n\nWe sincerely thank you for this suggestion. We will enhance the baseline description and implementation details in the revised version.\n\n> `Question 1`: Robustness when static/dynamic distinction is ambiguous?\n\nThis is an excellent question. Our decoupling mechanism employs several strategies to maintain robustness:\n1) Orthogonality regularization ($L_o$): The loss function in Eq.8 enforces strict orthogonality between the static and dynamic subspaces, helping to separate features even when they appear entangled. Our experiments show that without this regularization, performance drops by 12.3% on average in OOD settings.\n2) Learnable subspaces: Rather than using predetermined feature splits, our approach learns the separation through end-to-end training. The embedding layers $S_{static}$ and $S_{dynamic}$ adaptively determine the optimal projection for each domain, allowing the model to discover natural separations even in ambiguous cases.\n3) We employ a synergy decoder (Eq.12) that allows information to flow from both static and dynamic components. This ensures that even if some features are misclassified during training, critical information is not lost during reconstruction.\n4) We introduce a DHPA that dynamically adjusts the importance of different feature patterns at multiple scales. This helps the model better handle ambiguous cases by learning to focus on the most relevant patterns for dynamic separation at each level of abstraction. Our experiments show this further improves robustness in Figure 5.\n\n\n> `Question 2`: Stability across different training configurations?\n\nWe have conducted extensive experiments to verify GREAT's stability across different training configurations. The final reported results from all tables are the result of 5 random seeds.\nHere, we also tested various learning rates and training epochs on the SPRING dataset. The results demonstrate that GREAT maintains stable performance across a range of configurations:\n\n| Metric | LR=1e-3 | LR=5e-4 | LR=1e-4 | LR=5e-5 | LR=1e-5 |\n|--------|---------|---------|---------|---------|---------|\n| SPRING ID (RMSE) | 3.854 | 3.792 | 3.736 | 3.705 | **3.687** |\n| SPRING OOD (RMSE) | 3.814 | 3.769 | 3.726 | 3.683 | **3.619** |\n\n| Epochs | 100 | 200 | 300 | 400 | 500 |\n|--------|-----|-----|-----|-----|-----|\n| SPRING ID (RMSE) | 4.243 | 3.986 | 3.782 | **3.687** | 3.721 |\n| SPRING OOD (RMSE) | 4.279 | 3.947 | 3.751 | **3.619** | 3.655 |", "claims_and_evidence": "Based on the description, the claims in the paper appear to be well-supported by evidence. The paper does an excellent job in aligning its problem description with the motivation provided by the SCM causal analysis. It accurately and intuitively identifies the issues in existing methods.", "methods_and_evaluation_criteria": "The paper uses RMSE and MAPE as key evaluation metrics. RMSE captures overall accuracy and penalizes larger errors, while MAPE provides a clear percentage-based evaluation, making it ideal for comparing relative errors. The use of both ID and OOD scenarios is also robust. ID testing ensures good performance in familiar conditions, while OOD evaluation tests the model’s ability to generalize in new, unseen environments.", "theoretical_claims": "The paper provides a strong, systematic causal analysis using causal graphs to identify and explain the key issues in existing methods. The causal analysis is thorough and offers an insightful understanding of the challenges faced by GraphODE models, leading to constructive Generalizable GraphODE Design Principle. Furthermore, Theorem 3.1 is a key contribution, providing a clear and convincing theoretical result that underpins the effectiveness of the proposed causal mediation approach. The theorem articulates a critical relationship between state evolution and coupling dynamics, offering a rigorous justification for the model's ability to generalize across both ID and OOD scenarios.", "experimental_designs_or_analyses": "The experimental design is robust and comprehensive, incorporating both ID and OOD evaluations across simulated systems like SPRING, CHARGED, and PENDULUM. The comprehensive ablation studies (Figure 5) effectively isolate and highlight the contributions of both the DyStaED and the CMCD modules.", "relation_to_broader_scientific_literature": "The paper leverages advanced AI techniques to deepen our understanding of physical systems. It builds on earlier work that applied machine learning to nonlinear dynamics, offering fresh interdisciplinary insights that expand the boundaries of AI in physics.", "essential_references_not_discussed": "All related references have been included.", "other_strengths_and_weaknesses": "### Paper Strength\n\n-\tThe GREAT framework rethinks traditional approaches by decoupling static and dynamic attributes using causal inference, which allows the model to effectively mitigate confounding factors and achieve superior generalization.\n-\tThe paper establishes a robust theoretical framework by integrating causal inference into the coupling patterns within GraphODE models, while its extensive experiments—spanning multiple simulated datasets and testing both ID and OOD conditions—thoroughly validate the proposed approach.\n-\tThe paper features a clearly articulated problem description and a compelling motivation grounded in Structural Causal Models (SCM), which together provide a great foundation for the proposed method.\n-\tThe paper’s diagrams and figures are exceptionally well-designed, combining aesthetic appeal with clarity to effectively convey complex concepts and experimental results.\n\n\n###  Paper Weakness\n\n-\tAlthough the paper acknowledges some inherent constraints, it does not thoroughly explore the potential limitations of its approach.", "other_comments_or_suggestions": "The author should provide clearer details on the baseline in the appendix—specifically, parameter settings, configurations, or any implementation differences.", "questions_for_authors": "1.\tIn scenarios where the distinction between static and dynamic features is ambiguous, how robust is the decoupling mechanism, and what measures are in place to prevent misclassification?\n2.\tAre there experiments to verify the stability of the GREAT model under different training rounds and initial conditions? If the model exhibits instability or significant fluctuations in certain situations, can it be interpreted as limitations in some assumptions or parameter choices of the model?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UxooUUZoej", "reviewer": "Reviewer_CHTQ", "summary": "The paper investigates the generalization challenges of GraphODE models when used for coupled dynamical systems. It shows that mixing static attributes with dynamic states during initialization, along with an over-reliance on context-specific coupling patterns, can hurt the model’s performance in new settings. To overcome these issues, the paper introduces the GREAT framework, which features a Dynamic-Static Equilibrium Decoupler to clearly separate static and dynamic components, and a Causal Mediation for Coupled Dynamics module that uses variational inference to reduce misleading correlations. Experiments on systems like SPRING, CHARGED, and PENDULUM demonstrate that GREAT outperforms existing methods in both familiar and new environments.\n\n## Update after rebuttal\nAfter reviewing all the Reviewer-Authors discussions, I would like to express my strong support for this paper. Most reviewers have recognized the novelty and contributions of this work. The authors have presented an innovative framework that addresses fundamental challenges in GraphODE generalization for coupled dynamical systems.\nI initially raised concerns about parameter sensitivity analysis and baseline method discussions. The authors' rebuttal thoroughly addressed these points with clear technical explanations and comprehensive empirical evidence. Their responses not only resolved my concerns but further strengthened my appreciation of their technical contributions.\nGiven the quality of both the original submission and their thorough rebuttal, I am raising my score.", "questions": "", "limitations": "", "rating": 5, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Response to Reviewer CHTQ\n\nThank you for your positive review and insightful questions. We are grateful for your recognition of our work's novelty and contributions. We address your comments below:\n\n> `Weakness 1`: Need more parameter sensitivity analysis.\n\nWe appreciate this valuable feedback. For example, the orthogonality loss weight $\\lambda_o$ is critical for balancing disentanglement and prediction accuracy. Our preliminary analysis shows:\n\n| Metric | $\\lambda_o$=0.1 | $\\lambda_o$=0.3 | $\\lambda_o$=0.5 | $\\lambda_o$=1.0 | $\\lambda_o$=2.0 |\n|--------|--------------|-------------|------------|------------|-------------|\n| SPRING ID (RMSE) | 3.945 | 3.803 | **3.687** | 3.714 | 3.892 |\n| SPRING OOD (RMSE) | 3.986 | 3.754 | 3.651 | **3.619** | 3.785 |\n\nAs shown above, $\\lambda_o$=0.5 and $\\lambda_o$=1.0 provide the optimal balance for ID and OOD settings respectively. Too small a value ($\\lambda_o$<0.3) results in insufficient separation of static and dynamic components, while too large a value ($\\lambda_o$>1.0) overly constrains the model's expressiveness. The model maintains strong performance across a reasonable range of values.\n\n> `Weakness 2`: Limited discussion of baseline methods.\n\nThank for the friendly reminder. We will provide a more comprehensive discussion of baseline methods in the revised version.\n\n\n> `Question 1`: Implementation details of variational inference?\n\nOur CMCD module implements variational inference by treating coupling factors $\\eta_i(t)$ as confounding variables, parameterized via $q(\\eta_i(t)|h_i(t))$ with $\\zeta_i(t) = \\text{Softmax}(W_\\eta h_i(t))$. We employ Gumbel-Softmax sampling for differentiability: $\\eta_{i,k}(t) = \\frac{\\exp((\\zeta_{i,k}(t)+g_k)/\\tau)}{\\sum_j \\exp((\\zeta_{i,j}(t)+g_j)/\\tau)}$. The KL divergence term $\\text{KL}(q(\\eta_i(t)|h_i(t))\\|p(\\eta_i(t)))$ regularizes against a context-agnostic prior, reducing spurious correlations. This approximates the interventional likelihood $\\log p(y_i(T)|\\text{do}(h_i(t_0)),\\mathcal{G})$, enabling our model to learn universal physical dynamics rather than domain-specific patterns. All components are jointly optimized, allowing the coupling inference to leverage both static and dynamic information for robust generalization.\n\n\n> `Question 2`: Reason for superior performance in longer predictions?\n\nGREAT's superior long-term prediction performance stems from several key mechanisms working in concert: First, our effective disentanglement prevents error accumulation by maintaining consistent static components while allowing dynamic components to evolve naturally. Second, the CMCD module enables coupling-aware evolution that explicitly models inter-node influences, capturing complex interdependencies crucial for extended horizons. Third, the DHPA component captures self-exciting temporal patterns at multiple scales, effectively modeling both short-term fluctuations and long-term trends. Finally, empirical stability analysis shows our error growth rate is significantly lower in Figure 6. These advantages make GREAT particularly suitable for applications requiring extended forecasting horizons in complex physical systems.", "claims_and_evidence": "All claims are well supported by evidence.", "methods_and_evaluation_criteria": "The paper uses RMSE and MAPE as key evaluation metrics. The proposed model and the evaluation criteria make sense for the problem.", "theoretical_claims": "Theoretical claims are correct and easy to understand.", "experimental_designs_or_analyses": "Overall, the experimental setup and analyses clearly demonstrate the framework's enhanced performance and generalization capabilities.", "relation_to_broader_scientific_literature": "In terms of machine learning, the paper draws on work related to neural differential equations and graph neural networks. While these techniques have been applied successfully in other domains, their application to epidemic forecasting is a significant innovation. By leveraging these advanced techniques, the paper contributes to the growing body of research on using deep learning to enhance predictive modeling in epidemiology. This approach also connects to broader trends in the literature about the fusion of classical and modern machine learning techniques to address real-world problems.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Strengths:\n\n1)\tMethod is novel. The introduction of the DyStaED provides a novel mechanism to separate these features effectively, enabling better modeling of system evolution. The CMCD module employs variational inference for a nuanced mediation analysis of state evolution, offering deeper insights into how coupling dynamics influence outcomes.\n\n2)\tThe paper innovatively combines causal inference with dynamic modeling to address the entanglement of static and dynamic components. It presents a systematic analysis using causal graphs, particularly in section 3.3, which details the state evolution and clarifies the influence of coupling dynamics. Theorem 3.1 is a standout, offering a rigorous theoretical foundation that significantly bolsters the framework’s claims.\n\n3)\tThe paper is well-organized, with a clear progression from problem definition to theoretical analysis and experimental validation.\n\nWeaknesses:\n\n1）\tAlthough the paper presents a solid experimental design, it would benefit from a more detailed sensitivity analysis regarding the choice of parameters, such as those used in variational inference or coupling patterns. This would help understand how sensitive the model is to variations in these parameters and further enhance its robustness.\n\n2）\tThe paper provides limited discussion on baseline methods. A more detailed introduction of relevant baselines would offer clearer context for the improvements claimed.", "other_comments_or_suggestions": "Please see weaknesses.", "questions_for_authors": "1.\tHow is variational inference implemented within the CMCD module? Is it integrated as part of the joint optimization during training, or is it applied as a separate post-processing step after feature extraction?\n\n2.\tWhy does GREAT exhibit the lowest RMSE in both ID and OOD settings? Is there a specific characteristic of this method that makes it more robust in longer prediction lengths?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper aims to improve the generalization of Graph Ordinary Differential Equations (GraphODEs) in the case of coupled dynamical systems. The authors use a Structural Causal Model to examine current GrapheODEs and design mitigation modules in the form of two systems:  Dynamic-Static Equilibrium Decoupler (DyStaED) and Causal Mediation for Coupled Dynamics (CMCD). The former focuses on the disentangling of static and dynamic states at the initialization step, while the latter estimates latent causal factors through variational inference. The authors show that these methods can improve in- and out- of distribution performance and apply the method to a number of datasets. 

While the reviewers noted a few shortcomings, such as the potential simplicity of some of the examples, the overall interactions with the authors were able to mitigate most of these concerns and so I recommend this work be accepted.
Decision: Accept (spotlight poster)