Non-stationary Equivariant Graph Neural Networks for Physical Dynamics Simulation
Authors: Chaohao Yuan
Maoji Wen
Ercan Engin KURUOGLU
Yang Liu
Jia Li
Tingyang Xu
Deli Zhao
Hong Cheng
Yu Rong
Venue: NeurIPS 2025 poster ()
Keywords: Equivariance
Non-stationarity
Primary area: machine_learning_for_sciences
TL;DR: We propose NS-EGNN, which is capable of modeling non-stationary physical dynamics with equivariance.

Abstract: To enhance the generalization ability of graph neural networks (GNNs) in learning and simulation physical dynamics, a series of equivariant GNNs have been developed to incorporate the symmetric inductive bias. However, the existing methods do not take into account the non-stationarity nature of physical dynamics, where the joint distribution changes over time. Moreover, previous approaches for modeling non-stationary time series typically involve normalizing the data, which disrupts the symmetric assumption inherent in physical dynamics. To model the non-stationary physical dynamics while preserving the symmetric inductive bias, we introduce a Non-Stationary Equivariant Graph Neural Network (NS-EGNN) to capture the non-stationarity in physical dynamics while preserving the symmetric property of the model. Specifically, NS-EGNN employs Fourier Transform on segments of physical dynamics to extract time-varying frequency information from the trajectories. It then uses the first and second-order differences to mitigate non-stationarity, followed by pooling for future predictions. Through capturing varying frequency characteristics and alleviate the linear and quadric trend in the raw physical dynamics, NS-EGNN better models the temporal dependencies in the physical dynamics. NS-EGNN has been applied on various types of physical dynamics, including molecular, motion and protein dynamics. In various scenario, NS-EGNN consistently surpasses the performance of existing state-of-the-art algorithms, underscoring its effectiveness. The implementation of NS-EGNN is available at https://github.com/MaojiWEN/NS-EGNN.

Non-stationary Equivariant Graph Neural Networks for Physical Dynamics Simulation
Chaohao Yuan 12 Maoji Wen '; Ercan Engin Kuruoglu Yang Liu?, Jia Li
Tingyang Xu Deli Zhao², Hong Cheng", Yu Rong
Tsinghua Shenzhen International Graduate School, Tsinghua University
2 The Chinese University of Hong Kong
3 DAMO Academy, Alibaba Group Hupan Lab,
Hong Kong University of Science and Technology (Guangzhou) chaohaoyuan@link. cuhk, edu. hk, smj240mails tsinghua edu. en, kuruogluOsz. tsinghua. edu. en, ru.roonghotmail.com com
Abstract
To enhance the generalization ability of graph neural networks (GNNs) in learning and simulating physical dynamics, a series of equivariant GNNs have been devel- oped incorporate the symmetric inductive bias. However, the existing methods do not consider the non-stationarity nature of physical dynamics, where the joint distribution changes over time. Moreover, previous approaches for modeling non- stationary time series typically involve normalizing the data, which disrupts the symmetric assumption inherent in physical dynamics. To model the non-stationary physical dynamics while preserving the symmetric inductive bias, we introduce Statioatonn Equivariant Graph Neural Network (NS-EGNN) to capture the non-stationarity in physical dynamics while preserving the symmetric property of the model. Specifically, NS-EGNN employs Fourier Transform on segments physical dynamics extract time-varying frequency information from the trajecto- ries. then uses the first and second-order differences to mitigate non-stationarity, followed by pooling for future predictions. Through capturing varying frequency characteristics and alleviate the linear and quadric trend in the raw physical dynam- ics, NS-EGNN better models the temporal dependencies in the physical dynamics. NS-EGNN has been applied on various types of physical dynamics, including molecular, motion and protein dynamics, and consistently surpasses the existing state-of-the-art algorithms, underscoring its effectiveness. The implementation NS-EGNN is available http://gtthbb.commMajjWEENNNS
Introduction
Accurately simulating real-world physical dynamics is crucial in numerous fields, including molecular dynamics, motion capture [48) drug discovery [37 54), and protein folding [1]. The challenge lies in capturing complex interactions among system components. However, traditional methods are either computationally expensive (e.g., Density Functional Theory (2I)) or fail to model the complex human intention. Hence, various equivariant Graph Neural Networks (GNNs) [38 16 39 [8] 19 24
57 22 5) have been developed model such physical interactions while incorporating 58 26, 4
fundamental symmetry constraints. Specifically, these methods ensure their outputs are equivariant with respect to a specific group. such as E(3), any 3-dimensional translationdrienaattoorrreeeoooo
"Equal Contribution.
Corresponding Authors
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Despite their success, most existing models for physical dynamics focus on single-step frame-to- frame forecasting. That is, they only leverage a single historical frame as input predict the future states. Such frameworks are insufficient to simulate physical dynamics due to the follow- ing issues: (1) Non-Markovian. According to the Markovian assumption, future states depend only on the current state and are independent of past states. However, single frame of the phys- ical dynamics does not comprehensively capture all the details of a given environment because of hidden interactions, such as those involving the solvent; (2) Non-stationary. Non-stationarity refers to a dynamic object whose statistical properties and joint distribution change over time. The time-na distribution in physical dynamics can lead to poor generalization ability in deep learning models. More intuitively, in molecular dynamics, the potential energy of molecules are varying, which results the mean, variance (amplitude of vibration) and covariance (the connections between edges) change over time. Besides, in motion capture dataset, since velocity and physio- logical state are dynamie, the trajectories of human will also exhibit strong non-stationary property.
It is non-trivial model Non-stationary and Non-Markovian dynamics. Previous works (28] for modeling non-stationary time-series data mostly adopt normalization approaches to
RocationTTanndaaioo/PPfoiioto
30 [
stationarize the series. However, these methods will break the symmetry in the dynamics. An
Patch
Patch FT
Gax
S1~St
G/m
effective altermative, Fourier Transform [41] con- verts the physical dynamics from time domain to frequency domain, revealing the intensity of each frequency component within the dynamics. The frequency components and power spectrum in the frequency domain can be used to analyze the energy distribution of the physical dynam- ies (52), which in turn can be used to infer time- domain characteristics, such as variance. As illustrated in Figure I] the Fourier frequency, Si. S2 and S3. are distinct over different period, exhibiting the dynamic statistical property of dynamics in time domain.
Patch
Patch
G:a
S2=St
G/uz:
Sy
Note: S S
Figure 1: The illustration of non-stationarity and equivariance. Non-stationarity: Applying Fourier Transform (FT) to each period of dynamic object will result in different frequency pattern. Invari- ance: With rotation, translation or reflection of physical object, the model should capture the same frequency information.
In this paper, to incorporate the non-stationarity into the equivariant models, we propose Non- Stationary Equivariant Graph Neural Network (NS-EGNN), which adopts Fourier transform on patched physical dynamics to capture the vary- ing distribution throughout the entire trajectory. Moreover, the Fourier frequency also reflects the non-Markovian interactions inside the physical systems. Specifically, to capture the dynamic statistical properties inside the non-stationary physical objects, NS-EGNN segments patches of tra- jectory with overlap and applies Fourier transform to extract the marring frequency information from these dynamics. Notably, the extracted frequency features are E(3) E(3)-invariant, thus preserving the symmetric properties of the model. Subsequently, to capture the spatial relationships. NS-EGNN employs an equivariant GNN backbone to learn these spatial connections. Moreover, since the Patch Fourier Transform already captures the dynamic patterns, NS-EGNN does not additionally incorporate temporal learning modules, such as attention [47). thus achieving lower computational complexity. Finally, NS-EGNN performs multi-step prediction utilizing the updated spatio-tempora graph through the non-stationary temporal pooling module based on the first order and the second order differencing 32 Our contributions are summarized as follows:
We reveal the non-stationary property widely exists in real-world physical objects, such as molecules, proteins and human motions, via Augmented Dickey-Fuller (ADF) 46] test.
We design a Non-Stationary Equivariant Graph Neural Network (NS-EGNN), leveraging Patch Fourier Transform with window function, to explicitly capture the time- viryiig frequency infor- mation in the dynamic object. Moreover, we design a novel equivariant temporal pooling layer to further alleviate the influence of non-statiaonarity.

---

We apply NS-EGNN across various applications, such molecular-, protein- and macro-level simulation. In the various scenarios, NS-EGNN consistently indicates performance improvements.
Background
2.1 Problem Definition
Notation. A physical object can be represented as graph G (V, The node features n, E V include non-geometric features h Rd such as the types of the atoms and 3D coordinate vector X, IR3 and the edge features Giy &, E R describes the connection between node and node j. In historical trajectory of this object {9] the scalar features and edge features are constant while the position vectors change over time.
Task Definition. In the trajectory simulation task, given the past trajectory {9). the target is to T+
learn function fo that predicts the future trajectory {9)/==
(9) fo((9)]
Specifically, since only the position vectors are dynamic, the primary focus is on predicting X.
2.2 Equivariance and Invariance
In the group of E(3), the transformation g X can be expressed as: X = OX + t, where O E (3) = fo E R O I} represents orthogonal transformations (including rotation and reflection), and e 3 represents translations.
Given above definitions, to enhance generalization ability, when a physical object undergoes transfor- mations within the group E(3), the equivariant model fo should be able to produce the corresponding prediction for the coordinates. the context of a spatio-temporal graph, this can be formulated as:
{g = EV,8, X) = So(1S (V, E, 8)
(2)
2.3 Non-stationarity
non-stationary time series exhibits dynamic statistical properties and joint distribution, resulting difficult to be modeled by deep learning models [5]] 36]. Formally, such property can be defined as:
Definition 1. (Non-stationary) Physical dynamics (Xi) can be considered
non-stationary if there exists distinct time interval t and t2 such that at least one of the following conditions is met: E(X,,) + E( Var (x,,) * Var (x,,). or Cov(X, (x,,t)) * Cov(X,,, Kigth) )) for any lag k.
In other words, if the mean, variance, covariance function of physical object evolve time, then the object is considered to be non-stationary. Furthermore, Fourier frequency also details how the variance of the data is distributed across different frequencies. the following work, we will utilize this frequency information to model and capture the non-stationary characteristics.
3 Methodology
3.1 General Framework
As shown in Figure given an EGNN backbone, NS-EGNN consists of Patch Fourier transform (PFT) (Section 3.I.I and non-stationary pooling layer (NS-Pooling) (Section 3.1.2) to model the non-stationary dynamics equivariantly. Specifically, the brief procedure can be represented as:
PFT(((()),
(3)
ML4) (L) (X(1) 2- EGNN(h, 8, x(I)T=))
(4)
X NS-Pooling( (oooling(X(()1440 (X())(4)) )-
(5)
3

---

Here, 1(D), sLL) (X(t) TTo denote the L-th EGNN layer output and X is the final pooled trajectory. PFT, we divide the trajectory into overlapping patches to capture the dynamic variance from the frequency domain, and in NS-Pooling, we employ difference- baseed method to minimize the impact by dynamic mean and perform the multi-step prediction by pooling the stationarized dynamics. PFT and NS-Pooling model the dynamics" variance and mean, respectively. First, PFT extracts the invariant frequency features 8 from the input trajectory. These features, along with
the original coordinates ( and scalar features h, are fed into the EGNN [39 backbone, as delineated in Section 3.2] which updates the node states. Finally, the NS- NS-Pooling layer is applied
to the output coordinates (X())()))T= from the EGNN to stationarize the features and make the final multi-step prediction X Finally, the training objective NS-EGNN is the mean square error
(MSB) loss L = Et=r N i=1 (1)-
3.1.1 Invariant Patch Fourier Transform
Discrete Fourier Transform (DFT). DFT is classic algorithm that converts the trajectory from temporal domain to frequency domain, which con- tain the periodical information in the physical dy- namies. Specifically, DFT F can extract the fre- quency information S; E CTx3 of the physical dynamics at node and be calculated as follows:
Patches input Trajectory
Showe
Sohwreh
Patch
Patch
(k(K)
F(X.) ((x)==(()))
Σ e tke (t) (t)),
where i' is the imaginary unit, k = 0, .,... , (6)
Tima-varying Fregouncy Graph
the frequency index and Xt(t) is the average po- sition of all nodes in the graph. Nonetheless, DFT computes single, global frequency spectrum for the entire trajectory, implicitly assuming the sig- nal's properties constant over time. We refer to this as static frequency information. Due to the inherent non-stationarity of physical dynamics, where statistical properties (and thus frequency components) change over time, this static spee- trum is insufficient. Therefore, DFT cannot com- prehensively capture the dynamic, time-varying frequency evolution inside the physical objects.
AX
Temporal
APR
Poaling
g
Equivariant MP
Predicted Trajectory
Figure 2: The overall framework of NS-EGNN. NS-EGNN applies window function the input trajectory and Fourier Transform to receive time- varying frequency graphs. Then, the equivariant GNN will learn from both spatial and spectral information to update the coordinates.
Patch Fourier Transform (PFT). To this end, inspired by [15,, we design the Patch Fourier Transform (PFT) to extract the frequency content of an object changes over time. Our target is to extract the Fourier frequency each stage along the trajectory. However, directly cutting off parts of the trajectory can result in spectral leakage (35), which means it might not properly capture the frequency information. Therefore, PFT applies window function w(:w) accurately extract the local frequency in the segmented trajectory, where w E R denotes window length as a hyper- parameter, to determine the number of samples in each segment where Fourier transform applied, defined as:
T Σ
PFT(
kp (X.(1) - ((())
where p E is the window index of Fourier transform applied, E R denotes the frequency index in p-th window, and / E E denotes the hop size, defined as the number of samples the window is moved forward at each time. Specifically, we adopt the classic Hamming window [35] w(t)= = 0.54 0.46 cos( 2 the window function. We also conduct additional experiment in Appendix C.8 to prove the specific window function will not greatly influence the performance
of NS-EGNN. By setting < w, the windows overlap, allowing for more thorough capture of the frequency information throughout the trajectory.

---

Invariant Frequency Feature. To capture the entire frequency of physical dynamics, we applied the PFT all dimensions of coordinates. In each dimension, PFT will result frequency matrix S; E IR" where T/h and K < 7 is the number of frequency basis. To integrate the frequency information from different dimensions, the frequency features will be calculated as:
S
Σ (s,
(8)
where n is the number of dimensions. We distribute the frequency feature s to each node involved in the Fourier transform calculation. Consequently, the dimension of s is extended to T by repeating the feature h times. If a coordinate takes part in multiple Fourier transforms, its corresponding frequency feature will be averaged.
Multi-Scale PFT. On the other hand, to prevent the relatively small window size w from limiting the ability to capture broader frequencies, we adopt a set of distinct window sizes and hop sizes in the implementation of NS-EGNN. By applying PFT g consecutive times with these varying hyper- parameters, NS NS-EGNN can effectively capture different scales of frequency, enhancing its ability to model non stationato trajectory. Hence, the final resulted frequency feature is given by E Ø
Lemma 3.1. The extracted frequency feature is ((n))-ivvaiint.
The proof can be found in Appendix A.1 In our model, aside from using PFT, we do not incorporate additional temporal modules such as attention mechanisms as PFT already integrates temporal information. An ablation study is set up in Section 4.5 This further enhances efficiency, as the complexity of Fast FT is O(N log N), while the complexity of attention mechanism is O(N2), and this is further validated by empirical results in Appendix C.7
3.1.2 Equivariant Non-stationary Temporal Pooling
Inspired by classical statistical algorithms H to reduce the non-stationary property, we propose NS-Pooling, to involve differencing the trajectory before pooling:
AX, (1) -() -()
-x)
(1)
- 2)] (9)
where AX, E (T-)x3 represents the differentiated. Furthermore, we also derive the second order difference A (t) AX,(t -1) Although it is possible to derive higher-order differences, we find that first-order and second-order differences are sufficient for pooling. The corresponding experiments
be found in Appendix [C. Moreover, while the previous works focused only on predicting the next single frame, our experiments extend the framework to a more challenge setting: forecasting the multi-step trajectory X; E Nx of the physical dynamics, where At represents the length of the forecasted trajectory. The formulation can be represented as:
x |AX, A A 7 X(
(10)
where y E 3x2 is learnable weight matrix. The differencing process removes linear and quadratic trends, making the trajectory more stationary and casier for the model to learn underlying patterns, rather than being distracted by absolute, non-stationary positions.
3.1.3 Equivariance Analysis
Let fa denote the overall NS-EGNN models, we have the theorem:
Theorem 3.2. For arbitrary orthogonal transformations and translation vectors O, E E(3),
fo([OG + () = Of ( +t.
The proof is provided the Appendix A.2 Since the composition of equivariant functions again equivariant, the PFT module extracts invariant frequency features to feed into the an equivariant backbone. The NS-Pooling layer aggregates the first and second-order differences of the trajectory along with the final position, which is also equivariant.
5

---

3.2 Spatial Model Backbone
To process the spatial information in the dynamies, we leverage EGNN [3 layers μ as equivariant backbone in NS-EGNN framework. With the time-varying frequency feature obtained by PFT, EGNN layers μl can iteratively update the system states as:
((+1)) g ((++1) x(++1) u(h X
(1I)
where h." and X are the scalar feature, frequency feature and geometric feature of node i at layer 1, respectively. Specifically, EGNN employs relative distance |X, X,yl as the invariant features:
(2) fo hn m
(1)
X
(12)
where fo is an MLP and i is the invariant message embedding between nodes and j at 1-th layer. Given the invariant message embedding, the node coordinates can be updated with equivariance:
[++ = (1)
(mm))
(13)
W
JEN,
where du denotes an MLP and N₁ is the neighbors of node in the physical system. In terms of invariant features, with message embedding m, the hidden representations h and s can be updated as:
jzt
(1+1) h = h
+ Σ
(h
(1+1)
-s
+
(14)
JEN,
JEN,
where fo, on and are learnable neural networks.
While we here exploit EGNN [39 as the backbone. the patched dynamics modeling proposed
a plug-and-play module that can be integrated into other models based on specific scenarios. For instance, in constrained and bounded physical systems, we could develop NS-GMN from GMN (19), and NS-DEGNN from DEGNN [58], respectively. We leave these extensions as future exploration.
Experiments
4.1 Experimental Settings
4.1.1 Datasets & Non-stationary Analysis
We perform experiments on three classic datasets: MD17 6,, 2) CMU Motion Capture Database 77. and 3) AdK cquilibrium trajectory dataset [40]. Note that these datasets all exhibit strong stationary property. Specifically, the molccular and protein dynamics in the MD17 and AdK equilib- rium trajectory datasets have varying potential energy, which influences the amplitude of vibration. In the motion capture dataset, the human's 's velocity and physiological state also vary over time.
To confirm the non-stationary nature of the datasets, we conduct Augmented Dickey-Fuller (ADF) tests [00 on each dataset. The ADF test quantifies the degree of stationarity by providing two primary metrics: the p-value, which reflects the significance of stationarity, and the ADF statistic, where smaller values indicate higher stationarity. Physical dynamics with p-values and ADF statistics below critical thresholds are deemed as stationarity, while higher values sug- gest the objects possess stronger non-stationary properties. More details can found in Ap- pendix B.
Table Summarized ADF results for MD17, CMU Motion Capture, and AdK datasets.
Subset
Non-Stat. Ratio
p- V-luuue Mean
ADF Menn
MD17 Dataset
Aspirin
0.8371
0.6684
00.9855
0.0896
Ethanol
0.9427
0.4841
-1.4999
Malonaldehyde
0.9972
0.7741
-0,4416
Naphthalene
0.7710
0.3050
-2.0771
Salicylic
0.9650
0.4540
-1.6494
Toloene
0.7840
0,2641
-2,2781
Uracil
0.4143
0.3161
-2.0884
CMU Motion Capture Dataset
Walk Run
0.9601
0,8350
1.9866
0.9449
0,7884
1.0581
AdK Dataset
Table summarizes the ADF test results for the MD17, CMU Motion Capture, and AdK
AdK
0.4873
0.1739
-3.0586

---

Table 2: Averaged prediction error for consecutive forecasts on the MD17 dataset. The reported mean
and the standard deviation (x10-3 3) are computed over 5 runs.
Aspirin
Benzene
Kthanol
Makonaldehyde
Naplsthalene
Salicylic
Tolueme
Uracil
ST-TEN
0.823ge.007 007
1.457 ao 0e3
2.573g0000
1.171
2.491 me
2.078ga.0r
1.75020
ST-GNN
10.508
15
14.34916.20
4,006 L0.00
14.725g1.95
3,064
2,401 1o
2.324
ST-SE/
2.048
1.319 0os
3.136-02216
1.063_0000
2,858
2.00
1.754
ST-EGNN
3:237 e0.544
0.877
1.878-0.225
0.922220065
1,009 2.009 so.se
1.491 equ
1.548
Fi TaMotion
3.700ggo0us
2.100
1.88220.001
2.700-00005
3:201 =0.00
3. 3,258 3.258
2.917
3. 3.258 3.258=0000
STOCN
4.175
20.0
234
3.455
3.454
3. 4,453
3,110
3. 3.570
STAN
587
2.914
303.
285
63. 260-221.11
93.0555x..00
2.250
8.038
10.500-0.301
ESTAG
0.740ga00s
j..02
0,1755
0.874 0.874_..379
0.405 0.0000
0.636_o.300
0.376
0.533 1.03
NS-EGNN
20.21
0.050=0.006
0.4100..010
0. 0.589.0.00m
0. 0.275
0.387 20,0200.00
0. 20308.00009
0.37920.077
Table 3: Final prediction error for consecutive forecasts on the MD17 dataset. Bold font indicates the best result. The reported mean and the standard deviation (x10- 3) are computed over 5 runs.
Aspirin
Benzene
Kthanol
Malonabdehyde
Naphthalene
Salieylie
Toluene
Uracil
ST-TEN ST-GNN
6.026 nos
La15
2.051
1.506ge.00
1.456 oo
1.7
2. 2.700
2 000
3,059 10:
186586 2on
8.081
8,361 L1.00
1.270
10100
3.05
ST-SE/3/TR
7.177 con
g.700
8.769 1.70926.0m
1.46320.om o00
3.428.04m
2.869
2 28872000s
STEGNNN
4.387
1. 111221
1.161gn
2.1 2.172
1.097
2:559
1 1.67320.15
2. 2.127
EgMation
7.86920..om
2. 064
2.975enou
2.48920..00
6.24820012
4 4.80820001
6.420
STOCN
7,628440.00 con
200
2.957
20.05
5.616
3.630gc.co cas
6:258220000
4 48800
6.35420004
AGI STAN
634 6600699 ato
6.028422081
271
29 2000
3. 3.798 20.41
8.418-22ss
10006--0
ESTAG
1.41620.225
0. 0000
2.898ga00s
1.112
0.801
1.05800.0
0.7901g0.u
1006
NS-EGNN
0.147 0.833g0.287
0 0..90.0.000
00,05
0.508_..149
0.518
0.3
0.719,0011
datasets. For the MD17 dataset, molecules such as Benzene exhibit higher p-values and positive ADF statisties, reflecting pronounced non-stationary behavior. The CMU Motion Capture dataset's walk and run subsets show high p-values and positive ADF statistics, confirming their -tatiitio nature. Lastly, the AdK dataset demonstrates substantial variability in its temporal dynamics, with approximately half of its nodes classified as non-stationary.
4.1.2 Baselines
We compare the performance of our proposed model with several widely used baselines in spatio- temporal trajectory modeling. STGCN (53) adopts a spatio-temporal convolutional architecture and is adjusted to predict residual coordinates between frames rather than absolute positions, directly predicting the latter often leads to suboptimal performance. AGL-STAN 42) which combines adaptive graph learning with self-attention mechanisms, is modified to handle weighted temporal ag- gregation to better capture intra-temport temporal dependencies. ST-GNN [14, ST-SE(3)-T (aSEE(3)TTaassfomeee [12]. denoted as ST-SE(3)RR, ST-TEN [43), and ST-EGNN (39] are included as GNN baselines. Except for ST-GNN, which is based solely on the message passing framework, other approaches leverage rotational and translational invariance for trajectory prediction. Another representative model, EqMo- tion (88.. integrates spatio-tempora information using attention- fusion for modeling the object dynamics. ESTAG 447 first models the non-Markovian nature of physical dynamics and proposes an equivariant temporal attention module to capture the latent interaction. All baselines are modified process the full historical trajectory (e.g., using linear encoders or, in the case of ESTAG, its native temporal attention) to ensure a fair comparison against our multi-step input model.
4.2 Molecular Dynamics
Setting. We evaluate the performance of our proposed model on the MD17 dataset, which includes molecular trajectories generated by MD simulation. The length of the input time series is set to 100, predicting the next 10 timesteps, with A = 5, as time series requires more timesteps to observe non-stationarity. We also conduct the experiments with fewer timesteps in Appendix C.5) aligning with the setting in ESTAG, which also has satisfactory performance improvement. The dataset is split into training, validation, and testing sets with ratios of 0.2, 0.4, and 0.4, respectively.
Evaluation Metrics. We use two standard evaluation metrics: 1) Average Displacement Error (ADE), which measures the average & distance between the predicted and ground truth molecular trajectories over all timesteps. 2) Final Displacement Error (FDE), which evaluates the l distance between the predicted and ground truth positions at the final predicted step.
Results. Tables 3nd summarize the ADE and FDE across all models. Notably, NS-EGNN emerges as the most effective model, surpassing bascline models considerably in both ADE and FDE metrics.
7

---

Table 4: Prediction error (MSE) for Walk (x10-1) and Run (x10%) cases under different time
intervals (5ts, 10ts. 15ts, 20ts). The reported mean and standard deviation computed over 5 runs.
Dataset
Walk ox10r))
Run (x
Thoe
Sth
18t
201a
Average
Stw
10t=
20t
Average
STGNNN
20155..475
u..00
335
1 106
1.538 ac.as
1.779 .n 275
1.250 Locoss
ST-TFN
1,520
6.800
0.786 c
1.708
2,08
ST-SEIS/T
0.570
0.760
a.319
0.600
asass
axa
0.
ST-EGNN
0.188 24.25
2.0877.0.00
0.977
0.464
1,082
2.876ax.a0
3.784_.
1..21
EaMatinz
21,076120r3
15,200
21,074.2.m3
18.634.2.90
10.018.2244
STOCN
0.307
0.878 1..00
1,516
1,585
1 1
6:151 Loaaes
0.582.. 323
1.
1.508
0.851 10.003
AGL-STAN
1,728
1.78
2,050
2. 2.165
1.926
0.511
0.625_8
0.618 asuze
0 0.831 10.360
0.051
ESTAG
0.051
0.215
0.530
1/85
0.47]
0.041 scoass
0,250
0.771
2.707
0. 207
NS-EGNN
0.051
0.100
0.307.0..57
0.775.0.065
0.347
0.033...003
0.187.0.000
0. 0.084_
1,228 215
0.508
Walk ESTAG
Walk- NS-EGNN
Run- ESTAG
Run NS-EGNN
GT Prodiction
GT Prodiction
GT Prodiction
GT Prodiction
20
10
10
10
10
11
-14 -5
-14
10
10
Figure 3: Visualization of predicted trajectories for rum and walk motions time gap 10 and 20. Ground truth is represented in red, and predictions are in blue.
Specifically, compared with the current ofate-of-the-art model, NS-EGNN achieves an impressive relative improvement of 29. 79% on ADE and 33.33% on FDE. Particularly with complex molecules like Aspirin and Malonaldehyde, NS-EGNN delivers an even more pronounced enhancement, reach- ing a 42.74% relative improvement for Aspirin. The second-best algorithm, which also utilizes a spatio-temporal approach to capture physical dynamics, underscores the importance of leveraging his- torical trajectory data. Nevertheless, NS-EGNN excels by explicitly modeling the dynamic statistical properties of physical objects, thereby achieving the best performance. Baseline models like STGCN and AGL-STAN exhibit high errors, as they are not equivariant. AGL-STAN's particularly poor performance, consistent with findings in other works [47, due to its non- -quivvarai architecture struggling to model symmetric physical dynamics. Equivariant baselines such as ST-SE(3)TR and ST-EGNN perform better but are less robust in capturing the dynamic behaviors of larger molecules.
4.3 Motion Capture
Setting. CMU Motion Capture Database involves the trajectories of human motion under various scenarios. For this experiment, we focus on the walk and rwn motions, selecting trajectories with sufficient length. The input sequence consists of the past 10 frames used to predict the subsequent frame, with data frame interval of At 1. Additionally, gap, representing the delay between the last observed frame and the target frame to be predicted. Experiments are conducted with four values of time gap: 5ts, 10ts, 15ts, and 20ts. In run dataset, we additionally normalize the dataset, following previous works [47]
introduce hyperparameter time
Results. Table presents MSE across different models for both walk and run motions, indicating NS-EGNN consistently achieves the best performance in most settings. Specifically, NS-EGNN achieves an average improvement of 21.52% % in walk and 15.29% in run, leading to an overall relative gain of 18.41 % across the entire dataset. This highlights the model's ability to capture the temporal and spatial dependencies crucial for human motion prediction. While AGL-STAN achieves the best performance in one case, NS-EGNN also remains highly competitive in that case. Overall, NS-EGNN demonstrates strong performance in both short-term and long-term predictions across different motions. Given that the motion dataset exhibits strong non-stationarity, it is not surprising that NS-EGNN achieves superior performance.
Visualization. Additionally, we provide visualizations of the predicted trajectories compared to the ground truth for both motions at time gap equals 10ts and 20ts, respectively. The visualiza- tions, shown in 3 include predictions from both ESTAG and our NS-EGNN. As illustrated, NS-EGNN predictions align more closely with the ground truth trajectories compared to ESTAG, particularly for complex joint movements. In the walk scenario, NS-EGNN exhibits smoother and more stable predictions, with trajectory paths closely following the ground truth points. For the run
8

---

motion, ESTAG struggles with maintaining coherence in the limb trajectories, resulting in noticeable distortions and erratic movements. In contrast, NS-EGNN better preserves the structural integrity of the motion, particularly in the torso and overall body posture, leading to a more realistic and physically plausible trajectory.
Table 5: Mean Squared Error (MSE) on the AdK dataset for protein dynamics prediction. The reported mean MSE values are computed over 5 runs. Bold font indicates the best result.
Model
STGEN
ST-NNN
ST-EGNN
ST-GMN
AGL-STAN
ESTAG
NS-EGNN
MSE
3.007
2.267
1,751
1,743
18833
1.758
1.738
4.4 Protein Dynamics
Setting. We evaluate our model on the AdK protein dynamics dataset, which involves predicting protein motions. For this dataset, we disable NS- NS-pooling because it tends to overfit to noise. The input sequence consists of 60 timesteps used to predict the subsequent 10 timesteps, with At 5. The dataset is split into training, validation, and testing sets with ratios of 0.6, 0.2, and 0.2, respectively. For consistency, all models are configured with 4 layers.
Results. Table presents the MSE of all models on the AdK dataset. While the overall improvement margin is less pronounced than in previous experiments, NS-EGNN still achieves the best performance with an MSE of 1. 1.738. This aligns with our findings from the ADF test (Table where the AdK dataset exhibits a smaller proportion of non-stationary nodes compared to other datasets. As a result, the impact of explicitly addressing non-stationarity is naturally less significant. This further demonstrates NS-EGNN's generalization ability, allowing make accurate predictions even in datasets where non-stationary effects are less dominant.
Table 6: Ablation study results on the MD17 dataset. The table reports ADE values (x 10 averaged over three runs. Lower values indicate better performance.
Method
Aspirin
Benzene
Ethanol
Malonaldehyde
Naphthalene
Salicylic
Toluene
Uracil
Average
NS-EGNN
0.421
0.050
0.407
0.600
0.270
0.387
0.308
0.379
0.353
w/ Attention
0.499
0.039
0.389
0.751
0,280
0.446
0.317
0.377
0.387
w/o Patched FT
0.948
0.076
0.547
0.876
0,360
0.499
0.431
0.504
0.530
w/o Differentiation
2.622
0.265
0.700
1.929
0,949
2.249
1.103
1.448
1.408
w/o Equivariance
225.608
655.109
220.134
871.211
1365.762
10.397
281.836
7,861
454,740
4.5 The Effectiveness of Each Component in NS-EGNN
To evaluate the contributions of key components in NS-EGNN, we conduct experiments including removing specific modules to analyze their impact, and results are in Table From the table, we derive the following conclusions: 1) PFT extracts expressive temporal features. Prior work [47] introduced an equivariant temporal attention mechanism for learning physical dynamics. To explore potential improvements, we also incorporate this module. However, the overall performance shows slight decline, suggesting that PFT already generates sufficiently expressive temporal features. 2) PFT is capable of capturing the dynamic statistic property in the non-stationary trajectory. We replace the PFT with a standard Fourier Transform applied to the entire trajectory. The observed worse performance suggests that PFT plays critical role in modeling the temporal variance of non-stationary trajectories. This highlights the necessity of preserving localized frequency patterns. 3) Differentiation greatly reduces the negative impact of linear and quadric trend in the raw physical dynamics. The removal of this module results in a noticeable decline in performance, underscoring the critical role of differentiation in addressing mean shifts within non-so con-staiionary dynamics. Furthermore, without this component, the model loses crucial equivariant property. leading to suboptimal predictions. 4) Equivariant backbone enhances the generalization ability in dynamic simulation. The standard message passing GNN struggles to capture the symmetrical properties inherent in Fuclidean space, leading to significantly poor performance. We further provide the NS-EGNN removing Fourier Transform entirely in Appendix C.4

---

Related Works
Graph Neural Networks for Geometric Trajectory. TFN 43 and
[2 employ
spherical harmonics to extract high-order geometric representations. To reduce the computation in high-order representations, EGNN [39 updates invariant messages using relative distances and then derives directional vectors from these messages. For specific scenarios, many variants of EGNN 17
have been proposed, such SGNN (17), GMN [19]. DEGNN [58], EGHN [88] and EGNO 44)
EqMotion [48] and ESTAG 477 propose extracting the invariant and equivariant geometric features from historical dynamics, capturing temporal dependencies. However, these methods overlook the non-stationarity of physical dynamics, leading to suboptimal performance.
Non-stationarity in Time Series Forecasting, Before deep learning models, the classic ARIMA [ 2] algorithm addressed non-stationarity by differencing the time series. The varying distribution of non-stationary data presents additional challenges for deep models. Pre-processing offers straightforward and effective method to make the time series stationary. Adaptive Norm 33
normalizes fragment of the series according to the global statistics. Moreover, DAIN [3), ReviN 20)
SAN [29]. FAN [52]. DDN 9 and IN-Flow [T]] introduce learnable neural networks to normalize
time series data. Furthermore, Transformers [28
23
1
45
44
56
55 25 have been utilized to
model non-stationary series with specialized attention mechanisms. However, direct stationarization
of physical object coordinates of physical objects will unavoidably break the symmetric property of the deep models. To bridge this gap, -SSENGNN captures dynamic distributions while maintaining cquivariance.
6 Conclusion
We show that trajectories physical dynamics are highly non-stationary. Our NS-EGNN framework leverages the PFT and NS-Pooling modules to capture these patterns. By modeling non-stationary physical dynamics with equivariance, NS-EGNN greatly outperform the SOTA methods. A limitation is diminished gains on stationary data, where repeated FT increases computation with marginal benefits. Future work could extend NS-EGNN to domains to various settings and explore more spatial backbones (50) according the specific scenarios. Furthermore, other methods for modeling non-stationary data, such as wavelet transforms, also break the equivariance. However, the adaption of these advanced method into equivariance also worth exploration.
Acknowledgments
This work is supported by Shenzhen Science and Technology Innovation Commission under Grant JCY120220530
Tsinghua Shenzhen International Graduate School Start-up fund under Grant QD2022024C, Shenzhen Ubiquitous Data Enabling Key Lab under Grant ZDSYS202205271 1172271111000 171466015 Damo Academy (Hupan Laboratory) through Damo Academy (Hupan Laboratory) Innovative Research Program, Damo Academy through Damo Academy Research Intern Program, Research Grants Council of the Hong Kong Special Administrative Region, China (No. CUHK 14217622), and Guangzhou Industrial Information and Intelligent Key Laboratory Project (No. 2024A03J0628). Additionally, Chaohao Yuan would like to thank his fiancée, Siying Xu, for her companionship and great help in visualization (Figure 2) in this work.
References
[1] Bissan Al-Lazikani, Joon Jung, Zhexin Xiang, and Barry Honig. Protein structure prediction. Current opinion in chemical biology, 5(1):51-56, 2001.
[2] George EP Box and Gwilym M Jenkins. Some recent advances in forecasting and control. Journal of the Royal Statistical Society. Series C (Applied Statisties), 17(2):91-109, 1968.
[3] George EP Box, Gwilym M Jenkins, Gregory C Reinsel, and Greta M Ljung. Time series analysis: forecasting and control. John Wiley Sons, 2015.
[4] Jiacheng Cen, Anyi Li, Ning Lin, Yuxiang Ren, Zihe Wang, and Wenbing Huang. Are high- degree representations really unnecessary in equivariant graph neural networks? NeurIPS, 37:26238 262666 2024.
10

---

[5] Jiacheng Cen, Anyi Li, Ning Lin, Tingyang Xu, Yu Rong, Deli Zhao, Zihe Wang. and Wenbing Huang. Universally invariant learning in equivariant gnns, arXiv preprint arXiv:2510.1319. 2025.
[6] Stefan Chmiela, Alexandre Tkatchenko, Huziel E Sauceda, Igor Poltavsky, Kristof T Schütt, and Klaus-Robert Müller. Machine learning of accurate energy-conserving molecular force fields. Science advances, 3(5):e663305,, 2017.
[7] CMU. Carnegie-mellon motion capture database, 2003, URL Lnttpp:////oc cmu edu,
[8] Javier Contreras, Rosario Espinola, Francisco J Nogales, and Antonio Conejo. Arima models to predict next-day electricity prices. IEEE transactions on power systems, 18(3):0014-100 2003.
[9] Tao Dai, Beiliang Wu, Peiyuan Liu, Naiqi Li, Xue Yuerong. Shu-Tao Xia, and Zexuan Zhu. Ddn: Dual-domain dynamic normalization for non-stationary time series forecasting, Advances in Neural Information Processing Systems, 37:108490-108517, 2024.
[10] Graham Elliott, Thomas Rothenberg, and James H Stock. Efficient tests for an autoregressive unit root, 1992.
[1]] Wei Fan, Shun Zheng, Pengyang Wang, Rui Xie, Kun Yi, Qi Zhang, Jiang Bian, and Yanjie Fu. In-flow: Instance normalization flow for non-stationary time series forecasting, In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 1, pages 295-306, 2025.
[12] Fabian Fuchs, Daniel Worrall, Volker Fischer, and Max Welling. Se (3)-transformers: 3d roto-translation equivariant attention networks. Advances in neural information processing systems, 33:1970-1981, 2020.
[13] Zidi Gao and Ercan Engin Kuruoglu. Attention based hybrid parametric and neural network models for non-stationary time series prediction. Expert Systems, 13419, 2024.
[14] Justin Gilmer, SamuelS, Schoenholz, Patrick Riley, Oriol Vinyals, and GeorgeE. Dahl. Neural message passing for quantum chemistry, arXiv: Learning, aivv: Learning, Apr 2017.
[15] Daniel Griffin and Jae Lim. Signal estimation from modified short-time fourier transform. IEEE Transactions on acoustics, speech, and signal processing. (2(2):236-243, 1984.
[16] Jiaqi Han, Jiacheng Cen, Liming Wu, Zongzhao Li, Xiangzhe Kong, Rui Jiao, Ziyang Yu, Tingyang Xu, Fandi Wu, Zihe Wang, et al. A survey of geometric graph neural networks: Data structures, models and applications. arXiv preprint arXiv:2403.00483 2024.
[17] Jiaqi Han, Wenbing Huang, Hengbo Ma, Jiachen Li, Josh Tenenbaum, and Chuang Gan. Learning physical dynamics with subequivariant graph neural networks. Advances in Neural Information Processing Systems, 35:26256-26268 2022.
[18] Jiaqi Han, Wenbing Huang, Tingyang Xu, and Yu Rong. Equivariant graph hierarchy- neural networks. Advances in Neural Information Processing Systems, 35:9176-99 77879187,11 2022.
[19] Wenbing Huang, Jiaqi Han, Yu Rong. Tingyang Xu, Fuchun Sun, and Junzhou Huang, Equiv- ariant graph mechanics networks with constraints. In International Conference on Learning Representations, 2022.
[20] Taesung Kim, Jinhee Kim, Yunwon Tae, Cheonbok Park, Jang-Ho Choi, and Jaegul Choo. Reversible instance normalization for accurate time-series forecasting against distribution shift. In International Conference Learning Representations, 2022.
[21] Walter Kohn, Axel D Becke, and Robert Parr. Density functional theory of electronic structure. The journal of physical chemistry, (09431):12974-12980 1996.
[22] Zongzhao Li, Jiacheng Cen, Bing Su, Wenbing Huang, Tingyang Xu, Yu Rong, and Deli Zhao. Large anguagy-geometry model: When Imm meets equivariance. arXiv preprint 2rXiv:2502.11149 2025.

---

[23] Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X. Liu, and Schahram Dustdar. Pyraformer: Lompceepyy pyramidal attention for long-range time series modeling and forecasting, In International Conference on Learning Representations, 2022.
[24] Yang Liu, Jiashun Cheng, Haihong Zhao, Tingyang Xu, Peilin Zhao, Fugee Tsung, Jia Li, and Yu Rong. SEGNO: Generalizing equivariant graph neural networks with physical inductive biases. In The Twelfth International Conference Learning Representations, 2024.
[25] Yang Liu, Zinan Zheng, Jiashun Cheng, Fugee Tsung, Deli Zhao, Yu Rong, and Jia Li. Cirt: Global subseasonal-to-Sees -t-sssonaal forecasting with geometry-inspiret transformer. In ICLR, 2025.
[26] Yang Liu, Zinan Zheng, Yu Rong, and Jia Li. Equivariant graph learning for high-density crowd trajectories modeling. Transactions on Machine Learning Research, 2024.
[27] Yang Liu, Zinan Zheng. Yu Rong, Deli Zhao, Hong Cheng, and Jia Li. Equivariant and invariant message passing for global 2025.
-o-sasoona forecasting, In KDD, pages 1879-1890,
[28] Yong Liu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Non-stationary transformers: Exploring the stationarity in time series forecasting. Advances in Neural Information Processing Systems, 95:9881-9892 2022.
[29] Zhiding Liu, Mingyue Cheng, Zhi Li, Zhenya Huang. Qi Liu, Yanhu Xie, and Enhong Chen. Adaptive normalization for non-stationary time series forecasting: A temporal slice perspective. Advances in Neural Information Processing Systems, 36: 14273-14292, 2023.
[30] Zhiding Liu, Mingyue Cheng, Zhi Li, Zhenya Huang, Qi Liu, Yanhu Xie, and Enhong Chen. Adaptive normalization for non-stationary time series forecasting: A temporal slice perspective. Advances in Neural Information Processing Systems, 36, 2024.
[31] Ziqiao Meng, Liang Zeng, Zixing Song, Tingyang Xu, Peilin Zhao, and Irwin King. Towards geometric normalization techniques in (3) equivariant graph neural networks for physical dynamics simulations. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, pages 5981-5989, 2024.
[32] Brian K Nelson. Time series analysis using autoregressive integrated moving average (arima) models, Academic emergency medicine, 5(7):739-744, 1998.
[33] Eduardo Ogasawara, Leonardo C Martinez, Daniel De Oliveira, Geraldo Zimbrão. Gisele L Pappa, and Marta Mattoso. Adaptive normalization: A novel data normalization approach for non-stationary time series. In The 2010 International Joint Conference on Neural Networks (IJCNN), pages 1-8. IEEF, 2010.
[34] Nikolaos Passalis, Anastasios Tefas, Juho Kanniainen, Moncef Gabbouj, and Alexandros losifidis. Deep adaptive input normalization for time series forecasting. IEFE transactions on neural networks and learning systems, 31(9):37003365, 2019.
[35] KM Muraleedhara Prabhu. Window functionn and their applications in signal processing. Taylor & Francis, 2014.
[36] Dayu Qin, Yi Yan, and Ercan Engin Kuruoglu. Llm-based online prediction of time-varying graph signals (student abstract). In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 29472-29474, 2025.
[37] A Srinivas Reddy, S Priyadarshini Pati, Praveen Kumar, HN Pradeep, and G Narahari Sastry. Virtual screening in drug discovery-a computational perspective. Current Protein and Peptide Science, 844::329-351, 2007.
[38] Alvaro Sanchez-Gonzaler Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovec, and Peter Battaglia. Learning to simulate complex physics with graph networks. In International conference on machine learning, pages 8459-8866 PMLR, 2020.
[39] Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling. E (n) equivariant graph neural networks. In International conference on machine learning, pages 9323-9332. PMLR, 2021.
12

---

[40] Sean Seyler and Oliver Beckstein. Molecular dynamics trajectory for benchmarking MDAnaly- sis. 2017.
[41] Ian Naismith Sneddon. Fourier transforms. Courier Corporation, 1995.
[42] Mingjie Sun, Pengyuan Zhou, Hui Tian, Yong Liao, and Haiyong Xie. Spatial-Tempore Attention Network for Crime Prediction with Adaptive Graph Learning, page 656-669. Jan 2022.
[43] Nathaniel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang, Li Li, Kai Kohlhoff, and Patrick Riley. Tensor field networks: Rotation-and translation-cquivrrian neural networks for 3d point clouds. arXiv preprint arXiv: 022./022/22 19, 2018.
[44] Jiashan Wan, Na Xia, Yutao Yin, Xulei Pan, Jin Hu, and Jun Yi. Tedformer: A transformer framework for non-stationary time series forecasting based trend and change-point detection. Neural Networks, 173:106196, 2024.
[45] Muyao Wang, Wenchao Chen, and Bo Chen. Considering nonstationary within multivariate time series with variational hierarchical transformer for forecasting. In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pages 15563-15570, 2024.
[46] Annette Witt, Jürgen Kurths, and A Pikovsky. Testing stationarity in time series, physical Review E, 58(2):1800, 1998.
[47] Liming Wu, Zhichao Hou, Jirui Yuan, Yu Rong, and Wenbing Huang. Equivariant spatio- temporal attentive graph networks to simulate physical dynamics, Advances in Neural Informa- tion Processing Systems, 36, 2024.
[48] Chenxin Xu, Robby Tan, Yuhong Tan, Siheng Chen, Yu Guang Wang, Xinchao Wang, and Yanfeng Wang. Eqmotion: Equivariant multi-agent motion prediction with invariant interaction reasoning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14200, 2023.
[49] Minkai Xu, Jiaqi Han, Aaron Lou, Jean Kossaifi, Arvind Ramanathan, Kamyar Azizzadenesheli, Jure Leskovec, Stefano Ermon, and Anima Anandkumar. Equivariant graph neural operator for modeling 3d dynamics. In Proceedings of the 41st International Conference on Machine Learning, pages 55015-55032, 2024.
[50] Yi Yan and Ercan Engin Kuruoglu. Binarized simplicial convolutional neural networks. Neural Networks, 183:106928, 2025.
[51] Yi Yan, Changran Peng, and Erean E Kuruoglu. Graph signal adaptive message passing. IEEE Signal Processing Letters, 2025.
[52) Weiwei Ye, Songgaojun Deng, Qiaosha Zou, and Ning Gui. Frequency adaptive normalization for non-stationary time series forecasting. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
[53] Bing Yu, Haoteng Yin, and Zhanxing Zhu. Spatio-temporal graph convolutional networks: deep learning framework for traffic forecasting. In Proceedings of the 27th International Joint Conference Artificial Intelligence, pages 3634-3640, 2018.
[54] Chaohao Yuan, Songyou Li, Geyan Ye, Yikun Zhang, Long-Kat Mai Huang, Wenbing Huang, Wei Liu. Jianhua Yao, and Yu Rong. Annotation guided protein design with multi-level domain aligmment. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. I, pages 1855-1866, 2025.
[55] Chaohao Yuan, Tingyang Xu, and Yu Rong. Transformer and drug design. In Deep Learning in Drug Design, pages 93-108. Elsevier, 2026.
[56] Chaohao Yuan, Kangfei Zhao, Ercan Engin Kuruoglu, Liang Wang, Tingyang Xu, Wenbing Huang, Deli Zhao, Hong Cheng, and Yu Rong. A survey of graph transformers: Architectures, theories and applications. arXiv preprint 3xXh::2502.16533,20023
13

---

[57] Yuelin Zhang, Jiacheng Cen, Jiaqi Han, Zhiqiang Zhang, Jun Zhou, and Wenbing Huang. Improving equivariant graph neural networks on large geometric graphs via virtual nodes learning. In ICMI., 2024.
[58] Zinan Zheng, Yang Liu, Jia Li, Jianhua Yao, and Yu Rong. Relaxing continuous constraints of equivariant graph neural networks for broad physical dynamics learning. Proceedings of the 30th ACM SIGKDD Conference Knowledge Discovery and Data Mining, pages 4548 4558, 2024.
14

---

A Proofs
A.1 Proof of Lemma 3.1
Lemma 3.1 (). The extracted frequency crequeccy feature 8 is F(n)) Eina-innaaiant.
Proof. Recall Eq. 7
T
(FT(X.)(p.H)
Sp (X.(1) ((())((-kxxo)))
(15)
After E(3) transformation, orthogonal O combined with translation the trajectory, the updated Fourier frequency is denoted as S.:
S
kt
(OX, +t - (1, t) - h hxppw)
(16)
1=
1 (1) - (x hx p:w)
(17)
1=
T
0
F (X,(t)
(ttt))((t-h h p:w)
(18)
OS;
(19)
We denote the transformed frequency feature as s" with 00 I,
(18;2)/n
(20)
((Ss.12)/n
(21)
(22)
(23)
A.2 Proof of Theorem 3.2
Lemma A.1. Equivariant Message Passing is E(n)--quivvarinn.
Proof.
m ( fo( h ||( (1) +t) - - + +)
(24)
.n fo
(1) ||( -X(())1
(25)
= fo(hh h [ 113
(26)
+t= (1) +t+ 1 #
OX! (+11)
Σ (X (1)
- -x- tm
(1)
(27)
Wi
jEN.
15

---

iti
0 11)
( - -x, (mm 1) +
(28)
Ni
JEN
Lemma A.2. Equivariant Temporal Pooling is Eqnneeruiaariant.
Proof.
x = - A
(29)
With orthogonal transformation O and translation t in E(n) group.
OAX, (OX, +t (t- 0x - -(x - (t 1)}-
t)
(30)
(31)
OA = 014X, (8)
(32)
(OAX (8) OAX 1)
(33)
Then, have
OX +t= OOAX, OA 7 (T
(34)
0
Theorem 3.2 (.). For arbitrary orthogonal transformations and translation vectors O, E E(3),
fo((Og + t) = 0f
Proof. As shown in Lemma 3.1 Lemma A.1 and Lemma A.2 since extracted spectral feature is invariant, and the rest of two components, Equivariant Message Passing and Equivariant Temporal Pooling, are equivariant, the model, fo()). )), is equivariant as well.
B
Implementation Details
B.1 More Details on ADF Test
To provide more comprehensive view the Augmented Dickey-Fuller (ADF) test results, we present additional statistical measures that are omitted in the main paper for brevity.
To mitigate the influence of outliers, we exclude the top and bottom 3% of values when computing the mean and standard deviation of the ADF test metrics. For each node's spatial coordinates x, we determine the stationarity individually. A node classified as non-stationary if any of its x, y. or z dimensions fails the stationarity test. This ensures comprehensive evaluation of the node's 's temporal behavior across all spatial dimensions.
Table g provides detailed statistics for each dataset, including the total number of trajectories, the count of non-stationary trajectories, the mean and standard deviation of the p-value, well as the mean and standard deviation of the ADF statistic.
Table 7: Critical values for the ADF test across different datasets. Each column represents the 1%, 5%, and 10% significance thresholds.
Dataset
Mean 1%
Mean 5%
Mean 10%
MD17
-3.444
-2.868
-2.570
Motion Walk
-5,0538
33.2466
-2.858
Motion Run
-5,0896
-3,5391
-2,8633
AdK
-3.5521
-2.9144
-2.5949
16

---

Table lists the critical values used in the ADF test across different significance levels (1%, 5%, and 10%) for various datasets. If the ADF statistic time series is lower than the critical value at given significance level, the null hypothesis of non-stationarity is rejected, confirming stationarity.
Table 8: Detailed ADF test results for the MD17, CMU Motion Capture, and AdK datasets.
Each
row summarizes key statistical measures for different subsets.
Subset
Total Trajectories
Non-Stationary Trajectories
Mean p-value
Sid p-value
Mean ADF Statistic
Std ADF Statistic
MD17
Aspirin
6500
5441
0.6684
0.2641
-0.9185
L.0322
Benzene
3000
2999
0,9112
0.1251
0.0896
0.8995
Eshanol
1414
0.4841
0.3082
-1,4999
1,0187
Malonaldchyde
2500
2493
0.7741
0.2464
-0.4416
1.1369
Naphihalene
5000
3855
0.305
0.1944
-2.0771
0.5945
Salicylic
5000
4825
0.454
0.2541
-1.5994
0.7126
Toluene
3500
2744
0.2641
0.2343
-2.2781
0.8181
Uracil
1000
1657
0.3161
0.2293
-2.0881
0.7278
CMU Motion Capture Dataset
Basketball
74400
71674
0.8507
0.2811
2,0209
3.7188
Walk
71300
68452
0.835
0.2907
1.9866
3,7633
1,8811
3,2893
AdK Dataset
Adk
818133
401996
0.1761
0.3493
-3.0401
1,2173
The additional data in Table further supports the non-stationary characteristies observed in the datasets. In particular, the MDT7 dataset exhibits higher p-values in certain molecules (e.g., Benzene), indicating stronger non-stationary behavior. The CMU Motion dataset also demonstrates significant non-stationary characteristics, particularly in the baskethall and walking sequences, where the mean ADF statistics are relatively high. The AdK dataset shows a relatively low mean p-value, indicating weaker stationarity.
The combination of these statisties and the stationarity assessment approach provides robust framework for analyzing non-stationary characteristics in various datasets.
B.2 Hyperparameter Settings
This section presents the hyperparameter configurations used for training different datasets. Table 9 summarizes the settings for MD17, CMU Motion, and AdK Protein datasets. The same model architecture is used across all datasets, but specific training configurations, such as the number of epochs, learning rate, and dataset splits, are adjusted to suit the characteristics of each dataset.
Table 9 presents the hyperparameter settings for each dataset. The learning rate and weight decay values are chosen based on dataset characteristics, with AdK Protein requiring lower learning rate due to its complexity. The CMU Motion dataset contains different train-validation-tesl splits for walk and run, which are provided separately the table. The number of layers, hidden dimension, and weight decay remain consistent across all datasets.
B.3 Compute Resources
All experiments reported in this paper were dedicated rerforerformnnc server. The system is equipped with a single Intel@ Xcon@ Platinum 8358P CPU clocked at 2.60 GHz (32 cores, 64
Table 9: Hyperparameter configurations for different datasets. provided separately for walk and run.
The dataset splits for CMU Motion are
Hyperparameter
MD17
CMU Motion
AdK Protein
Lipochs
500
500
150 5 x 10-5
Learning Rate (Ir)
5 x 10-3
5 x 10-3
Weight Decay
1 x 10-12
1 x 10-12
1 x 10-12
Number of Layers
4
4
Frame
5
5
Hidden Dimension
16
16
Train/Val/Tes Split
(2:4:4)
walk: [22:12:12]
[6:2:2]
non: [5:4:2]
17

---

threads), 251 GiB of DDR4 main memory, and four NVIDIA H20 GPUs (each with 96 GiB of VRAM). We used NVIDIA driver version 550.120 and CUDA 12.4. This configuration remained constant across all training and evaluation runs.
Table 10: Compute resources used for all experiments.
Resource
Specification
CPU
Intel(R) Xeon(R) Platinum 8358P @ 2.60 GHz
socket, 32 cores/socket (64 threads)
Memory
251 GiB DDR4 RAM
GPUs
x NVIDIA H20
96 GiB HBM3 VRAM each, 4.0 TB/s bandwidth NVIDIA driver 555.50
Driver
CUDA
CUDA 12.4
C More Experimental Results
C.1 Experiments on Differential Orders
We further conduct the experiments incorporating each order differences in Table II] and higher order information may lead potential over-fitting in the deep model and result suboptimal performance. Hence, ES-NGNN, we only incorporate lst-order and 2nd-order differencing.
Table 11: The experiments on incorporating different orders while pooling.
Aspirin
Benzene
Ethanol
Malonaldehyde
Naphthalene
Salicylic
Toluene
Uracil
Average
None(ESTAG)
0.677
0.086
0.422
0.632
0.328
0.629
0.369
0.366
0.439
0.467
0.726
0.419
0.634
0.368
0.45
0.33
2nd
01.21
0.05
0.407
0.6
0.27
0.387
0.308
0.379
0.353
3rd
0,501
0.058
0.408
0.583
0.273
0,341
0.3
0.4
0.358
0.509
0.067
0.419
0.607
0.302
0.352
0.278
0.425
0.37
C.2 Ablations study that completely omitting Fourier Transform
Table 12: The ablation studies on removing PFT but keeping FT and completely omitting the FT.
Aspirin
Benzene
Ethanol
Malonaldehyde
Naphthalene
Salicylic
Toluene
Uracil
Average
NS NS-EGINN EGNN w/o PFT
0.421
0.05
0.407
0.6
0.27
0.387
0.308
0.379
0.353
0.948
0.076
0.547
0.876
0.360
0.499
0.431
0.504
0.530
ES-N w/o FT
0.564
0.062
0.436
0.619
0.385
0.425
0.358
0.467
0.480
We further conduct the ablation study that completely omitting the FT. The results are shown in Ta- ble [2] Surprisingly, we find totally remove FT even can outperform EGNN with FT, which indicates FT cannot accurately extract the intrinsic spectral information. The experiments demonstrates this inaccurate spectral feature also harms the convergence of the model.
C.3
Additional non-stationary and equivariant normalization baselines
C.4 Ablations study that completely omitting Fourier Transform
equivariant normalization
31
C.5
Experiments on Original Settings in ESTAG
We further present the results of the original setting of ESTAG in Table 14 NS-EGNN still outper- forms ESTAG in 7 out of cases under the settings in ESTAG, achieving 10.3% relative performance improvement in average.
18

---

Table 13: The ablation studies on removing PFT but keeping FT and completely omitting the FT.
Aspirin
Benseme
Ethanol
Malomaldehydee
Naphthulene
Salicylic
Toluene
Uracil
Average
NS-EGNN
0.421
0.05
0.407
0.27
0.387
0.308
0.379
0.353
BGNN w'normalization
4,286
1,238
1.298
4,661 853.422
1.084
1.824
0.762
1471
2,078
Non-stationary Transformer
773.835
351.856
1449.999
18.570
290.497
9.793
531,483
Table 14: The performance (MSE) of NS-EGNN in the setting of ESTAG.
Aspirin
Benzene
Ethanol
Malonaldchyde
Naphthalene
Salicylic
Toluene
Uracil
Average
ESTAG NS-EGNN
0.063
0.003
0.099 0.097
0.101 0.100
0.068 0.059
0.047 0.057
0.079
0.066 0.058
0.068
0.052
0.003
0.065
0.061
C.6 Sensitivity of the Hyperparameters
Since the window length must be factor of the number of past frames, and the hop length is typically selected as half of the window length to effectively capture dynamic frequencies, we explored various combinations of hop lengths for NS-EGNN, which applies PFT multiple times with different window lengths. The combinations are outlined as in Table [S
Table 15: The ADE results NS- ES-EGNN with different set of hop length on MD17 dataset.
Hop
Aspirin
Benzene
Ethanol
Malonaldehyde
Naphthalene
Salicylic
Toluene
Uracil
Average
2, 5, 10]
0.497
0.044
0.378
0,585
0.261
0.37
0.286
0,346
0.346
5. 10, 20
0.421
0.05
0.407
0.6
0.27
0.387
0.308
0.379
0.353
10. 50]
0.446
0.049
0.392
0.627
0.35
0.458
0.329
0.345
0.374
As observed in Table 15 these hyperparameters do not significantly impact the overall model performance.
C.7 Complexity and Epoch Training Time
Theoretically, the attention mechanism requires O(N²) complexity, while the Fast Fourier Transform (FFT) only requires ONN log N) complexity. Experimentally, measured the average per-epoch training time (in milliseconds) of NS-EGNN and the baselines on the MD17 dataset across seven molecules. The results in Table 16 indicate that NS-EGNN is the most efficient algorithm compared with the baselines.
C.8 Sensitivity of Window Function
We further conduct the experiments on Blackman and Hann window functions on MD17 in Table 17] As observed in Table [7] NS-EGNN is insensitive the type of window function. Additionally, the worst performance the Table still outperforms all the baselines.
D Limitation
On the ADK dataset, method shows smaller improvements than MD17 and CMU Motion. This happens because our approach designed for non-stationary data and less effective when the data are more stationary.
E Broader Impacts
Our work contributes to more accurate and efficient dynamics simulations.
Broader impacts include:
Accelerating drug discovery: Faster, more accurate simulations can reduce time and cost in identifying candidate compounds.
19

---

Table 16: Average epoch training time (ms) on MD17.
Method
Aspirin
Benzene
Ethanol
Malonakdehyde
Naphthalene
Salicylic
Toluene
Uracil
Average
ESTAG
40.3
21.1
12.9
17.5
32.7
32.6
23.8
21.8
25.34
Eqmotion
159.0
157.9
17.2
142.5
160.3
160.3
161.6
120.8
134.95
AGL-STAN
299,4
305.
281.9
277.5
281.9
281.5
280.4
219.8
278.44
NS-EGNN
20.7
12.5
11.0
11.4
18.0
17.9
13.8
13.3
14.83
Table 17: The impact of different window functions MD17 dataset.
Window
Aspirin
Benzene
Ethanol
Malonaldehyde
Naphthalene
Salicylic
Toluene
Uracil
Average
Hamming
0.409
0.038
0.389
0.584
0.338
0.331
0.337
Blackman
0.445
0.046
0.389
0.59
0.244
0.365
0.281
0.374
0.342
Hann
0.421
0.05
0.407
0.6
0.27
0.387
0.308
0.379
0.353
Environmental chemistry: Improved modeling of reaction pathways may aid in designing greener catalysts and processes.
20

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide a short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include it (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked While "[Yes] " is generally preferable to "[No]" ", it is perfectly acceptable to answer "[No] providded a proper justification is given (e.g., "error bars expensive" or "we were unable to find the license for the dataset we used"). In general, answering binary way,
the checklist one of the factors in their evaluation.
reported because it would be too computationally
"[No] " or "[NA] is not grounds for rejection. While the questions are phrased acknowledge that the true answer often more nuanced, so please just use your best, judgment and write justification elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The Abstract and Introduction (Sec. D clearly enumerate the contribu- tions namely, explicit modeling of non-stationarity via Patch Fourier Transform, inte- gration into an Equiiaquaa GNN backbone, and extensive experiments validating performance which match the scope and results presented.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
21

---

Answer: [Yes]
Justification: See Appendices for limitation.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e-g., the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance the ap- proach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: All stated theoretical claims (Lemma 3.1 and Theorem 3.2 explicitly list
their Equivquuivee assumptions and are accompanied by full proofs Appendix A.] and
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theo- rems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be comple- mented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Answer: [Yes]
22

---

Section details datasets, splits, ADF test procedures, baselines, optimizer settings, and hyperpurameters: Appendix B.1 further describes implementation specifics and ADF-test preprocessing.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to hosted model (e.g., in the case of large language model), releasing of model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submissions provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily new algorithm, the paper should make clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., large language model), then there should either be a way to access this model for reproducing the results or way to reproduce the model (e.g.. with open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, may be that access the model limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Following NeurIPS instructions, we include our full codebase, detailed training and evaluation scripts, and step-by-step run instructions in the supplemental material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.. cc/ public/guides http://ooeSSSueiiieeeeeeee www.imissioooooooioooomo. for more details.
While we encourage the release code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g.. ., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines thttps://nips.. (tttpp://nipg..cpppbbccc wttpc//cppcccccieeeeeeeeeeeeeeee Sttee///oooSuummiiiiioooooo
for more details. The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
23

---

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) is recommended. but including URLs data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Section Hand Appendix B.2 enumerate data splits, batch sizes, learning rates, weight decay, number of epochs, optimizer choices, and layer configurations for each experiment.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Table and Table 4 report mean standard deviation over multiple runs, clearly indicating variability across seeds.
Justification: Results tables (c.g.. Table 2
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by error bars, con- fidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
24

---

Answer: [Yes]
Justification: See Appendices B.3 for compute-resource details and C.7 for execution times.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well as estimate the total compute.
The paper should disclose whether the full rescarch project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: httc:s/s/eaarrr...c//o/oal////be
Answer: [Yes]
Justification: We have reviewed and adhered to the NeurIPS Code of Ethics through- out ensuring anonymity, proper data handling, and no conflicts with ethical guidelines.
Guidelines:
The answer NA means that the authors have reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special considcration due to laws or regulations their jurisdiction..
Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We discuss potential benefits for drug discovery and green chemistry, as well minimal misuse risk, in Appendix E
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative socictal impacts include potential malicious or unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point out. For example, is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
25

---

11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: The work does not involve high-risk models or datasets requiring such safe- guards.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards allow for controlled requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
of the model, for example by
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification:
The paper does not use existing assets.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from particular source (e.g., website), the copyright and terms of service that source should be provided.
If assets are released, the license, copyright information, and terms of use in the pack- age should be provided. For popular datasets, paperrsi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the detaset/code/moodll part of their submissions via structured templates. This includes details about training, license, limitations, etc.
26

---

The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: This work involves only physical-dynamice datasets and no human-subject studies.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contri- bution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, cura- tion, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: No numan-subjedt research is conducted in this study.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMS if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification:
The proposed methodology does not involve any large-language models.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer LLM policy (https://neur http://wpp....oooooeeeeeee///000 for what should or should not be described.
27


Reviews:
[{"review_id": "heyZbbMNBl", "reviewer": "Reviewer_eQFU", "summary": "This paper introduces a Non-Stationary Equivariant Graph Neural Network (NS-EGNN) to improve the modeling of physical dynamics by addressing the challenge of non-stationarity while preserving symmetric inductive bias. Unlike existing equivariant graph neural networks that ignore the time-varying nature of physical systems, NS-EGNN incorporates Fourier Transform on trajectory segments to extract time-varying frequency features. It further reduces non-stationarity through first- and second-order differencing, followed by temporal pooling for prediction. By capturing dynamic frequency patterns and mitigating linear and quadratic trends in the data, NS-EGNN effectively models temporal dependencies. The method is evaluated on diverse physical dynamics tasks—including molecular motion, human motion, and protein dynamics—and demonstrates superior performance over existing state-of-the-art models.", "questions": "See Weaknesses.", "limitations": "YES", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n- 1.Frequency Feature with local windows intuitively helps capture key features while avoiding redundant information.\n- 2. The paper is clearly written and has strong reproducibility.\n\n**Weaknesses and Questions**\n- 1.The paper mentions that PFT is introduced because \"static frequency information is not sufficient.\" Why is the traditional DFT considered as \"static frequency,\" while PFT is regarded as \"dynamic frequency\"? Is there any experimental evidence proving that PFT performs better than DFT?\n- 2.What is the additional time consumption introduced by PFT?\n- 3.I would like to know the meaning of Equation (10). My understanding is that previous methods focused on predicting the conformation of the next frame, while your method additionally predicts the (approximate) second derivative and the conformation of the final frame. I would like to know the rationale behind this approach and whether the second derivative is used during inference. Predicting the final equilibrium conformation has also been proposed in many previous papers, such as eSCN [1] in their OC20 IS2RS [2] experiments. Should these works be discussed?\n\n[1] Reducing SO(3) Convolutions to SO(2) for Efficient Equivariant GNNs\n\n[2] The Open Catalyst 2020 (OC20) Dataset and Community Challenges", "was_revised": true, "final_justification": "This paper introduces Fourier features into equivariant dynamic transformations, which I believe is a rather scientific way of incorporating prior knowledge. During the rebuttal phase, the authors also addressed my concerns. I recommend a score of 4.", "rebuttal": "Thanks for your valuable feedback! Please kindly find the detailed responses below.\n\n**Q1.1: Why is the DFT considered as \"static frequency,\" while PFT is regarded as \"dynamic frequency\"?** \n\nThank you for raising this point. The distinction stems from their temporal resolution:\n- **DFT** computes the global frequency spectrum of an entire signal, implicitly assuming stationarity (hence \"static\").\n- **PFT** analyzes short overlapping segments, capturing time-localized frequency evolution (thus \"dynamic\").\nWe will clarify this terminology in Section 3.1 to avoid ambiguity.\n\n**Q1.2: Is there any experimental evidence proving that PFT performs better than DFT?**\n\nYes, in ablation study (Section 4.5), we conduct an experiment to replace PFT with DFT, and we found the average performance decreases, (Averaged prediction error from 0.353 to 0.530).\n\n**Q2: What is the additional time consumption introduced by PFT?**\n\nWe compared training times for 5 epochs across variants. The experiments are conducted on the machine introduced in Appendix B.3. As shown below, PFT adds minimal computational overhead:\n\n| |Aspirin|Benzene|Ethanol|Malonaldehyde|Naphthalene|Salicylic| Toluene|Uracil|Average|\n|-|-|-|-|-|-|-|-|-|-|\n|w/PFT|20.7|12.5|11.0|11.4|18.0|17.9|13.8|13.3|14.8|\n|w/o PFT|16.7|11.3|10.8|11.1|14.6|15.0|12.4|12.1|13.0|\n\n*Table: Training time (ms) per 5 epochs.*\n\n**Q3.1: The meaning of Equation (10)**\n\nWe apologize for any confusion. Equation (10) $\\vec{\\mathbf{X}}_{i}^{*} = [\\Delta \\vec{\\mathbf{X}}_i, \\Delta^2 \\vec{\\mathbf{X}}_i] \\cdot  \\mathbf{\\gamma} + \\vec{\\mathbf{X}}_{i}^{(L)}(T-1)$ integrates two components:\n1. **Stationarized features**: $\\Delta \\vec{\\mathbf{X}}_i$ (first-order differences from Eq. 9) and $\\Delta^2 \\vec{\\mathbf{X}}_i$ (second-order differences) are projected via a linear layer $\\mathbf{\\gamma}$.\n2. **Temporal context**: $\\vec{\\mathbf{X}}_{i}^{(L)}(T-1)$ denotes the EGNN module's final-layer output at $t=T-1$.\n\nThis approach reduces non-stationarity by predicting incremental changes rather than absolute states. For non-stationary data, these incremental values may be more predictable. For example, in molecular dynamics, the model can more effectively learn energy changes within the system than directly learn features from absolute energy values.\n\nWe will refine the explanation in Section 3.3.\n\n\n**Q3.2: Predicting the final equilibrium conformation has also been proposed in many previous papers, such as eSCN [1] in their OC20 IS2RS [2] experiments. Should these works be discussed?**\n\nWe appreciate this valuable suggestion. While eSCN advances equivariant representations, its primary focus on equilibrium conformation prediction makes direct adaptation to molecular dynamics simulations non-trivial. \n\n**Key Difference: Dynamics vs. Static Structures** - MD simulations require modeling molecular motion over time, whereas equilibrium predictions provide only static \"snapshot\" structures.\n\nAdditionally, eSCN analyzes individual frame conformations rather than *trajectory-based* forecasting. Since our baseline comparisons already include similar high-representation models (ST-TFN, ST-SE(3)-Transformer), we focused on directly comparable approaches. We will expand the discussion of conformation prediction's relation to our task in Section 5.\n\nWe sincerely appreciate your valuable suggestions. We will include above discussions into our revised manuscript to enhance clarity."}, {"review_id": "HEM4MnYx6C", "reviewer": "Reviewer_2Kiq", "summary": "This paper proposes Non-Stationary Equivariant Graph Neural Network (NS-EGNN) which uses patch Fourier transform to encode temporally varying features, equivariant GNN to encode spatial features, and temporal differences to predict non-stationary processes.  The method is evaluated on molecular dynamics, motion capture, and protein dynamics.", "questions": "- I'm not sure about the claim that \"most existing model focus on single-step frame-to-frame forecasting.\" In my experience (in fluids or robotics ,e.g.), many time series prediction method encode many past frames and predict many future frames.  Maybe this is just in MD?  \n- The paper makes the claim that \"normalization\" is the typical method for dealing with non-stationarity and that this is incompatible with equivariance.  Can you explain further?  Many normalization schemes are using for equivariant architechures, so maybe it is just naive normalization which is problematic? \n- Do the baselines encode multiple past time steps?  What about an ablation where the fourier transform is omitted completely and replaced with a encoder of enough past time steps to encode and track the non-stationarity?  \n- I am not clear how the NS-pooling operation works with the model.  I understand equation 10 as a mapping X^(L) to X^*.  From Eqn. 5, it seems this is being applied to the outputs of the EGNN layer.  Does this mean the EGNN outputs X^(L)  or does the EGNN output \\DetlaX and \\Delta^2 X.  If the former, it's not clear to me how this achieves the goal of making the NN prediction stationary. \n- On line 177, is the FFT O(N logN) ?", "limitations": "Yes.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 1, "significance": 2, "originality": 3, "strengths_and_weaknesses": "### Strengths \n- The paper addresses an important problem.  Non-stationary processes are commonly encountered in many time series.  Additionally many physical dynamics problems have symmetries which can be used to inform and improve prediction. \n- So far as I can determine the method is novel and appropriate to the problem.  Most other equivariant GNNs do not address non-stationarity and the experimental results back up the hypothesis that they will not perform that well out-of-the-box. \n- The datasets and baselines are reasonable and the model outperform the baselines. \n- The ablation shows the value of many of the design choices, but I would still like to see more ablations for example without the multi-scale and using no FT at all. \n\n### Weaknesses\n- The paper presentation is not great.  There are many spelling errors, missing words, and typos. The number is high enough that it is not just a matter of polish; the ability to understand the paper is compromised.\n- In particular, the description of the method in Section 3 is not very clear.  The indices and variable names are not clear.  For example in Eqn. 2, the index t doesn't appear in the variables. In Eqn. 3-5, \\vec{X} is redefined several times, making this hard to follow.  NS-Pooling is defined here, but not used in 3.1.2.  In 3.1.2 the notation \\vec{X}^* is used but not elsewhere, and so on. \n- While I found the method hard to understand, I also found it to involve several design choices for which I didn't understand the a priori motivation or see the empirical justification for.  For example, FT converts the signal to global frequencies which are by definition not temporally localized.  The PFT seems a bit ad hoc to me.  What is the performance of the method if we skip the FT entirely and just use encoded patches?  Could we use something temporally localized as well like wavelet transform?  I think the motivation to compute the norm in Eqn. 8 is to create an invariant feature to preserve equivariance in EGNN, but how much frequency information is preserved?  Could we use other norms directly on the original signal?  Is the Multi-scale PFT new or an existing idea? Is there theoretical support for multi-scale PFT? Does it correspond in some way to the scale-translation group? \n- It would be good to give some intuitive grounding the p-value and ADF Mean for example by computing them for some synthetic stochastic processes which can be visualized.", "was_revised": true, "final_justification": "I appreciate the author clarifications.  The rebuttal is quite good.  I particularly like the new ablations, clarifications on method, and explanations to help motivate the choices and novelty.  I am happy to raise my score.  (It's too bad you can't submit a revision -- it would be easier to evaluate clarity revisions.)", "rebuttal": "We sincerely thank the reviewer for the thorough evaluation and constructive feedback. We have carefully addressed each point below and will incorporate all changes in the final manuscript.\n\n**W1: The paper presentation is not great.**\n\nWe apologize for any typographical errors in the current draft. We will conduct thorough proofreading to enhance readability and ensure all notations are consistent in the revised version.\n\n**W2: The description of the method in Section 3 is not very clear**\n\nWe regret any confusion caused by the notation. To clarify:\n\n- Equation 2: $\\vec{X}$ will be updated to $\\vec{X}(t)$.\n- Equations 3-5 will be revised for consistency:\n\n$s = \\text{PFT}(\\{{\\vec{\\mathbf{X}}(t)}\\}_{t=0}^{T})$\n\n\n$(h, s, \\{\\vec{\\mathbf{X}}(t)^{(L)}\\}\\_{t=0}^{T}) = \\text{EGNN}(h, s, \\{\\vec{\\mathbf{X}}(t)\\}\\_{t=0}^{T})$\n\n$\\vec{\\mathbf{X}}^* = \\text{NS-Pooling}(\\{{\\vec{\\mathbf{X}}(t)}\\}_{t=0}^{T})$\n\nHere, ${\\mathbf{X}}(t)^{(L)}$ denotes the $L$-th EGNN layer output, and $\\vec{\\mathbf{X}}^*$ is the final pooled representation.\n- NS-Pooling remains conceptual as represented in Equation 10.\n- $\\vec{X}^*$ is further utilized to calculate the MSE loss (Section 3.1, line 135). \n\nWe are grateful for this feedback, which has strengthened our methodological exposition.\n\n**W3: The motivation of the design choices:**\n\nSince there are several sub-questions in this part. We carefully address each question individually. \n\n**W3.1 What is the performance if we skip the FT entirely and just use encoded patches?**\n\nWe further conduct the ablation study that completely omitting the FT. Please refer to our response of Q3.2 below.\n\n**W3.2 Could we use something temporally localized as well like wavelet transform?**\n\nNo, we cannot directly employ wavelet transform in our framework. While wavelet transforms are valuable for localized temporal analysis, they present challenges in equivariant settings:\n- The calculation of wavelet basis requires the operations of scaling and translation, disrupting the E(3) equivariance.\n- Subsampling in discrete wavelets breaks translation equivariance.\n\nOur PFT approach maintains strict E(3) equivariance required for physical dynamics. That said, we recognize wavelet transforms as a powerful alternative for non-stationary signal analysis. Future research directions could investigate modified wavelet constructions that maintain equivariance properties, and we will explicitly include this promising avenue in our future work discussion.\n\n**W3.3 How much frequency information is preserved?**\n\nPreservation depends critically on:\n1) Hop length reduction to increase window overlap [1]\n2) Diverse window lengths to capture multi-resolution spectra [2]\n\nPractically, high overlap combined with multi-scale sampling enables comprehensive frequency retention.\n\n**W3.4 Could we use other norms directly on the original signal?**\n\nYes, we include an equivariant normalization method as a baseline in Q2.\n\n**W3.5 Is the Multi-scale PFT new or an existing idea?**\n\nYes, multi-scale PFT is a new idea. While windowed Fourier analysis exists in signal processing field (e.g., wavelet transforms use a similar ideology), our contributions are:\n1) First adaptation to E(3)-equivariant machine learning scenarios for non-stationary dynamics  \n2) Providing the method to integrate PFT with geometric GNNs while keeping equivariance of the framework\n3) Flexible window-length mechanism surpassing wavelet constraints\n\n\n**W3.6 Is there theoretical support for multi-scale PFT?**\n\nRegarding the equivariance property of the multi-scale PFT, we provide a rigorous proof of Theorem 3.2. From the perspective from spectral retention, empirically, we observe that NS-EGNN with diverse window lengths and high overlap rates captures more comprehensive spectral features. However, while we can experimentally demonstrate multi-scale PFT's effectiveness, we currently lack a theoretical framework to quantitatively analyze spectral information retention.\n\n**W3.7 Does it correspond in some way to the scale-translation group?**\n\nNo, PFT does not demonstrate scale equivariance but E(3) equivariance (including translation). As established in Theorem 3.2, PFT is E(3)-equivariant (translation/rotation). Scale equivariance doesn't hold: scaling trajectories proportionally scales $s$, breaking EGNN equivariance. \n\nFrom an application perspective, we argue that NS-EGNN should not inherently possess scale equivariance. This is because changes in scale (manifested as altered edge lengths) typically correspond to meaningful physical property changes in the system.\n\n**W4: Visualize the meaning of p-value and ADF Mean**\n\nTo give a brief understanding of ADF test, we here provide two simplest examples to illustrate the calculation of ADF value and p-value. However, regretfully, since we are not able to upload image, we would include the illustration of example processes and detailed calculation in the Appendix.\n\n1. **Process Generation**:\n   - Stationary: $X_t = 0.6X_{t-1} + e_t, e_t \\in \\mathcal{N}(0, 1)$\n   - Non-stationary: $X_t = X_{t-1} + e_t, e_t \\in \\mathcal{N}(0, 1)$\n2. **ADF Test**:\n   - Regression: $\\Delta X_t = \\alpha + \\beta X_{t-1} + \\gamma \\Delta X_{t-1} + e_t$\n   - Test statistic: $\\text{ADF} = \\hat{\\beta}/\\text{SE}(\\hat{\\beta})$\n     - Stationary: $\\hat{\\beta} = -0.4 \\rightarrow \\text{ADF} = -4.3$\n     - Non-stationary: $\\hat{\\beta} = -0.02 \\rightarrow \\text{ADF} = -1.1$  \n3. **p-value Interpretation**:  \n   - Stationary: p = 0.001 (strong evidence against null)  \n   - Non-stationary: p = 0.72 (weak evidence against null)  \n\n**Q1: 'm not sure about the claim that \"most existing model focus on single-step frame-to-frame forecasting.\" In my experience (in fluids or robotics ,e.g.), many time series prediction method encode many past frames and predict many future frames. Maybe this is just in MD?**\n\nOur statement primarily refers to the specific context of geometric graph neural networks for physical dynamics simulation, where many existing approaches (particularly those cited in our related works section) do indeed focus on single-step prediction.\n\nWe will clarify the domain specificity in our revised claims.\n\n**Q2: Is normalization incompatible with equivariance?**\n\nIndeed, as demonstrated by recent work [3], naive normalization method is incompatible with equivariance. However, we also find a more advanced normalization method maintaining equivariance, and we have included it as our baseline [3] (without FT). The results are below:\n\n||Aspirin|Benzene|Ethanol|Malonaldehyde|Naphthalene|Salicylic|Toluene|Uracil|Average|\n|-|-|-|-|-|-|-|-|-|-|\n|NS-EGNN|0.421|0.050|0.407|0.600|0.270|0.387|0.308|0.379|0.353|\n|EGNN w/ normalization|4.286|1.238|1.298|4.661|1.084|1.824|0.762|1.471|2.078|\n\n\nThe results clearly demonstrate that normalization significantly degrades model performance (average error increases from 0.353 to 2.078). This performance deterioration stems from normalization's dual effect: while it helps stationarize the trajectories, it simultaneously removes critical temporal patterns that are essential for accurate predictions.\n\n\n**Q3.1: Do the baselines encode multiple past time steps**\n\nYes, ESTAG employs an attention layer to encode the multiple past time steps. Other baselines are not designed for time-series in their original paper, and these baselines are modified by encoding the past time steps by linear layers.\n\n**Q3.2: An ablation where the FT is omitted completely**\n\nWe further conduct the ablation study that completely omitting the FT. The results can be presented below: \n\n||Aspirin|Benzene|Ethanol|Malonaldehyde|Naphthalene|Salicylic|Toluene|Uracil|Average|\n|-|-|-|-|-|-|-|-|-|-|\n|NS-EGNN|0.421|0.050|0.407|0.600|0.270|0.387|0.308|0.379|0.353|\n|NS-EGNN w/ FT|0.948|0.076|0.547|0.876|0.360|0.499|0.431|0.504|0.530|\n|NS-EGNN w/o FT|0.564|0.619|0.436|0.619|0.385|0.425|0.358|0.467|0.480|\n\nSurprisingly, we find totally remove FT even can outperform EGNN with FT, which indicates FT cannot accurately extract the intrinsic spectral information. The experiments demonstrates this inaccurate spectral feature also harms the convergence of the model.\n\n**Q4: How the NS-pooling operation works with the model to make the output stationary?**\n\nWe apologize for any misundetstanding. We clarify:\n1. The output of EGNN is $\\{{\\vec{\\mathbf{X}}(t)}\\}_{t=0}^{T}$.\n2. NS-Pooling derives *stationarized* $\\Delta \\vec{\\mathbf{X}}$ and $\\Delta^2 \\vec{\\mathbf{X}}$ from the output of EGNN $\\{{\\vec{\\mathbf{X}}(t)}\\}_{t=0}^{T}$ (Equation 9).\n3. This stationarization improves NN generalization by providing NN more stationary features.\n\nPlease kindly note that our goal is stationarizing *inputs*, not outputs (predictions actually should be inherently non-stationary).\n\n\n**Q5: On line 177, is the FFT O(N logN)?**\n\nCorrect. We will update this in the manuscript. Thanks for pointing this typo.\n\n**Q6: Ablation of multi-scale PFT**\n\nWe employ reduce utilizing multiple windows to only utilize single window, the results can be found below:\n\n||Aspirin|Benzene|Ethanol|Malonaldehyde|Naphthalene|Salicylic|Toluene|Uracil|Average|\n|-|-|-|-|-|-|-|-|-|-|\n|NS-EGNN (multi-scale)|0.421|0.050|0.407|0.600|0.270|0.387|0.308|0.379|0.353|\n|NS-EGNN (single-scale) PFT|0.454|0.058|0.415|0.598|0.301|0.410|0.309|0.392|0.366|\n\nMulti-scale PFT consistently outperforms single-scale, validating our design.\n\nAll discussed modifications will be implemented in the revision. We thank the reviewer again for their insightful feedback that has significantly improved our work.\n\n[1]. Window and Overlap Processing Effects on Power Estimates from Spectra. 2000\n\n[2]. A Simple Adjustable Window Algorithm to Improve FFT Measurements. 2002\n\n[3]. Towards Geometric Normalization Techniques in SE(3) Equivariant Graph Neural Networks for Physical Dynamics Simulations. IJCAI 2024"}, {"review_id": "08a1Lfo2yL", "reviewer": "Reviewer_ZoES", "summary": "This paper proposes NS-EGNN, a Non-Stationary Equivariant Graph Neural Network, to better simulate physical dynamics. The core idea is to extract time-varying frequency information from segmented trajectories using a Patch Fourier Transform (PFT), followed by a non-stationary pooling layer employing first and second-order differencing for trend mitigation. The method is evaluated on molecular, motion capture, and protein dynamics datasets.", "questions": "The questions are listed above.", "limitations": "This paper does not explicitly discuss its limitations, such as the method's constraints, and focuses solely on analyzing the experimental results.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The paper clearly motivates the need to handle non-stationarity in physical dynamics, identifying specific limitations in existing equivariant GNN models.\n- The methodology is grounded in established time series and signal processing techniques (Fourier Transform, differencing) combined with equivariant learning.\n- The paper is organized logically, and clearly defines notation and distinctions.\n\nWeakness:\n- **Limited Conceptual Novelty**: The components leveraged (windowed Fourier Transform, differencing for non-stationarity, equivariant GNNs) are all well-known techniques in their respective domains. The main contribution is their integration and careful adaptation.\n- **Incomplete baselines**: lacks the comparison with existing Non-Stationary methods, such as PG-ODE and others. Additionally, there may be issues with the baseline implementation, as the AGL-STAN results appear to deviate significantly.", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your feedback and suggestions. We appreciate the opportunity to address your concerns. Below are our point-by-point responses:\n\n\n**W1: Limited Conceptual Novelty**\n\nOur work makes two key conceptual contributions:\n1. We identify and formally address the previously overlooked *non-stationarity* in physical dynamical systems;\n2. Due to the non-stationarity, a naive FT is not sufficient to capture the comprehensive frequency information. We introduce windowed FT to capture the dynamic frequency.\n3. We establish novel theoretical foundations (Section 3.1.3) for integrating windowed FT within equivariant frameworks, which is a non-trivial extension requiring rigorous proof.\n\nWe will further clarify these contributions in our manuscript revision.\n\n**W2.1: Incomplete baselines**\n\nWe appreciate this valuable suggestion. To the best of our knowledge, PG-ODE is not currently open-sourced (we would be happy to include it as a baseline if codes are available), which makes direct comparison challenging. Recognizing the importance of non-stationary baselines, we have included another well-known non-stationary method, Non-stationary Transformer [1], for comparison. The results demonstrate NS-EGNN's significant advantages over existing non-stationary approach:\n\n||Aspirin|Benzene|Ethanol|Malonaldehyde|Naphthalene|Salicylic|Toluene|Uracil|Average|\n|-|-|-|-|-|-|-|-|-|-|\n|Non-stationary Transformer|773.835|351.856|526.389|853.422|1449.999|18.570|290.497|9.793|531.483|\n|NS-EGNN|0.421|0.050|0.407|0.600|0.270|0.387|0.308|0.379|0.353|\n\n*Note: All values are MSE (lower is better)*\n\nThis performance gap occurs because standard normalization breaks physical symmetry priors, inducing severe overfitting. We will include this baseline and emphasize this analysis in our revision. Additionally, we would also be pleased to include additional baselines within this realm.\n\n**W2.2: Issue of AGL-STAN implemetation**\n\nWe confirm our AGL-STAN implementation was directly adapted from the official ESTAG repository without any modification. As noted in the ESTAG paper, AGL-STAN underperforms equivariant methods due to its non-equivariant architecture. The significant deviation in AGL-STAN's results arises because it lacks equivariance considerations, making it prone to overfitting in physical dynamics scenarios. We will clarify this methodological distinction in the Experiment section.\n\n**Limitation Discussion**\n\nThank you for highlighting this. We have discussed:\n(1) Broader limitations in Appendix D, and\n(2) NS-EGNN's specific constraints regarding complex graph relations (Section 3.2).\nIn revision, we will expand these discussions and explicitly note that advanced graph backbones may be required for highly intricate systems. Furthermore, we maintain that these limitations do not diminish our core contribution of addressing non-stationarity in physical dynamic systems.\n\nWe thank you again for your valuable insights. All suggested improvements will be reflected in the revised manuscript.\n\n[1]. Non-stationary Transformers: Exploring the Stationarity in Time Series Forecasting"}, {"review_id": "fP8tEGkZrf", "reviewer": "Reviewer_6j6b", "summary": "This paper introduces NS-EGNN, a non-stationary equivariant graph neural network designed for modeling physical dynamics. NS-EGNN leverages Fourier Transforms to extract time-varying frequency features and applies first- and second-order differencing to mitigate linear and quadratic trends in the data. A pooling mechanism is employed to support future prediction. The proposed framework demonstrates consistent improvements over state-of-the-art baselines across various physical domains, including molecular, motion, and protein dynamics.", "questions": "- In Section 2.2, why not directly set $n=3$?\n- What is difference between $s$, $\\vec{s}$ and $S$, and what are their corresponding dimensions? \n- In Equ (7), why does the overall time remain $T$ after applying the windowing function?\n- Please check line 142 and 149, both sentences appear to be incomplete.", "limitations": "- The methodology section lacks clarity, making it difficult to follow. A general rephrasing is recommended. In particular, symbol usage should be carefully reviewed for consistency (see questions above). It is unclear how the PFT module is integrated with the NS pooling module, and further clarification is needed. \n- An ablation study is necessary to validate the individual contributions of the PFT and NS pooling modules.\n- The reported experimental results for ESTAG are weaker than those in the original paper. The authors should clarify this discrepancy.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "- The proposed framework is conceptually novel and addresses an important challenge in modeling non-stationary physical dynamics.\n- This paper provides some theorical guarantees for the equivariance and Invariance of the extracted features, that is appreciated.", "was_revised": true, "final_justification": "This paper presents a novel method and provides sufficient experimental results to demonstrate its effectiveness. Additionally, the authors have committed to addressing the typos and inconsistency in symbol usage. Based on these considerations, I am raising my score to Borderline Accept.", "rebuttal": "Thank you for your thorough and constructive feedback. We appreciate your time and insights, and will address all points below. Corresponding revisions will be incorporated in the manuscript.\n\n**Q1: In Section 2.2, why not directly set n=3?**\n\nWe acknowledge that setting n=3 in Section 2.2 would be appropriate to maintain consistency with the E(3) group mentioned earlier. We will clarify this choice in the revised manuscript.\n\n**Q2: What is difference between $s$, $\\vec{s}$ and 𝑆, and what are their corresponding dimensions?**\n\nWe apologize for the notational confusion. For clarity:\n\n* $\\vec{s} \\in \\mathbb{R}^{n\\times T \\times 3}$: Frequency information extracted by DFT.\n* $S \\in \\mathbb{R}^{n\\times r \\times 3}$: Frequency information extracted by PFT.\n* $s \\in \\mathbb{R}^{n\\times r}$: Normalized version of $S$.\n\nWe realize it may be more appropriate to change $S$ as $\\vec{S}$ to avoid misunderstanding, and this will be reflected in our revised manuscript.\n\n**Q3: In Equ (7), why does the overall time remain 𝑇 after applying the windowing function?**\n\nIn signal processing, instead of directly cutting off the signal, windowing functions (e.g., Hamming, Hann) attenuate signal discontinuities at segment boundaries *without altering the temporal length* \\(T\\). This preserves the time dimension while minimizing spectral leakage, consistent with standard signal processing practice. We will add a brief clarification to the manuscript.\n\n**Q4: Please check line 142 and 149, both sentences appear to be incomplete.**\n\nThank you for catching these oversights: \n- Line 142: Corrected to \"$\\bar{\\vec{X}}_{i}(t)$ is the average 142 position of all nodes in the graph.\"\n- Line 149: Revised to \"However, directly cutting off parts of the trajectory can result in spectral leakage.\" \n\n**W1: The methodology section lacks clarity**\n\n- **Symbol usage should be carefully reviewed for consistency**\nWe will make the correponding changes mentioned above.\n\n- **How the PFT module is integrated with the NS pooling module**\nWe combine these modules by the flow of Equation 3-5. We first extract the invariant features from PFT, and using EGNN to update the coordinates $\\hat{X}$, then NS-Pooling is applied to make the final predication.\n\n\n**W1: The methodology section lacks clarity**\nWe will improve clarity by addressing both aspects raised:\n\n1. **Symbol consistency**: We will implement all notational adjustments discussed previously (particularly in Q2), ensuring uniform symbol usage throughout.\n2. **Module integration**: The PFT and NS-pooling modules connect through(Equation 3-5):\n   - PFT first extracts invariant features\n   - These features update coordinates ($\\hat{X}$) via EGNN\n   - NS-pooling then generates final predictions\nWe will add a clear pipeline description in Section 3.3 to explicitly articulate this workflow.\n\n\n**W2: An ablation study is necessary to validate the individual contributions of the PFT and NS pooling modules.**\n\nWe presented ablation studies in:\n\n- **Table 6 (PFT)**: Replacing PFT with standard DFT increases relative MSE by 50.14%, confirming PFT’s necessity.\n- **Appendix C.1, Table 11 (NS-pooling)**: Substituting ESTAG’s pooling (equivariant but stationary) improves relative MSE by 24.36%, validating the importance of NS-pooling.\n\nNotably, both ablated variants still outperform all baselines, underscoring the framework’s robustness.\n\n**W3: The reported experimental results for ESTAG are weaker than those in the original paper. The authors should clarify this discrepancy.**\n\nThe discrepancy of ESTAG stems from different experimental settings. Our main results use a **more challenging multi-step forecasting setting** than ESTAG’s original work. For direct comparison, we include results under identical settings in **Appendix C.2 (Table 12)**, where our implementation matches ESTAG’s reported performance. In this setting, NS-EGNN **still** outperforms ESTAG in 7 out of 8 cases, and has 10.3% relative performance improvement in average. We will emphasize this distinction in the revised version. \n\nWe sincerely appreciate your valuable suggestions, which have strengthened our work. All changes will be implemented in the revised manuscript."}]

Author rebuttal:

Meta-review: This paper proposes NS-EGNN, combining Patch Fourier Transform (PFT) with non-stationary pooling to address non-stationarity in physical dynamics while preserving E(3) equivariance. The method applies windowed Fourier transforms and differencing operations to extract time-varying frequency features for improved temporal modeling. The authors identify that existing equivariant GNNs inadequately handle non-stationary dynamics in physical systems. Their solution integrates signal processing techniques with geometric deep learning, providing theoretical guarantees for equivariance preservation. Experiments across molecular, motion, and protein dynamics show consistent improvements over baselines.

Strengths: Important problem addressing a gap in equivariant neural networks. Well-motivated approach supported by statistical analysis (ADF tests). Sound technical integration of established methods with novel adaptations for equivariant settings. Comprehensive experimental validation with thorough ablation studies demonstrating individual component contributions.

Weaknesses: Limited conceptual novelty since core components (windowed FT, differencing, equivariant GNNs) are well-established techniques. Initial presentation lacked clarity with inconsistent notation and incomplete explanations of module integration. Baseline comparisons could be more comprehensive for non-stationary methods.

Authors effectively addressed technical concerns with extensive additional experiments including PFT vs DFT comparisons, computational overhead analysis, and complete FT removal studies. Reviewers eQFU and 2Kiq raised scores after satisfactory responses. Reviewer ZoES maintained novelty concerns despite acknowledging improvements. Reviewer 6j6b appreciated clarifications but emphasized need for better initial presentation quality.

The paper makes a solid technical contribution by identifying and addressing an important gap in equivariant modeling. While individual components are known, their integration requires careful theoretical work to maintain equivariance properties. The experimental validation is thorough, and positive rebuttal responses with two score increases validate the merit. Despite novelty and presentation limitations, the technical soundness, problem importance, and comprehensive evaluation support acceptance.
Decision: Accept (poster)