Permutation Equivariant Neural Controlled Differential Equations for Dynamic Graph Representation Learning
Authors: Torben Berndt
Benjamin Walker
Tiexin Qin
Jan Stühmer
Andrey Kormilitzin
Venue: NeurIPS 2025 poster ()
Keywords: Temporal Graph Representation Learning
Neural Differential Equations
Equivariance Theory
Continuous Graph Neural Networks
Dynamical Systems
Primary area: general_machine_learning
TL;DR: We propose Permutation Equivariant Graph Neural CDEs, an equivariant and parameter-efficient extension of Graph Neural CDEs for dynamic graph representation learning.

Abstract: Dynamic graphs exhibit complex temporal dynamics due to the interplay between evolving node features and changing network structures. Recently, Graph Neural Controlled Differential Equations (Graph Neural CDEs) successfully adapted Neural CDEs from paths on Euclidean domains to paths on graph domains. Building on this foundation, we introduce \textit{Permutation Equivariant Graph Neural CDEs}, which project Graph Neural CDEs onto permutation equivariant function spaces. This significantly reduces the model's parameter count without compromising representational power, resulting in more efficient training and improved generalisation. We empirically demonstrate the advantages of our approach through experiments on simulated dynamical systems and real-world tasks, showing improved performance in both interpolation and extrapolation scenarios.

Permutation Equivariant Neural Controlled Differential Equations for Dynamic Graph Representation Learning
Torben Berndt >* Benjamin W Tiexin Qin
Jan Stühmer Andrey Kormilitzin
Heidelberg Institute for Theoretical Studies, Heidelberg, Germany Mathematical Institute. University of Oxford, UK City University of Hong Kong, Hong Kong *AR, Karlsruhe Institute of Technology, Karlsruhe, Germany Department of Psychiatry, Warneford Hospital, Oxford, UK
Abstract
Dynamic graphs exhibit complex temporal dynamics due to the interplay between evolving node features and changing network structures. Recently, Graph Neural Controlled Differential Equations (Graph Neural CDEs) successfully adapted Neu- ral CDEs from paths on Euclidean domains paths on graph domains. Building on this foundation, we introduce Permutation Equivariant Neural Graph CDEs, which project Graph Neural CDEs onto permutation equivariant function spaces. This significantly reduces the model's ' parameter count without compromising represen- tational power, resulting in more efficient training and improved generalisation. We empirically demonstrate the advantages of our approach through experiments on simulated dynamical systems and real-world tasks, showing improved performance in both interpolation and extrapolation scenarios.
Introduction
Graph Neural Networks (GNNs) [57 56 38 633 have emerged as a leading framework for modelling graph-structured data, demonstrating significant success in applications such protein folding [I], social recommender systems (2 or traffic forecasting 30 However, real-world graphs are often dynamic. Protein-proo protenni- interactions vary as relationships shift, and roads close due to building works and car crashes. Effectively capturing these temporal dynamics is crucial for accurate modelling and robust predictions.
time due to cellular processes, social networks evolve
For the last century, differential equations have been the cornerstone of modelling continuous change. However, in recent years, deep learning has revolutionised data analysis with its ability to learn complex patterns from vast amounts of data. While these approaches initially developed in parallel, the concept of Neural Differential Equations (NDEs) [46 52 has bridged the gap, demonstrating their interconnectedneess.
we propose Permutation Equivariant Neural Graph Controlled Differential Equations (PENG-CDES), novel framework which addresses key limitations of prior models Our primary contributions are as follows:
Building upon the approach introduced in 48
Work done in part while the University of Oxford.
2The source code is available at: htttpp///gttaaaccccttttmm//prppp
neural adesal-cdes
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Motivated by temporal and spatial symmetries, we derive PENG-CDEs from first principles. Our framework strikes balance between expressiveness and parameter efficiency (Section
We formalise this trade-off and prove that, under simplified assumptions, our proposed model is the optimal approximation to Graph Neural CDEs within the space of permutation-
equivariant functions (Section 3.2 Theorem B.I..
We further prove that the resulting models are equivariant under both time reparametrisations and permutations of the node set (Section 3.2 Proposition 3.2)
We empirically validate the effectiveness of PENG-CDEs on dynamic node- and graph-level tasks, consistently outperforming both differential equation-based and other spatio-temporal baselines. In particular, our model sets new faat--of-the-ar on the TGB- TGB-genre node affinity prediction task (Section 4
1.1 Related work
Continuous-depth GNNs. Recent work has linked GNNs to continuous-time dynamics on graphs
by interpreting hidden layers as discretised time axis. G study convolutional GNNs as
discretisations of heat diffusion processes on graphs, while [8 proposes architectures that incorporate
both parabolic and hyperbolic PDE terms. reaction-difffssion dynamics are explored in [12 17 and
advection terms are added in [9). Higher-order temporal derivatives are considered in [20 21.
proposes graph-based formulation of Neural ODEs 77. and Neural SDEs are extended to graph domains in [2]. While these models typically assume static graphs, some work addresses dynamic settings.
and [48] extend Neural CDEs to graph domains, by modelling node-level time series and graph structure hierarchically, and by treating the evolving topology
control, respectively.
Spatio-temporal GNNs. A well-established line of work in dynamic graph representation learning models spatial and temporal dynamics using separate modules for each. DCRNN [40] combines a diffusion-based GCN (8 with recurrent GRU (9] in an encoder-decoder architecture. STGCN
667 applies GCN [16] for spatial modelling and ID CNNs for temporal processing. Attention
extensions such as ASTGCN
and ASTGNN (27 introduce distinct spatial and temporal attention mechanisms. WaVeNet-GCN 42 fuses graph and temporal convolutions, while STIDGCN 4 uses
266
interactive learning handle heterogencous time scales.
Temporal Graph Networks. Another approach focuses on event- based message passing over time- stamped interactions. TGN (53) introduced this framework, later extended by TGNv2 [59] which considers an identification between source and target nodes. Node-centric methods such as DyRep (60) and TCL (65) compute embeddings from temporal neighbourhoods and aggregate across edges. In contrast, edge-centric methods like CAWN 666] and GraphMixer [14] embed edge interactions directly for prediction.
2 Background
2.1 (Temporal) Graph representation learning
We denote the set [1,..., n by [n] and the set of functions from X to Y by x(x Y).
Graphs. A graph G = (V, E) consists of collection of nodes V [n] and edges & C V. The graph topology is described by an adjacency matrix A E Ra*=, where is 1 if (i.j) E and 0 otherwise. The degree d, Ejev A of node is the number of edges incident on it. Let D
be the diagonal degree matrix with D = d, The graph Laplacian is defined as = D A and
the normalised graph Laplacian is defined as L = - AD where I, is the x identity matrix. We assume the nodes are equipped with d diddiiensi node features x; E Rds, stacked to
form node attribute matrix X I=' xxd, The goal of Graph Representation Learning is to leam latent node representation Z E gaxd, which embeds the nodes into some Euclidean space.
Temporal Graph Representation Learning. In this setting, we observe a sequence of graph snapshots G = ((o,Grr)) (to),.... (IN; Gtx)) generated by some unknown continuous-time underlying process. Each snapshot (tx, G.z) captures the graph state time tg, with Gt. (V, E.,) and

---

corresponding adjacency matrices As, E R The objective is to learn dynamic non- non-lineat linear latent representation Z, E Rxnx: for the nodes for all times t E (to, ty].
Equivariance. Let G be a group and X, Y be two sets with (left) G-action. A function f: X - Y is G-equivariant if fg 2) = g. f(x) for all g E and æ E X. the action is trivial (i.c., g y = for all g E G and y Y), then is called invariant, meaning fg 2) f(x) for all g E G and x E X.
In the context of static graph representation learning, permutation equivariance ensures that the model's output does not depend on the arbitrary ordering of nodes. Time-warp equivariance important in time series modelling when the task depends on the sequential structure of the signal rather than the absolute timing of events. For example, in classifying the nodes of social media network as to whether they are friends with specific node or friends-of-a-fire important characteristic is the order in which the edges appeared (you cannot connect with a friend- of-a-friend without first connecting with the friend) and not the time in-between those connections appearing. We now formalise both types of equivariance:
with that node, the
1. Permutation equivariance: Given permutation p [n] - [n] with corresponding matrix P E Rax=, a function f Rn x [ + [aad is permutation equivariant if f(PX, PAP") Pf(X, A) for all permutations p and all graphs with node features X and adjacency matrix A.
Time-warp equivariance: Let X [0, 7] + be a continuous path. time-warp a diffeomor-
phism (a smooth bijection with smooth inverse) r [0, [0, - [0, T] satisfying r(0) 0 and r ((T) = T. function f x(00, T), Y) > x(00, T). Y) time-warp equivariant if f(X a T) = f(X) o T for all time-warps and paths X.
For a more formal treatment of group actions and equivariance, we refer the reader to Appendix
2.2 Neural differential equations for time-series data
Neural Differential Equations (NDEs) (446 52 7 34) have emerged as a powerful tool at the
intersection of dynamical systems and deep learning. In [8) the authors viewed the layers of a neural network as the time variable of an Ordinary Differential Equation (ODE), introducing the Neural Ordinary Differential Equation (NODE). In NODEs, the time dimension is merely internal detail of the model, and the trajectories z are entirely determined by the initial condition. To extend the NODE framework to sequential data, (36) introduced Neural Controlled Differential Equations (NCDEs) via controlled differential equations (CDEs). For a continuous driving path X [0, + Rd. NCDEs learn a latent path z : [0, [0, + Rdd via the CDE
20) = (X)(),), 2(t) = + J. fo(x(s))dX(s)
output path v(t) s (=(())). Here, (s) denotes the Bremann-Stieltjee integral. In practice, we often only have discrete observations XG,)..., (tv. Xxx)) with t, E R and Xe, E Rdd, To address this, we interpolate the obser- vations into a continuous driving path X : [to, tN] + d2 such that X(t,) = (t,, Xx,) for all This natively enables NCDEs to process discrete-time sequences or hybrid continuous-discrete dynamics. If the path X is differentiable and has bounded derivative, the Sremann-Stieltiee integral, and hence NCDEs, can be rewritten the following ordinary integral:
which then returns either scalar output y R (z(2(T))
=(t) = z(0) + / fa()(s))
dx(x) ds
ds fort E (0,7).
The choice of interpolation scheme can impact the performance of an NCDE, with 45 providing discussion on the theoretical properties and practical performance range of choices. Additionally, Log-NCDEs (64) extend NCDEs to non-differentiable paths X by leveraging the Log-ODE method to approximate solutions to Equation (I). Empirically, this improves training stability and model performance, especially for long time-series.
3

---

2.3 Permutation invariant and equivariant linear functions
Graph Neural Networks are typically constructed as permutation equivariant functions by propagating information locally on graphs following a message passing paradigm. An alternative approach, proposed by [44, represents d-dimensional graph data on k-tuples of nodes as a single matrix Y E IR" xd For example, the case
corresponds to node signals and 2 to signals on edges. In their work, the authors provide a full characterisation of linear maps L I xd R= that are equivariant with respect to permutations of the underlying node set. Remarkably, they show the dimension of the vector space formed by these equivariant maps, denoted by Ez (k: dyd depends only on k, d, and d', and is independent of n. In Section 3 will be interested in the basis of
Cx. (2, 1)', the vector space of linear maps L R= - R. between edge-vane valued data, which has dimension of 15. For a list of terms spanning this basis, see Appendix A
3 Permutation equivariant neural graph controlled differential equations
This section brings together Neural CDEs and temporal graph representation learning. We begin by introducing a recent approach proposed in [48], called Graph Neural Controlled Differential Equations. Through the lens of Geometric Deep Learning [5], we will identify key theoretical limitation of this approach and propose an improved solution.
3.1 Graph Neural Controlled Differential Equations
The concept of neural controlled differential equations has been extended to graphs by incorporating dynamic adjacency matrices to drive the CDE dynamics [48)
Graph Neural CDEs. Let Ço R2xxx2 > nxdd and fo Raxd, x RRax - R(nxd=)x (mxxxx))
be two graph neural networks. The neural controlled differential equation for dynamic graphs is defined as
Z: = Zio + fo,, for t (to, ty]
where Zi, = (A,,), is the initial condition, [to. tN] + R2xax2 such that A = (tR, A and the product fo AAA, is a tensor contraction over mxxxxx The final prediction Y, Rxd, attained by row-wise application of another linear function to Rdd Bdd, i.e. by slight abuse of notation Y. = 6 (Z.).
Practical Considerations. Implementing Equation directly is computationally intractable due to the large output dimension of fo and the complexity of the tensor contraction under the integral sign. Hence, [48] proposes the following simplification based on a message passing paradigm:
Z Zoo + [ z for t (fo.tx)
(4)
where z = σ (A,Z! (1-1) will 1))
for 4,, the adjacency matrix and its derivative are with (FF) R222n being a learnable fusion matrix, and Z Z.
A,
fused via A, (F)
JA,
Importantly, this simplification preserves the multiplicative interaction between the hidden state and control path which are critical for expressivity [33
3.2 Inducing permutation equivariance
As motivated in Section 2 it would be natural require equivariance with respect permutation of the node and edge sets for any function on graph data. However, neither the original GN-CDE
We note subsequent version of the GN-CDE introduced an additional fusion matrix, A
A.
W P
JA
W.P
49
Our theoretical analysis still is valid, as the fusion remains linear map.
ds

---

formulation in Equation 3 nor its approximation in Equation satisfy this property. See Appendix E for a detailed proof.
In the following, we introduce a fully permutation equivariant variant of the GN-CDE framework.
By leveraging the characterisation of linear permutation equivariant layers presented in Section 2.3 our goal is to approximate Equation while maintaining equivariance. Since the equivariance breaks in the fusion step. our strategy is project the fusion operation onto the subspace of linear equivariant functions. Specifically, if we interpret the fusion as the application of linear maps L2 Ruxn + 1R to the matrices A, and dA,, respectively, we can project L and L2 onto Cx, (2, According to the Projection Theorem (see Appendix B this yields the best possible approximation. This motivates us propose the following:
The model. Let L1, L E εx,, (2, 1)a be two learnable permutation equivariant linear maps, i.e. weighted combination of the basis terms in Appendix A its time derivative as
Using these, combine the adjacency and
dAA ds
A. -
L2
Now, let be non-linear activation function and denote by _.0) the latent representation obtained by an iterative convolution operation of the form Z o (A,Z, (((=!"""""")) for (L....,L) L where the w are learnable matrices. The Permutation Equivariant Neural Graph Controlled Differential Equation (PENG-CDE) then takes the form
Z. Zio + !"d ((A. for 1 LN]-
with initial condition Zu = (A
Theoretical Properties. In the purely linear case (i.e., without applying the non-linearities a in between the layers), one show that the notion of optimality above, motivated by the Projection Theorem, is satisfied, as formalised in the following theorem:
Theorem 3.1. In the absence of non-linearities the PENG-CDE model in Equation is the projection of the model in Equation onto the space of equivariant linear finctions.
Proof sketch. Projecting the flow of an ODE onto a function space is equivalent projecting its associated vector field. Moreover, note that the vector field in Equation 4 can be decomposed as the composition of standard convolutional graph neural network, which is iherently equivariant, and the adjacency fusion, which is not. Thus, projecting Equation is equivalent to projecting only the fusion operator. For detailed proof, see Appendices and D
Although the projection- based notion of optimality holds strictly only in the linear case, by construc- tion, model satisfies both equivariance constraints outlined in Section 2.1
Proposition 3.2. The PENG-CDE model in Equation 6 is both permutation equivariant in the spatial domain and warp equivariant in the temporal domain.
Proof. See Appendix
3.3 Including dynamic node features
So far, our framework only incorporates dynamic adjacency matrices into the latent dynamics, leaving dynamic node features unaddressed. To remedy this, we draw on the approach of (TT). Concretely, suppose node feature snapshots (X,, T are available, where each X,, E Rd We interpolate these snapshots to obtain continuous, differentiable path X [to, tv] + Rd as in Section 2.2 Next, we choose (LL) E Rdd. (d, x(d=+1)) to set the output dimension of dA Z! W( to
(d, (d)+ enabling riw-wise (i.e., node-wise) Hadamard multiplication, denoted by G. The resulting system
Z: = Zio + [ ('(o(A. wil)) c
d ds
ds for t (to, tx].
(7)
5

---

Building on Proposition 3.2 this extension preserves both permutation equivariance in the spatial domain and ware equivariance in the temporal domain. Intuitively, this is because a permutation
of the node indices reorders both of A, Z (D) w and dX,, in the same manner, ensuring that the overall model remains equivariant. A detailed proof is provided in Appendix D
da
4 Numerical experiments
In this section, we evaluate the PENG-CDE model on a range of synthetic and real-world tasks. First, replicate the experiments from 48 and compare them with the non-permutation-eqiivaaii version. Next, we evaluate the model on well-established real-world dynamic graph benchmarks against other common approaches. Finally, we conduct an ablation study to examine the weighting of different basis terms of Es,, (2. 1)' in Appendix 4.3 For supplementary information regarding implementation details, see Appendix G Appendix contains a statistical analysis of the results.
4.1 Synthetic experiments: heat diffusion and gene regulation
Task. We randomly sample initial graphs from four distinct graph distributions (grid, small-world, power-law, and community), each comprising 400 nodes. We then take 120 irregularly sampled time-stamps spanning T' 0 to T 5. At 12 time steps, sampled uniformly at random from these snapshots, we randomly add or remove edges following Bernoulli trial. The final 20 snapshots are allocated for extrapolation validation, while from the remaining 100, a random subset of 20 is used for interpolation validation and the remaining 80 for training. This means the expected number of times the graph topology changes during training and validation phases are 8 and 4, respectively. A batch of four such time series are generated for each of training, validation, and testing. We then simulate node features according the heat diffusion dynamics governed by Newton's law of cooling and the gene regulatory dynamics governed by the Michaelis-Menten equation. For additional experiments on personal capital and opinion dynamies, Appendix
Baselines. We compare the performance of several models, including state-of-the-atr re- current (DCRNN [40]), interactive (STIDGCN 411). and attentional (ASTGCN (26) baselines. For differential equation- based baselines, we first consider simple model with constant vector field (Const), that is Z!() b for all E tN] in Equation 4 Next, we consider the Graph Neural ODE (GNODE) model 47), in which the adjacency matrix within the vector field is discretised by flooring the time index. The GNODE is governed by the equation:
UNIDE PENC-DDE
Phe Sul. UNGDE Graph topelag change
Acj
Hest Diffusion
Regulation
(KSE)
Loss (NSE)
10
Time
Time
Test losses, plotted against simula- tion time, for the Graph Neural ODE, three GN- CDE variants, and proposed Permutation- Equivariant GN-CDE model on the heat diffusion (left) and gene regulation (right) tasks. Dashed ver- tical lines mark changes in graph topology, while the bold black line indicates the final time point in the training set. Results are reported as means (solid) and ranges (shaded) over test set with batch size of four.
Figure 1:
Z, Zio + {. fo(Z.. Ass
(8)
for all t E (fo.tN] where A is the adja- cency matrix corresponding the graph Gr. with tx [s]. Additionally, we consider sev- eral variants of the GN-CDE. Firstly, we include simplified fusion model, named Adjacency GN-CDE, which employs interpolated adjacency matrices, i.e. A, = Ag. We note that for the experiments on these two datasets. the GN-CDE 48 model implemented the fusion by a simple element- wise summation between the adjacency matrix and its derivative (i.e. A, = A, dA, which implicitly achieves permutation equivariance a property not present in the original formulation. To ensure fair comparison with a non-equivariant GN-CDE variant, we also consider the Premulti- plication Fusion GN-CDE (Pre Mult GN-CDE) model. In this variant, the fusion step is performed via premultiplication by learnable matrices W, and Wz, such that A, W,A, + W, dA, we include our proposed Permutation Fquivariant Neural Graph CDE (PENG-CDE), Experimental results for the heat diffusion and gene regulation tasks are summarised in Table I Moreover, we visu-
de ),
W,dA,, Lastly,

---

alise the test loss over simulation time for models initialised from the community graph distribution
Figure !
Finding I: Equivariance improves performance. Incorporating permutation equivariance (found in models Adjacency GN-CDE, Original GN-CDE, and PENG-CDE) leads performance improve- ments of an order of magnitude over the non-equivariant Pre Mult GN-CDE across all considered tasks and graph distributions.
Finding II: Enhanced expressivity via 15 basis terms boosts performance. Integrating all 15 basis terms of linear equivariant maps into the fusion of model significantly enhances its expressivity. Compared to the original implementation, the PENG-CDE achieves relative MSE improvements ranging from 30.44% 73.84% on the heat diffusion task and from 39.71% to 67.06% on gene regulation tasks. One plausible explanation is that summing across rows (hasis term 3 in Appendix A) corresponds to computing node degrees in unweighted graph, which facilitates degree normalisani as required Equation 26
Finding III: Additional terms enhance extrapolation behaviour. As shown in Figure for the heat diffusion task, our PENG-CDE is the only model capable of maintaining constant losses during the extrapolation phase (i.e. over the final 20 snapshots). This suggests that the added equivariant terms contribute to more robust extrapolation, likely due to the mechanisms discussed above.
Table 1: Comparison of GN-CDE variants and basclines the heat diffusion (top) and gene regulation (bottom) tasks. Mean MSEs with 95% confidence intervals are reported, with the best mean highlighted in bold, and all results within the corresponding confidence interval are underlined. The final row in each table reports the relative improvement of PENG-CDE over the original GN-CDE formulation.
Heat Diffusion Task (MSE 4)
Model
Community
Grid
Power Law
Small World
DCRNN
40
0.722 +:2 1.145
70.392 + 108.473
1.690 + 1.456
84.690 +:: 105.696
STIDGCN ASTGCN
0.554 1 0.393
4.297 1 0.975
0.907 + 0.268
1.826 11: 0.240
26
2.188 1 0.656
15.480 1 1.757
3.849 1 0.832
8.269 1 1.094
STG-NCDE
2.091 + 0.645
11.989 + 1.090
3.518 + 1.105
6.902 1.303
Constant
1.936 + 0.550
11.155 + 0.669
3.147 + 0.850
6.286 + 1.056
Graph Neural ODE
47
0.237 + 0.322
1.001 + 0.751
0.270 + 0.310
0.311 + 0.268
Adjacency GN-CDE
0.208 + 0.240
0.691 + 0.887
0.258 + 0.288
0.248 + 0.240
Pre Mult GN-CDE
1.968 = 0.548
12.262 + 1.437 1.437
7.440 + 4.458
6.829 = 0.644
Original GN-CDE
48
0.366 + 0.400
1.324 + 0.630
0.417 + 0.314
0.552 + 0.170
PENG-CDE (ours)
0. 1096=00000
0.481 + 1.15
0.290 0.265
0.247 I 0.215
Relative Improvement
73.84%
63.70%
30.44%
55.20%
Gene Regulation Task (MSE L)
Model
Community
Grid
Power Law
Small World
DCRNN
159.095 + 149.970
28.784 + 16.926
77.469 + 28.178
27.300 + 11.641
STIDGCN
14.579 + 2.815
0.633 + 0.163
4.828 + 0.694
0.611 + 0.162
ASTGCN
13.366 + 3.302
0.695 + 0.178
4.722 + 0.644
0.379 + 0.116
STGNCDE
88.354 + 15.126
8.753 1 0.898
20.498 1 2.633
6.760 1 0.787
Constant
36.307 + 2.609
1.390 + 0.168
7.099 + 0.382
0.772 + 0.126
Graph Neural ODE 47
8.548 3.212
0.167 1 0.191
0.372 1 0.398
0.294 + 1.308
Adjacency GN-CDE
8.909 + 6.311
1.476 + 2.504
0.498 + 0.126
0.195 + 0.046
Pre Mult GN-CDE
153.084 + 149.609
2.553 4 0.251
6.978 + 3.045
1.591 + 0.146
Original GN-CDE
48
10.717 + 7.079
0.457 + 0.167
0.822 + 0.299
0.323 + 0.154
PENG-CDE (ours)
4.566 1 2.780
0.247 1 0.090
0.526 1 0.220
0.186 1 0.484
Relative Improvement
67. 67.06%
45.90%
39.71%
54.14%

---

Table 2: Results of experiments for the england-cove covid and switter-tennle tasks from the Pytorch Geometric Temporal datasets [55]. Mean values and 95% confidence intervals are reported, with the best mean highlighted in bold and all results within the confidence interval around the best mean are underlined.
england-covid MSE +
twitter-tennis MSE
Model
Validation
Test
Validation
Test
DCRNN 40
0.898 0.126
1.021 0.203
0.472 1 0.136
0.455 + 0.087
ASTGCN 26
1.235 + 0.139
1.283 + 0.257
0.545 + 0.128
0.530 + 0.078
STIDGCN
0.916 0.111
0.933 0.100
0.524 0.140
0.495 + 0.059
GNODE 47
0.802 + 0.184
0.945 + 0.251
0.419 + 0.068
0.525 = 0.043
STG-NCDE
1.484 1 0.515
1.778 + 1.084
0.372 1 0.108
0.453 1 0.035
GN-CDE 48
0.892 1 0.143
0.962 0.278
0.369 1 0.070
0.443 1 0.053
PENG-CDE (ours)
0.836 0.122
0.913 0.200
0.391 1 0.069
0.440 1 0.053
Oversampling and irregularity. In Figure 2 we compare the performance of the PENG-CDE with DCRNN 40). STIDGCN 41 and ASTGCN [26] on data generated from the graph SIR disease spread model 33 Its parameters are chosen to produce trajectories both with and without an outbreak, and the task is to classify each trajectory into one of the two categories. To study the effect of oversampling, we simulate each trajectory up to fixed terminal time T = 1 and train and evaluate the models on datasets with an increasing number of observation points (left and middle panels). We also investigate how performance varies with the regularity of the sampling grid: by drawing observation times from a Gamma distribution with controlled shape parameter k; we obtain series with different degrees of irregularity and plot test accuracy as irregularity increases (right panel). For implementation details, see Appendix G.3
Finding IV: PENG-CDE is robust to oversampling and irregular sam- pling. CDE-based models such as
STIDOON
ASTSON
DORNN
FENG-CDE
Randam
Test Accuracy (%)
Iifrencce Time (6)
PENG-CDE decouple the computa- tional complexity of their forward passes from the number of input ob-
Acpuraty
servations: the number of vector field evaluations determined by the ODE solver, not by the sampling rate of the data. This property makes them in- herently robust to oversampling. As shown in Figure 2 increasing the sam- pling frequency negatively impacts both the performance and runtime of the recurrent baseline (DCRNN), while our model remains largely unaffected. Although STIDGCN also maintains performance under oversampling, its computational cost increases with the number of observations, in contrast to the constant runtime of our approach. Likewise, PENG-CDE sustains high performance across all levels of irregularity, while the other models exhibit degraded performance.
samples
samplos
Irregllntty 3
Figure 2: STIDGCN, ASTGCN, DCRNN, and our PENG-CDE as a function of the number of observed timesteps (left) and sampling irregularity (right). Inference time with increasing numbers of observations is shown in the middle panel.
Classification accuracy on the SIR model for
4.2 Real-world tasks
We evaluate framework two types of real-world datasets: snapshot-based and event-based. First, we consider two node regression tasks from PyTorch Geometric Temporal [53] and then two
node affinity prediction tasks from the Temporal Graph Benchmark (TGB) 229 24 Implementation details are provided in Appendix G.4
Snapshot-based datasets. In this setting, we observe a sequence of dense graphs, one at each timestep. To demonstrate that our modifications enable our model to better capture dynamic graph topologies, we compare it against several differential equation-based models: the Graph Neural ODE 47], the the original GN-CDE [48) Additionally, include comparisons with recurrent (DCRNN 40)).
Graph Neural Controlled Differential Equation (STG-NCDE) [1, and
8

---

attentional (ASTGCN 26)), and interactive (STIDGCN A models. We benchmark performance using the england andacndidiiooo and switter-tente datasets from PyTorch Geometric Temporal (55] the only datasets in the library exhibiting dynamic graph topologies. Table 2 reports the mean MSE with 95% confidence intervals on both the validation and test sets over 10 random seeds.
Finding V: PENG-CDU generalises well on dynamic snapshot datasets. While not achieving the lowest validation error, PENG-CDE attains the lowest test error on both tasks, indicating better generalisation and reduced overfitting compared to other models.
Table 3: Experimental results for the tgbn-trade and tgbn-geere node affinity prediction tasks from the Tem- poral Graph Benchmark datasets (29 224] Results marked with j are from [29]. those with & are from [68), and those with are from (59 Mean values and standard deviations
Event- datasets. In many ap- plications, a temporal graph is repre- sented as sequence of events G ((e,, (4, x;))]==> where each event e; (e.g., interaction between nodes and v) occurs at time 1; and is asso- ciated with data æq E Rd. Since GN- CDEs are not inherently designed for processing individual events, we ag- gregate events into snapshots. In more detail, given a time window of length At. we partition the overall time span [to, fn] into m A non- overlapping intervals. For each inter- val indexed by k E {0, 1..
are reported with results within one standard deviation of the best mean highlighted in bold (for deterministic and learned models separately).
trade
genre NDCG@10
Model
Persistent Forecast (L)*
0.855
0.357
Moving Avg (L)*
0.823
0.509
Moving Avg (M)
0.777
0.472
1}. we define the snapshot graph Ge to include all events (or edges) that occur within the time window (oo + k to + (k + 1)At). This then gives sequence of snapshots ( 1m-1), which we can in- tegrate into the setting above. We evaluate our framework in this event- based setting on the trade and genre node affinity prediction tasks from the Temporal Graph Benchmark (TGB)
JODIE
39
0.374=0000 09
0.350=0.01
TGAT
15
0.375=0.07
0.352#0.03
CAWN+
0.374.10.09
TCL
65
0.375.10.09
0.354
GraphMixer
14
0.375=20.11
0.352=0.00
DyGFormer
68
0.388=0.04
0.365=#0.20
DyRep
60
0.374:t.000
0.351#t00001
TGN 53
0.374::.000
0.367 058
TGNv2
559
0.735=t0.000
0.469 0002
STG-NCDE 10
0.618+0.024
0.438
29 24). These two tasks were se- lected from the four available in the benchmark, as the dense adjacency matrices required by Neural ODE- based models make the remaining tasks computationally infcasible within our constraints.
GN-CDE [48]
0.71310.026
0.460
PENG-CDE
0.71640.022
0..23=0.011
Sourre/Target ld
0.734+4.022
Baselines. For the event-based experiments, first compare against three simple heuristics based on ground-truth labels and messages: 'Persistent Forecast (L)' and 'Moving Average (L)', which operate labels, and 'Moving Average (M)', which is applied to messages. We then benchmark several learned models, including JODIE 339. TGAT [5),, CAWN (66). TCL [65] GraphMixer (4), DyGFormer (68], DyRep [60], TGN (53), and TGNv2 (59] Additionally, we include STG-NCDE [10]. GN-CDE [88 and our proposed PENG-CDE. It is known issue that the heuristics above often outperform learned models on node-affinity prediction tasks (24 see (59 for detailed discussion.
This motivated TGNv2 to introduce a mechanism that learns node embeddings to distinguish source and target nodes for each interaction, feature tailored specifically to the node affinity prediction task. To ensure a fair comparison, we incorporate an analogous mechanism into a variant of our model. Further details are provided in Appendix G.3 The experimental results are summarised in Table 3
Finding VI: PENG-CDE achieves state-of-the-as performance on TGB. Without source-targed identification, our model is outperformed only by TGNv2 on the egbn-trade task; with tarree-target identification, it performs on par. Moreover, on tgbn-genre, our model significantly outperforms all baselines; notably, it is the only machine-learned approach to surpass all heuristic baselines.
'Here, the CDE employs the original element-wine additive fusion, A. = A. dA, 48 Concurrently
da to this work. Qin et al. [4] proposed a non lonelinear element-wing fusion for GN-CDE tailored to the TGBN datasets that achieves comparable performance the permutation-equuiariann fusion of PENG-CDE.

---

4.3 Ablation studies
Fusion. To understand how model fuses the adjacency matrix and its derivative, we analyse the learned weights of the 15 basis terms (Table in the heat diffusion task described in Section [4.]] Both GCN layers assign the largest weights to the identity op- erations. This is expected, given heat diffusion aggregates over node neigh- bourhoods. Because the constructed graph is undirected, the transpose op- eration is equívalent to the identity, allowing the corresponding weight in layer 1 to be disregarded. Layer also assigns importance to non-identity components. Large weights are given to the summations of adja- cency matrix rows and columns, corresponding to node degree computation, which indicates attention to global graph properties. Similarly, large weights are observed for the total sums of both the adjacency matrix and its derivative, which may correspond to form of normalisation. Overall, we see that the model utilises the additional expressivity over GN-CDEs.
Table Fusion operation weights of the Permutation Equiv- ariant GN-CDE model. Weights with absolute value larger than 0.1 are in bold.
Layer
Layer
Operation
A,
dA.
A,
dA.
Identity Transpose
1.6129
0.9591
1.0747
0.9100
-0.0270
0.0535
Diagonalisation
0.0297 -0.0754
0.0072
0.0317
rows
0.7753 - -0.0635
0.0358
-0.0114
Sum rows
columns
0.3407
0.0015
-0,0242
0.0790
diagonal
- -0,0875
- -0,0222
-0.0707
0.0869
all
0.4315 0 0.4260
0.0039
0.0020
Sum all diagonal
0.0002
-0.0001
0.0362
0.0088
5 Conclusion
In this work, we introduced Permutation Equivariant Neural Graph CDEs a geometrically grounded approach to temporal graph representation learning that balances parameter efficiency with formal expressivity. In the linear setting, we showed that our model can be derived as a projection of Graph Neural CDEs onto the space of permutation equivariant functions. Through synthetic experiments, we demonstrated that both the imposed equivariance and enhanced expressivity improve performance in both interpolation and extrapolation tasks. Additionally, our model retains the strengths of Neural CDEs in handling oversampling and irregular time intervals. On the TGB- dataset, our model achieves state-of-the-are becoming the first learned method to surpass moving average baseline.
5.1 Limitations and future work
Like the original GN-CDE, PENG-CDEs have quadratic memory complexity in the number of nodes due to the need to store dense adjacency matrices, which restricts scalability to large, sparse graphs. This constraint arises primarily from JAX's limited support for sparse matrix operations. As shown in Table i in Appendix[[ [[] once memory allocation constraints are met, PENG-CDE scales much more favourably in runtime compared to GN-CDE demonstrating its potential for efficient modelling at larger scales.
Future work includes addressing this scalability bottleneck and investigating the impact of more tailored hyperparameter configurations, such as the choice of interpolation schemes, ODE solvers, and vector field architectures. In addition, advanced solvers like Log-NCDE (64, originally developed for Euclidean paths, could be adapted to the graph setting. While we chose dynamic adjacency matrices as the control signal for the CDE, this is just one possible vectorisation of graph structure. Prior work (25] suggests that adjacency matrices are often suboptimal for representing graph properties. Identifying more expressive or task-aligned graph representations could further improve performance and remains an exciting research direction. Finally, while Neural CDEs are known to be universal approximators on Euclidean domains 36). and recent work has also provided generalisation bounds 33 no such theoretical guarantees currently exist for wraph-based Neural CDEs. Establishing these would be a valuable theoretical contribution to the growing field of temporal graph representation learning.
10

---

Acknowledgements
This work is supported by the Helmholtz Association Initiative and Networking Fund on the HAICORE@ KIT partition. Benjamin Walker is funded by the Hong Kong Innovation and Technology Commission (InnoHK Project CIMDA).
References
[1] J.P. Aubin. Applied Functional Analysis. Wiley-Intersciencee publication. Wiley, 1979. ISBN 9780471021499 [RLL/tttps:/bboks.goog.. co. wttp:/uowu..ooooooddddadaaaaaaaa
[2] Richard Bergna, Sergio Calvo-Ordonez, Felix Opolka, Pietro Liò, and Jose Miguel Hernandez- Lobato. Uncertainty modeling in graph neural networks via stochastic differential equations, 2025.
[3] Linus Bleistein and Agathe Guilloux. On the Generalization Capacities of Neural Controlled Differential Equations. In Workshop on New Frontiers in Learning, Control, and Dynamical Systems the International Conference on Machine Learning (ICML 2023), Honolulu (Hawai), United States, July 2023. URL hetph:tts://ie //bal wcweencechaale.e..e 0487660
[4] James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman- and Qiao Zhang. JAX: composable transformations of Python+NumPy programs, 2018. URL
http://github com/ httpg//twwbcoooooog/o//eeeeee//e google/jax
[5] Michael M. Bronstein, Joan Bruna, Taco Cohen, and Petar Veličković. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges, 2021.
[6] Ben Chamberlain, James Rowbottom, Maria I. Gorinova, Michael Bronstein, Stefan Webb, and Emanuele Rossi. GRAND: Graph Neural Diffusion. In Proceedings of the 38th International Conference on Machine Learning, pages 1407-1418. PMLR, July 2021. URL https:// proceedings mlr. www.ppressvvvie /oress/v139/oe whwmee/aainaaa.lai........ html
ISSN: 2640-3498.
[7] Ricky T. Q. Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. Neural ordinary differential equations. In Proceedings of the 32nd International Conference on Neural Infor- mation Processing Systems, NIPS' 18, page 3572-6583, Red Hook, NY, USA, 2018. Curran Associates Inc.
[8] Ricky T. Q. Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018. URL
neurips. httpppper_fllessppeeeeeee .ddf
[9] Kyunghyun Cho, van Merrienboer, Caglar Gulcehre, F Bougares, H Schwenk, and Yoshua Bengio. Learning phrase representations using mn encoder-decoder for statistical machine translation. Conference on Empirical Methods in Natural Language Processing (EMNLP 2014), 2014.
[10] Jeongwhan Choi and Noseong Park. Graph neural rough differential equations for traffic forecasting. ACM Trans. Intell. Syst. Technol., 14(4), July 2023, ISSN 2157-6904. doi: 10.1145/3604808. URL
htg.//00.........oo/ 11T.1....111// 1145/3604803
[11] Jeongwhan Choi, Hwangyong Choi, Jechyun Hwang, and Noseong Park. Graph neural con- trolled differential equations for traffic forecasting. In AAAI, 2022.
[12] Jeongwhan Choi, Seoyoung Hong, Noseong Park, and Sung-Bae Cho. Gread: Graph neural reaction-diffusion networks. In ICML, 2023.
[13] Nicola Muca Cirone, Antonio Orvieto, Benjamin Walker, Cristopher Salvi, and Terry Lyons. Theoretical foundations of deep selective state-space models. Advances Neural Information Processing Systems, 2024.

---

[14] Weilin Cong, Si Zhang, Jian Kang, Baichuan Yuan, Hao Wu, Xin Zhou, Hanghang Tong, and Mehrdad Mahdavi. Do we really need complicated model architectures for temporal networks?. 2023. URL https:///axxi owg/abg.////ooooooooo 2330222. 2302. 11636,
[15] da Xu, chuanwei ruan, evren korpeoglu, sushant kumar, and kannan achan. Inductive represen- tation learning on temporal graphs. In International Conference on Learning Representations (ICLR), 2020.
[16] Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. Convolutional neural networks on graphs with fast localized spectral filtering. In Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS' 16, page 3844-3852, Red Hook,
NY, USA, 2016. Curran Associates Inc. ISBN 9781510838819,
[17] Moshe Fliasof, Eldad Haber, and Eran Treister. Graph Neural Reaction Diffusion Models. 46(4):C339-C4420 ISSN 1064-8275, 1095-7197. doi: 10.1137/23M1157670 URL https
/ //epubs. siam. orw//oi/////ooooooooo /10///0111.0// 1137/23M1156700
[18] Moshe Eliasof, Eldad Haber, and Eran Treister. Pde-gen: novel architectures for graph neural networks motivated by partial differential equations. Proceedings of the 35th International Conference Neural Information Processing Systems, NIPS '21, Red Hook, NY, USA, 2021. Curran Associates Inc. ISBN 9781713845393.
[19] Moshe Eliasof, Eldad Haber, and Eran Treister. Feature transportation improves graph neural networks. In Proceedings of the Thirty-Fighth AAAI Conference on Artificial Intelliencce and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence,
AAAI Press, 2024. ISBN 978-1-57735-88 //ooi.org// 1609 aaai v38111.29075
doi: 10.1609/aaii 338. 1.29073. URL https
[20] Moshe Eliasof, Eldad Haber, Eran Treister, and Carola-Bibiane Sch" Data-driven higher order differential equations inspired graph neural networks. In ICLR 2024 Workshop on A44Differeeniallliuatto In Science, 2024. URLptttps: ptttpp:///ppnrevveee
IJeeXWBByt
[21] Moshe Eliasof, Eldad Haber, Eran Treister, and Carola-Bibiane B onh'on. On the temporal domain of differential equation inspired graph neural networks. In International Conference Artificial Intelligence and Statistics, pages 1792-1800 PMLR, 2024.
[22] Wenqi Fan, Yao Ma, Qing Li, Yuan He, Eric Zhao Jiliang Tang, and Dawei Yin. Graph neural networks for social recommendation. In The World Wide Web Conference, WWW '19, page 417-426, New York. NY, USA, 2019, Association for Computing Machinery, ISBN 978 450466466 doi: 11114//330558833314444 URL http::ttt///ooot ott/10..r./11..111. 3308558 3313488
[23] Matthias Fey and Jan E. Lenssen. Fast graph representation learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds, 2019.
[24] Julia Gastinger, Shenyang Huang, Mikhail Galkin, Erfan Loghmani, Ali Parviz, Farimah Pour- safaei, Jacob Danovitch, Emanuele Rossi, Ioannis Koutis, Heiner Stuckenschmidt, Reihaneh Rabbany, and Guillaume Rabusseau. Tgb 2.0: A benchmark for learning on temporal knowledge graphs and heterogeneous graphs. Advances in Neural Information Processing Systems, 2024.
[25] Martin Grohe. word2vec, node2vec, graph2vec, x2vec: Towards theory of vector embeddings of structured data. Proceedings of the 39th ACM SIGMOD-SIGACCT-SICA Symposium on Principles of Database Systems, PODS 20, page 1-16, New York, NY, USA. 2020. Association for Computing Machinery. ISBN 978 14503710877 doi: 00.1145/33539953388666 URL
https://doi: http///i..ogg//../ 1145/3375395. 3387641
[26] Shengnan Guo, Youfang Lin, Ning Feng, Chao Song, and Huaiyu Wan. Attention Based Spatial-Tempprra Graph Convolutional Networks for Traffic Flow Forecasting. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01):922-929, July 2019. ISSN 2374-3468, 2159-5399. doi:
12

---

[27] Shengnan Guo, Youfang Lin, Huaiyu Wan, Xiucheng Li, and Gao Cong, Learning Dynamics and Heterogeneity of Spatial-Tempora Graph Data for Traffic Forecasting. IEEE Transactions on Knowledge and Data Engineering. 34(11):54155 1):411554222 November 2022. ISSN 1041-4347, doi: 10.1 (0000//TKDDD22213335002
1558-2191, 2326-3865.
[28] Charles R. Harris, K. Jarrod Millman, Stéfan J. van der Walt, Ralf Gommers, Pauli Vir- tanen, David Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathamicl J. Smith, Robert Kern, Matti Picus, Stephan Hoyer, Marten H. van Kerkwijk, Matthew Brett, Allan Hal- dane, Jaime Fernández del Río, Mark Wiebe, Pearu Peterson, Pierre GGrard-Marchant, Kevin Sheppard, Tyler Reddy, Warren Weckesser, Hameer Abbasi, Christoph Gohlke, and Travis E. Oliphant. Arry programming with NumPy. Nature, 585(7825):357 362, September 2020. doi: 10.1038/s41586-00 020-866-002220 020-0049-22222222 Uttpttps::
org/ 10 1038 1031100081188800 s41586 264-2
[29] Shenyang Huang, Farimah Poursafaei, Jacob Danovitch, Matthias Fey, Weihua Hu, Emanuele Rossi, Jure Leskovec, Michael Bronstein, Guillaume Rabusseau, and Reihaneh Rabbany. Tempo- ral graph benchmark for machine learning on temporal graphs. Advances in Neural Information Processing Systems, 2023.
[30] Weiwei Jiang and Jiayun Luo. Graph neural network for traffic forecasting: A survey. 207: 117921. ISSN 09574174. doi: 10. w0... 119211. latpss:/::// com http://c.wc..rerrraea///////////
[31] John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ron- neberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Žídek, Anna Potapenko, Alex Bridgland, Clemens Meyer, Simon A. A. Kohl, Andrew J. Ballard, Andrew Cowie, Bernardino Romera-Paredes, Stanislav Nikolov, Rishub Jain, Jonas Adler, Trevor Back, Stig Petersen, David Reiman, Ellen Clancy, Michal Zielinski, Martin Steinegger. Michalina Pacholska, Tamas Berghammer, Sebastian Bodenstein, David Silver, Oriol Vinyals, Andrew Senior, Koray Kavukcuoglu, Pushmeet Kohli, and Demis Hassabis. Highly accurate protein structure predic- tion with alphafold. Nature, 596 589, 2021. doi: 10. 10.1038/44133--01-03333 htLpss/s:://ooi org 10 0ww.0/////488810 021 031-1-999222
[32] W.O. Kermack and A.G. McKendrick. A contribution to the mathematical theory of epidemics.
115(772)700--21. ISSN 0950-1207, 2053-91155 doi: 10. 1098/spap 1927.01 00188. URL https
/rooallsciettyubllsshii...oogoge http:///ii...la..oooooooo//.../.
192770015
[33] C. Kerr. The Mathematical Theory of Infectious Diseases and its Applications. By Norman T.
J. Bailey, M.A., D.Sc.; second edition, 1975. London: Charles Griffin & Co. Ltd. 9*x6*, pp. 430, with diagrams. Price: f14.00. 1(18):674. ISSN 0025-729X, 1326-5377. doi: 10.5694/ j.1326 5377, 197..1144....199.. 5694/j 1326 5377 1976. tb140951.x
URL
cttp://coo////////////////////// abs//0
[34] Patrick Kidger. On Neural Differential Equations. PhD thesis, University of Oxford, 2021.
[35] Patrick Kidger and Cristian Garcia. Equinox: neural networks JAX via callable PyTrees and filtered transformations. Differentiable Programming workshop at Neural Information Processing Systems 2021, 2021.
[36] Patrick Kidger, James Morrill, James Foster, and Terry Lyons. Neural Controlled Differential Equations for Irregular Time Series. Advances in Neural Information Processing Systems, 2020.
[37] Diederik P. Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization. International Conference on Learning Representations (ICLR), 2015.
[38] Thomas Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. ArXiv, abs/1609.02207, 2016. URL
org/
CorpusID:3144218
[39] Srijan Kumar, Xikun Zhang, and Jure Leskovec. Predicting dynamic embedding trajectory temporal interaction networks. In Proceedings of the 25th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 2019.
13

---

[40] Yaguang Li. Rose Yu, Cyrus Shahabi, and Yan Liu. Diffusion convolutional recurrent neural net- work: Data-driven traffic forecasting, In International Conference on Learning Representations (ICLR 18), 2018.
[41] Aoyu Liu and Yaying Zhang. TeepprllTempora Dynamic Graph Convolutional Network With Interactive Learning for Traffic Forecasting. IEEE Transactions on Intelligent Transportation Systems, 25(7):7445-7660, July 2024. ISSN 1524-9050, 1558-0016. 3362145.
doi: 10.1 109/TITS.2024,
[42] Shixuan Liu, Jijun Zhu, Wei Lei, and Ping Zhang. Spatial-Tempont Attention Graph WaveNet for Traffic Forecasting. In 2023 5th International Conference on Data-driven Op-
timization of Complex Systems (DOCS), pages 1-8. IEEE. ISBN 979-8-3503-9352-1. doi: 10.1 /ttp/DDOSSS00972222.0022222
Utt. ieee hwrg/document//
10294485/
[43] S. G. Lobanov and O. G. Smolyanov. Ordinary differential equations in locally convex spaces. Uspekhi Mat. Nauk, 39:93-168, 1994.
[44] Haggai Maron, Heli Ben- Hamu, Nadav Shamir, and Yaron Lipman. Invariant and equivariant graph networks. In International Conference on Learning Representations, 2019 URL https
//openreview.. peeeareeieee.ne//ooumllill
[45] James Morrill, Patrick Kidger, Lingyi Yang, and Terry Lyons. On the choice of interpolation scheme for neural CDEs. Transactions of Machine Learning Research, 2022.
[46]
Pearlmutter. Learning state space trajectories in recurrent neural networks. In International Joint Conference on Neural Networks, pages 365-372 vol.2. IEBE, doi: 10. 10.1109/ICNNN.1199111188 URL [RLLitttp//ieeexplorre iees wtw..eeeeeeeeeeeeee.eeee wttp///mmmmmeetem111////////
[47] Michael Poli, Stefano Massaroli, Junyoung Park, Atsushi Yamashita, Hajime Asama, and Jinkyoo Park. Graph neural ordinary differential equations. CoRR, abs/1911.075322 2019. URL
http://arxivy org/ abs/
[48] Tiexin Qin, Benjamin Walker, Terry Lyons, Hong Yan. and Haoliang Li. Learning dynamic graph embeddings with neural controlled differential equations, 2023, URL https://arxiv
axg/pdf/2300 11800.113350 11354v1
[49] Tiexin Qin, Benjamin Walker, Tenry Lyons. Hong Yan, and Haoliang Li. Learning dynamic graph embeddings with neural controlled differential equations. IEEE Transactions Pattern Analysis and Machine Intelligence, 2025.
[50] Jason Rader, Terry Lyons, and Patrick Kidger. Lineax: unified linear solves and linear least- squares in jax and equinox. Al for science workshop at Neural Information Processing Systems 2023, arXiv:2331. 17283, 2023.
[51] J. Rapin and J.-R. King. Exca Execution and caching.
https://github.com
facebookressercch/ excaa 2024.
[52] R. Rico-Martínez, K. Krischer, I.G. Kevrekidis, M.C. Kube, and J.L. Hudson. DISCRETE- vs. CONTINUOUS-TIME NONLINEAR SIGNAL PROCESSING OF Cu ELECTRODISSOLU- TION DATA. 118(1):25 48. ISSN 0098-6445, 1563-5201. doi: 10. 1...0000008844442099966604. URL httph:tt:/ tandfonling com http///o/fff///ooe /ttp/////./////11/00//0906662000 000.00000009999993333333
[53] Emanuele Rossi, Ben Chamberlain, Fabrizio Frasca, Davide Eynard, Federico Monti, and Michael Bronstein. Temporal graph networks for deep learning on dynamic graphs. In ICML 2020 Workshop on Graph Representation Learning, 2020.
[54] James Rowbottom, Benjamin Chamberlain, Thomas Markovich, and Michael M Bronstein. Understanding convolution on graphs via energies. 2022.
[55] Benedek Rozemberezki, Paul Scherer, Yixuan He, George Panagopoulos, Alexander Riedel, Maria Astefanoaei, Oliver Kiss, Ferenc Beres, Guzman Lopez, Nicolas Collignon, and Rik Sarkar. PyTorch Geometric Temporal: Spatiotemporal Signal Processing with Neural Machine Learning Models. In Proceedings of the 30th ACM International Conference on Information and Knowledge Management, page 4564-4573, 2021. 2011.
14

---

[56] Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. The graph neural network model. IEEE Transactions on Neural Networks, 20(1):61-80, 2009. doi: 10.1 (O.9//NNN.00..000000
[57]
Alessandro Sperduti. Encoding labeled graphs by labeling raam. In J. Cowan, G. Tesauro, and J. Alspeccorr, editors, Advances in Neural Information Processing Systems, volume Morgan- neurips cc/paper.aa...c httppppprplllee//palpppppa fiw.s/ http:/apppprr9999/ Fi11///f33000770020clllld11113f3 Paper .df
Kaufmann, 1993. wtt/attss://pp//e//e/eoooo
[58] Fang Sun, Zijie Huang. Haixin Wang, Huacong Tang, Xiao Luo, Wei Wang, and Yizhou Sun. Graph fourier neural odes: Modeling spatial-temporal multi-scales in molecular dynamics, 2025.
URL httpttttts//aii......t http://axxxv...r/bbbbaa /411. 01600
[59] Benedict Aaron Tjandra, Federico Barbero, and Michael Bronstein. Enhancing the expressivity of temporal graph networks through source-target identification, 2024. URL htttp:///arat
org/abs/ 2411 03596
[60] Rakshit Trivedi, Mchrdad Farajtabar, Prasenjeet Biswal, and Hongyuan Zha. Representation learning over dynamic graphs, 2018. URL https://arxiv org/ http/bb//1880 1803.00030000000 .008..
[61] Ch Tsitouras. Runge-kutta pairs of order 5 (4) satisfying only the first column simplifying assumption. Computers & Mathematics with Applications, 62(2):770-775, 2011.
[62] Guido Van Rossum and Fred L Drake Jr. Python reference manual. Centrum voor Wiskunde en Informatica Amsterdam, 1995.
[63] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. 6th International Conference Learning Representations, 2017.
[64] Benjamin Walker, Andrew D. McLeod, Tiexin Qin, Yichuan Cheng, Haoliang Li, and Terry Lyons. Log neural controlled differential equations: The lie brackets make a difference. International Conference Machine Learning, 2024.
[65] Lu Wang. Xiaofu Chang, Shuang Li, Yunfei Chu, Hui Li, Wei Zhang, Xiaofeng He, Le Song, Jingren Zhou, and Hongxia Yang. Tel: Transformer-based dynamic graph modelling via contrastive learning, 2021. URL htttt:ttt///iiiiii org bargg/bbs/00 22tg//bbs/110..... 2105.07944.
[66] Yanbang Wang. Yen-Yu Chang, Yunyu Liu, Jure Leskovec, and Pan Li. Inductive representation learning in temporal networks via causal anonymous walks, 2022. URL htttts///aaxxi org/ abs/2101.05974
[67] Bing Yu, Haoteng Yin, and Zhanxing Zhu. Spatio-temporal graph convolutional networks: deep learning framework for traffic forecasting. In Proceedings of the 27th International Joint Conference on Artificial Intelligence, IJCAI' page 3634-3640, AAAI Press, 2018. ISBN 9780999241127.
[68] Le Yu. Leilei Sun, Bowen Du, and Weifeng Lv. Towards better dynamic graph learning: New architecture and unified library, 2023. URL http:: // /arxiv www..org//aoee 13047
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: In the abstract and introduction, we make the following claims that reflect the paper's contributions and scope:
a) Induce permutation equivariance into Graph Neural CDEs by projecting them onto permutation equivariant function spaces; foramlly prove this statement in the linear case
b) Demonstrate theoretically that our model is equivariant with respect to permutations and time-warps
Empirically validate our model on dynamic node-level tasks using synthetic experi- ments well as standard benchmark datasets
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss limitations anmd future work in Section 5.1
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms
and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best
16

---

judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: All theoretical results in the main text (Theorem 3.1 Proposition 3.2) are
proved Appendix [ All new results Appendices C and D are proved after their statement. All assumptions and auxiliary results are clearly stated.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
statement of any theorems. The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
All assumptions should be clearly stated or referenced
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide full and detailed description of all experiments in the main text in Section 4 with additional details in Appendix G An anonymised version of the source
code used to the experiments is provided at hhttps://aanoyynnn 4open. science/
x/ s/perm__o ltepp/eerii_aeee/eeeeeeeeepeeeee -dds BBF8/READDS /READEE, md
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
17

---

(c) If the contribution is a new model (e.g.. a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The TGB and Pytorch Geometric Temporal datasets used in Section are publicly available. The data generation procedures for the synthetic experiments are extensively described in Section 4 and Appendix G and part of the code base at
4open www....ppp..n......... www.sscceeeneeeeeeeeee.eeeeeeeee lecerrrmeeevvieee seuuiigg__aee -g_ BBFB/BBFDDRRRSEEE READMERAD....... md.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cem cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
//nips /nipps.ccppblle.
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: All chosen hyperparameters and the ranges we chose them from are detailed in Appendix G
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
18

---

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We repeat each experiment multiple times over different random seeds for data generation and model initialisations. We report either standard deviations or confidence intervals.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative eтror rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provide information about the type and amount of compute used for each experiment in Section G.I
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't it make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics /ttphng..htps////prp....../p///p
Answer: [Yes]
Justification: As this paper does not involve research involving human subjects or partici- pants, and all data has been properly standardised and anonymised. The work adheres to the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
19

---

If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We briefly discuss the societal impact of this work in Appendix J.1
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deploymentt of technologics that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path any negative applications, the authors should point it out. For example, it is legitimate to point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: This paper does not pose any of the mentioned risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
20

---

Answer: [Yes]
Justification: All datasets, programming languages and libraries used in this work have been attributed properly.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, wwwppsppppeaaaaeeeee thcode com datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
No new assets were introduced in this paper.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/mooel as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
Justification:
The paper does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
21

---

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: No crowdsourcing nor research with human subjects is part of this paper.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: No LMMs have been used for the method development of this paper.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our L.LM policy attt::://earrrpsccc/oooffeeeeeee for what should or should not be described.
22

---

A Permuation equivariant linear basis terms
Maron et al. [44 derive the 15 basis elements of all linear permutation equivariant maps L R g= on edge-valued signals A _R²xx They show the following maps span this space:
We denote by E RR" the vector whose entries are all equal to one, and by diag the operator that either places vector on the diagonal of matrix extracts the diagonal of matrix, depending context. With this notation, the basis elements of Ex (2, 1) can be listed explicitly as follows:
1. The identity and transpose operations: L(A) A, L(A) A²"
2. Elimination of non-diagonal elements: ((A) diag(diag( (A))
3. Sum of rows replicated on rows/columns/diagonl!!
L(A) 11TA, =
1(A1)"; L(A) diag( A1),
4. Sum of columns replicated on rows/colummnsdiiagonaa:: L(A) AT11",
1(A 1)² ((A) = diag(A 1),,
5. Sum of all matrix entries replicated on all entries/diagonal: L(A) ATA1 11 ((A)
A'A1- diag(1),
6. Sum of all diagonal entries replicated on all entries/diagonal:
L(A) 1 didag A)
11T [(A) 1ºt diag(A) diag(1).
7. Diagonal elements replicated rows/columns: L(A) diag(A)1" L(A) Idiag(A)"
B Projection theorem
Definition B.1 (Projection). Let H. be Hilbert space with inner product (..)). A linear map p: H - H is called a projection pf = p. A projection is called orthogonal if for all x E H. holds that for all H. p(y) 0 implies (p(x). 0.
Intuitively, this saying that applying a projection operator twice yields the same result as applying it once, which captures the idea of "selecting" or "filtering out" specific component of vector. The additional condition for an orthogonal projection ensures that the difference between any vector and its projected counterpart is perpendicular to the subspace, embodying the concept of the closest approximation in the geometry of the space.
Theorem B.2 (Projection Theorem, e.g. (1. Chapter 1). Let be Hilbert space and closed + u² where u E U, u E U = {y E H: (y, u) = Vu E U}. Moreover, the map Py - H defined as Po (x) = an orthogonal projection and satisfies
linear subspace. Then every E H can uniquely be writlen
[P)(x)) =|| inf 1| - z/|. w'EU
(9)
for all x E H.
The Projection Theorem asserts that every vector in a Hilbert space can be uniquely decomposed into sum of two parts: one lying in a closed subspace and the other in its orthogonal complement, thus clarifying the geometric structure of the space. Moreover, it guarantees the projection onto subspace the unique map that assigns to each element H the closest element from that subspace, thereby establishing rigorous method for optimal approximation in terms of the inner product.
C General theory of equivariance
In the following, we formalise notions of how symmetries can "act" on data and how we want to define functions that respect these symmetries.
C.1 Equivariance of static functions
Definition C.1 (Group Action). Let G be a group and X set. A group action of G on X
function : G x X + X satisfying:
23

---

1. Identity: for all x E X, e. = where e is the identity element of G,
2. Compatibility: for all g, E G and x E X, (gh) x = 9 (h x))
This definition formalises the idea that the elements of the group G can act on the set X in way that reflects the group's structure. It essentially captures the concept of symmetry by showing how each group element transforms or "moves" elements in X.
Definition C.2 (Group Representation), Let be a group and V vector space. group representa- tion of G on V is a homomorphism p G - GL(V) such that:
Identity: p(e) = Iv, where is the identity element of G and Iv is the identity map on V,
. Compatibility: For all 91,99 92 E G, p(g192) = p(g)))((2))
This definition realises abstract group elements conerete, invertible linear transformations on V, thereby representing the group's 's symmetry a linear framework. It provides a way to analyse the structure of G by studying how acts vector space.
Definition C.3 (Equivariance), Let p: G GL(V) and a G + GL(W) be representations of group G vector spaces V and W, respectively. A function f :V + W is said to be G-equivariant for all E and v E
(p(g)(v)) (f(O))
This definition states that applying the group action before after the function f produces the same outcome. Equivariance ensures that f preserves the symmetry structure imposed by G. making it a natural and symmetry-respecting mapping between the spaces.
C.2 Haar measure
Definition C.4 (Haar measure). Let G be a compact Hausdorff topological group. There exists unique regular Borel measure on G invariant under both left and right translations:
p(gE)= = (Egg)= = p(F) for all g E G, E C G Borel
with total mass 1, i.e. p(G) = call this measure the Haar measure.
Remark C.5. If G is finite of order [G), the Haar measure reduces to the uniform distribution:
[ (()) = G
p((g)) = G VgEG,
Σ f(g)-
gEG
In this discrete (hence compact) setting, Haar integration is exactly averaging group elements.
C.3 Projection of non-equivariant functions
Given Hilbert spaces V and W, the set of functions {f V W} form vector space under pointwise scalar multiplication and addition. Given group representations p and a of a group G over V and W, respectively, can consider the subspace of functions that equivariant with respect to these representations. The following lemma and corollary show what the projection onto this subspace looks like.
Lemma C.6. Let V and W be finite-dimensional vector spaces with an action of finite group G with representations p G GL(V) and G + GL(W) respectively. For any linear map T: V + W, define the averaged map by
Tavg (v) / (0)) (TT))T(((9g))) d
Then Tang is G-equivariant, and moreover, T is G-equivariant if and only if T Tavg.
Proof. To show that Tavg is G-oquivariant, take any h E G and E V. Then
/ a (((()))())) d
Tavg (avg(p(h)u)
j
[ ( (()))TT(p() du(g).
J

---

By substituting o' = gh (so that g g 1), we obtain
Tavg(p(h)u) [ o a du(g')
((1)0(())-TTpp du(g')
= o (h)Tavg())
where we used o(ght a(h)o(g')> because a is homomorphism. Thus, Tavg is G- equivariant. Next, if T is already G-equivariant, then for every EG,
a())~ -T()),
so that
Tavg( = /
JG T(v) T(o).
Corollary C.7. In the setting of the previous lemma, the map Qave Hom(V, W) - Hom V, W) defined by + Tawg the projection map onto the subspace of G-equivariant linear maps.
Proof. In the previous proof saw that (avg acts the identity the subspace of G-equivariant linear maps. Since applying (avg to any linear map T yields a G-equivariant map Tavg- Oavg is projection onto the subspace of G-cquivariant linear maps.
Next, we study what form the projection the composition of a linear equivariant function with another function takes.
Proposition C.8. Ler V, W be given by representations pV : + GL(V), px G - GL(X) and pw G GL(W) respectively. Denote the projection of Hom(V, W) onto the space of of G-equivariant functions with respect the representations pV and pw by Pv,w and similar for the other combinations. Now suppose we have functions T V W.R.
vector spaces with an action of a finite group G
S,S X W such that
T S R,
S G-equivariant and linear.
We then have Pv,((T) S Pv.x
Proof. Since Pv.w projects onto an equivariant subspace, by the above we know that
(Pv.wT)(u) = pw ((o) du(g)
for all U E V. Now by definition and the linearity of S,
(Pv.w)(()) / pw (S (g)
[ ((xy))))((n (equivariance of S)
(f. R())
(linearity of S)
((Pv.x)))
since S (0000 by the equivariance of S. This concludes
= - (w())
the proof.
25

---

C.4 Equivariance in Neural ODEs
Definition C.9. Let f: X X be a smooth function, where an interval containing and X is some Banach space. For each x E X, consider the initial value problem
dz dt f(ztt,
x(0)= =t.
(10)
The flow of the ODE the mapping x Xx, (f,z) x) the solution to the IVP in equation 10 with initial condition x evaluated at time t.
which assigns each pair
Proposition C.10. Let (x(z) be the flow of an ODE with vector field f X x 1 - X and let P be the projection operator of the space of functions {g X + }} onto the space G-equivariant maps with respect to a group representation p of G on X. Then the projected flow P( and the flow de of the projected vector field P(J) x X agree for all times E I, i.e. P(D:) = =dr-
Proof. First, note that by Corollary C.7
P can be realised by
(()) Jo (()
(().)(=)= - ()
Now, by the uniqueness theory of ODEs (see for example Theorem 6 in (3)), it suffices to show that P(D,) satisfies the same initial condition and ODE as p,, To that end, we calculate
d (p)
[ p(g) d (((()))))
|(o)) (100))))
= P(f)(x,tt
and
(x) = (b(()))())
J
p(g))' p(g)ad
This finishes the proof.
This theorem shows that, under the stated conditions, projects the full flow 4, using P. then this projected flow is exactly the same as the flow obtained by integrating the projected vector field P(f).
D Application to Graph Neural CDE
Definition D.1 (Representation of the Permutation Group). We define representation of the permu- tation group Sn. on the space of node feutures Roxd for each a E Sn by pla)(X) P.X where Pa is the permutation matrix associated with a. Similarly, we define representation on the space of node features ax by π (X) = P.XPT.
The pre-multiplication by Pa permutes the rows of a matrix, and the post-multiplication permutes the columns.
Firstly, by Proposition C.10 the projection of any Graph Neural ODE onto the subspace of permutation equivariant functions is the Neural ODE where the vector field is the projection the Graph Neural ODE, Hence, suffices to consider the projection of the integrand in Equation 4
Proof of Theorem 3.7
26

---

We can decompose this integrand as the composition of the following two functions:
R(Z, A, dA
W(DR) Z, W
A
ds
AA
ds
S(Z, A) (Z()
where ( are latent representations obtained by iterative convolutional process of the form Z (((--) for E [l,..., where the w are learnable matrices.
The function S is a standard convolutional GNN and hence both permutation equivariant and linear C.
in Z. Hence, can apply Proposition C.8 and realise only need to consider the projection of R.
But since W/DR) is learnable and in Section 2.3 we characterised the space of linear equivariant functions on the space of adjacency matrices, instead of projecting onto the space equivariant functions, we may directly parameterise an element of this space as linear combination of the basis elements. This concludes the proof.
E Proofs
Proof the GN-CDE (Equation) is not permutation equivariant. For notational simplicity, we flat- ten fo A,) and the control dÅ, into tensors of ranks and 1, respectively:
Fo 5: A,) vec fo A.)]
(11)
B vec [c.] 2n
(12)
Under a node permutation a E Sn, with permutation matrix Po. these transform
Fo A
+ (Po x [a.) Fe A.), B (P P. 1 B.
(13)
(14)
Requiring that the contraction
(Z. A.) Fe (Z. A.)
be permutatioon-equivrriant then gives
T' (P, Z., P. A.P. = (P, 2 la.) (Z. Aa) (P. )
@1) VoeSp
No general architecture for So (for example, standard multilayer perceptron) can satisfy this con- straint unless it is explicitly parametrised to output elements in the space of permutation-equivariann linear maps.
Proof the GN-CDE fusion operation (Equation is not permutation equivariant. Firstly we note that by writing (F))
w
(
for W
RRxn the fusion takes the form w(FF
A, = (F) (F)
w
w
(F)
dA
A
=
(F) A, + ( w
dA. ds
(15)
ds
ds
Thus, the fusion operation can viewed as pre-multiplying each of A, and dAg weight matrix and then summing the result. However, letting P be arbitrary permutation matrix, we have
de by learnable
PA.P" PW(F) (w(()A,P'' (F) dA. PT ds
(16a)
2
d ds
PA
P
(16b)
in general, since W! ((F) and W do not necessarily commute with P.
27

---

Proof of Proposition 3.2 First we show permutation equivariance which follows immediately from the construction of the linear fusion. For let be an arbitrary permutation matrix. Then
dA. pT) ds
dAA
LPPAPP + L2
(LI(A.) + +L2
p'
PA.P"
(17)
ds
Since Ço is implemented as GNN, So (PXt, PA P') PCo(Xeo Ato) all t E (to: tN| we have
pit, and so overall for
PZu + ( PZ
PZzo f Po(A, W('))ds
/o((A.Z!DW())d
(18a)
PZro + (AAZ ((())d
(18b)
Zio + I oAA, (.
(18c)
to
PZ.
(18d)
using the orthogonality of P and linearity of the integral. To show time-warp equivariance, suppose latent path Z: [0, T] - Rexd, satisfies
dz
dA
( Zo.
(t) fo(Z(t) (t))
di (t).
(19)
di
for some dynamic graph data A [0, T] + 1axn Then the warped path Ż = O satisfies
( (Z (0) Z(0) Zo.
(20)
and by simple application of the chain rule get
až dt
dz (T(t))+(() dt
(t) - (zo
(21a)
dt
= =I)((((()). dÀ d =)((((), (A (Ao
di (T(1))7(11
(21b)
(21c)
dt
which is what we wanted.
Proof of equivariance in Section
3.3 First we show that for general A E Rexmxd and B Raxd
we have P(A @ B) = PA @ PB. Einstein notation we have
(P(A B)}, = Pa & B)
(22a)
Σ AR
(22b)
At the same time
(PA PB), (PA) ji (PB)
(23a)
Pa Pu Bel
(23b)
= PaAAA
(23c)
(PAA B)};
(23d)
because Pa. Pi E 1), so the only terms contributing to the sum are those where P.k which happens if and only if k: k as P is permutation matrix.
Par
Hence, we see that the vector field equation permutation equivariant. By the above, we can conclude that the entire model is equivariant.
28

---

F
Additional Experiments
Here, we present the results of the additional experiments on personal capital and opinion dynamics.
Table 5: Additional comparison of GN-CDE variants and baselines the wealth (top) and opinion dynamics (bottom) tasks. Mean MSEs with 95% confidence intervals are reported, with the best mean highlighted in bold, and all results within the corresponding confidence interval are underlined. The final row in each table reports the relative improvement of PENG-CDE over the original GN-CDE formulation.
Wealth Dynamics (MSE !)
Model
Community
Grid
Power Law
Small World
DCRNN 40
2.639 + 1.981
22.619 + 17.115
113.564 +: 236.120
200.486 + 280.387
STIDGCN
1.862 + 0.947
7.296 = 2.547
2.124 = 1.012
3.133 + 1.286
ASTGCN
26
3.431 + 1.224
19.694 2.986
5.153 = 1.184
9.246 + 2.204
STG-NCDE
36.087 = 2.454
44.677 3.755
37.148 = 2.742
41.046 = 2.205
Constant
2.504 + 0.473
14.157 1.746
3.682 + 0.873
7.445 + 1.200
Graph Neural ODE 44
0.878 + 0.388
1.898 + 1.342
0.950 = 0.660
0.940 + 0.185
Adjacency GN-CDE
0.904 = 0.493
2
1.075 = 0.847
1.297 + 0.765
Pre Mult GN-CDE
7.740 = 1.122
24.522 6.594
39.715 = 32.698
16,568 + 6.902
Original GN-CDE
1.577 + 0.781
6.582 + 3.273
2.406 1 0.965
3.076 + 1.193
PENG-CDE (ours)
0.522 = 0.316
4.273 = 3.818
0.863 = 0.330
0.813 + 0.627
Relative Improvement (%)
66.89%
35.09%
64.13%
73.57%
Opinion Dynamics (MSE 4)
Model
Community
Grid
Power Law
Small World
DCRNN 40
20.659 + 32.149
92.165 = 132.026
91.687 = 81.638
173.554 + 183.913
STIDGCN
2.434 + 1.562
6.977 2.451
2.724 # 1.530
3.933 + 1.567
ASTGCN 26
22.001 = 1.519
34.855 2.370
22.517 + 1.490
27.439 + 1.554
STG-NCDE
57.495 9.245
85.405 6.479
59.446 + 9.294
68.092 + 8.988
Constant
22.989 = 1.372
33.124 1.275
23.775 1.480
27.239 1.929
Graph Neural ODE [47
0.665 = 0.620
2.338 + 1.277
3,073 = 6.114
0.779 = 0.417
Adjacency GN-CDE
0.777 + 0.798
1.912 + 0.566
0.802 + 0.652
0.909 + 0.416
Pre Mult GN-CDE
38.717 = 6.229
64.825 + 14.066
61.226 + 17.714
88.503 + 74.608
Original GN-CDE
1.060 + 0.436
6.963 + 4.698
6.781 11.159
5.223 1 4.240
PENG-CDE (ours)
0.525 1 0.581
1.851 1.488
0.674 1 0.606
1.161 1 6.011
Relative Improvement (%)
50.52%
73.41%
90.07%
77.77%
G Implementation details
We will release all code alongside the camera-ready version of this paper. The implementation is written in the Python programming language (62] and uses the JAX framework [4]. Key dependencies
include Diffrax [34] for diffecrentiabee ODE solvers, Equinox 33) for neural network modules in JAX.
Optax [4] for optimisation, and Lineax [50] for linear algebra routines. Additional dependencies
include NumPy 28 Exca [5]]. PyTorch (55), and PyTorch Geometric 23
All differential equation-baseee based models use the Tsitouras' 5/4 method
61 as implemented in Diffrax.
G.1 Compute resources
All experiments were conducted on computing cluster equipped with NVIDIA H100 (80 GB) and H200 (141 GB) GPUs. Each node in the cluster features 64-core AMD EPYC 9334 CPUs running at 3.90 GHz, along with 256 GB of RAM. The heat diffusion and gene regulation experiments described
29

---

in Section 4.1 required approximately 3.5 GPU days to complete, whereas the oversampling and irregularity experiments finished within total of 12 GPU hours. The PyTorch Geometric Temporal experiments ran for approximately 8 hours. Running STG-NCDE, GN-CDE, and both versions of our model on the two Temporal Graph Benchmark tasks required around GPU hours in total.
G.2 Heat diffusion and gene regulation
Two important examples of dynamical systems on networks are heat diffusion and gene regulation dynamics, both of which describe the local exchange of information state between connected nodes. While heat diffusion models symmetric spreading (c.g., temperature equalisation), gene regulation introduces asymmetric, nonlinear interactions typical of biological systems. These processes can be modelled as follows:
d (t.(t) = Lixu(t) =
xx(()
Heat Diffusion:
Σ
(24)
dt
duu
vEN.(()
d (..( -xu(fff
X (1)
Gene Regulation:
Σ
(25)
(t) +1
vENu(()
where (1) is the temperature of node u at time I, (t( denotes its neighbourhood, and L. the normalised graph Laplacian.
To extend our evaluation to economic networks and social interactions, we additionally include the following two dynamical systems, in which (t) models the personal capital opinion of node at time t, respectively:
d xxx(() (1)
Wealth Dynamics:
8; (1)0..
Σ v
(x)
x,) + (x,,
(26)
xx (t) (1. threshold
Opinion Dynamies:
Σ (1), PEN
(27)
Here, threshold denotes the thresholding function defined as threshold (z. y) = 1 iff æ 2 y and otherwise.
G.3 Synthetic experiments
To ensure a fair comparison, the vector fields all differential equation-based models are imple- mented using a two-layer GCN [38] with a hidden dimension of 16, and layer normalisation is applied. The models are trained for 2000 epochs using the Adam optimiser [37 with learning rate of 10 and weight decay of 10
G.3.1 SIR model
The Susceptible-Iniected- Recovered (SIR) model on graphs 33 is network-basee extension of the foundational compartmental model in epidemiology [32 that describes how infectious diseases spread through structured population. It partitions nodes in graph into three groups: susceptible (S,), infected (I.), and recovered (Ry) for each node v, with transitions governed by the system dS. dt
dl,
dR,
3S.
Σ Ins
BS,
Σ
1
7/v-
(28)
(EN()) di
di
wEN()))
where 3 is the transmission rate, y the recovery rate, and ((v)) denotes the neighbours of node in the graph.
A key insight from the graph-based SIR model is that the epidemic threshold depends not only on the infection ratio R 3/7, ,, but also on the structure the underlying graph, such as its spectral radius degree distribution. If local connectivity supports sufficient transmission, the infection can spread widely; otherwise, dies out. Despite structural complexity, the graph SIR model preserves essential features of epidemic dynamics and extends the classical model to networked populations.
30

---

G.3.2 Oversampling
To study the dependence of performance and compute time on the number of observed samples, construct grid graphs with 100 nodes and generate irregularly sampled time series between 7 -'=0 0
and T I, using n 20, 40, 60,
200 observations respectively. The dynamic graph topology is
generated as described in Section 4.1 Trajectories are produced by numerically solving Equation 28
with random initial conditions. We use two parameter settings: (31 = 0.25, "1 0.7), modelling a scenario where the epidemic dies out, and (32 = 0.3, = 0.3), modelling a scenario where the infection spreads. For each setting, we generate 50 trajectories each for training, validation, and test sets.
The task is perform binary classification, predicting whether a given trajectory corresponds to outbreak (i.e., sustained spread of infection) or a non-outbreak (i.e., rapid die-out).
We train the models using binary cross-entropy loss for 2000 epochs, applying early stopping with a patience of 200 epochs and a minimum of 200 epochs. Training is performed using the Adam optimizer [37] with learning rate of 10 2 and weight decay of 10 44 The model hyperparameters are summarised in Table
Table 6: Hyperparameters for SIR experiments.
Perm Equiv GN-CDE
DCRNN
Hidden Dimension
Number of Layers
3
Data Embedding Dimension
Layer type Chebyshev Order
GCN
G.3.3 Irregular observation spacing
To assess how the regularity of observation times affects model performance, we generate classifica- tion data exactly as before, except that now sample the irregular observation time-stamps from a Gamma-drives process. In more detail, let N be the total number of desired time- time-points on the interval (0, 7J. We begin by drawing N 1 independent inter-arriall increments
At, Gamma(k: 0), i= N -
where t: controls the shape (and hence burstiness) and 0 > 0 scales the raw increments. force the observations to span exactly [0, TJ, we normalise the increments by their sum:
AG Σj=1
At. =
T,
t,
At..
At,
j=l
with l = 0 and IN T. This construction guarantees 0 = to 4 < IN T and yields exactly N strictly increasing time-stamps.
In our experiments, we set T = 1 and draw N 20 observations per trajectory using this Gamma- based sampler for k {3.5 5, 10, 25,, 50, 100}. All other aspects of model training network architecture, optimisation schedule, and hyperparameters remain identical to those in the previous section.
G.4 Real-world experiments
All models are trained over 200 epochs with an early stopping criterion with patience of 15 and minimum epoch 20. all applications, we utilise cubic splines (34 Chapter 3.5] for interpolation of the data. All hyperparameters have been selected from the ranges in Table grid search. For the chosen values see Tables and TO All experiments have been averaged over 10 random seeds.
31

---

Table 7: Ranges the optimal hyperparameters (PENG) GN-CDE (top) and STG-NCDE (bottom).
Range
Hidden Dimension
[8, 16, 32, 64) (2,3, 4]
Number of Layers
Data Embedding Dimension
{8, (8,16, 32, 64)
Number of Recurrent Layers
{1,2,3) 3} {8, 16, 32, 64)
Recurrent Hidden Dimension
Number of GNN Layers
{1,2,3)
GNN Hidden Dimension
{8, 16, 32, 64)
Data Embedding Dimension
{8, 16, 32, 64) [2)
Chebychev Order
Node Embedding Dimension
Table 8: Hyperparameters for real-world experiments for the GN-CDE model.
tgbn-trade
tgbn-genre
twitter-tennis
england-covid
Hidden Dimension
8
32
Number of Layers
2
2
Data Embedding Dimension
16
8
Layer type
GCN
GCN
GCN
GCN
G.5 Source target identification
In [5)]. the authors address the well-known issue that machine-learned models underperform on node affinity prediction tasks compared simple heuristics [53] by modifying the message construction in TGN [53] as follows: Given an interaction between nodes u and v at time t with associated data
euv(t), the messages m. (t) and (t) sent to nodes and v, respectively, are defined as
m. (1) = msg, (s.. (1T) ), S, (1 dr. (At)) en (1), on (u), on (v)).
(29)
m. (L) (s. S. ((T), dr (At), Gno (1), dn (v), (n)).
(30)
where At is the time elapsed since their last interaction and d R - R" is an encoder function for node indices. In their work, the authors use (uu) (cos(w,u))
Inspired by this, we incorporate source-target identification into our equivariant fusion by transforming the adjacency matrix A, and its derivative d entry aue of either matrix is passed through MLP together with the encoding of both u and to form a new matrix B with entries
de using an attention-style approach. Specifically, each
MLP (awella (oL]))
(31)
where denotes concatenation. We selected the following hyperparameters: embedding dimension of n = 512, an MIP width of 8, and 2 hidden layers. We slightly deviated from the original approach by defining ((u) MLP(u) with hidden dimension of and 2 hidden layers.
Table 9: Hyperparameters for real-world experiments for the Permutation Equivariant GN-CDE model.
tgbn-trade
tgbn-genre
twitter-tennis
england-covid
Hidden Dimension
32
16
64
Number of Layers
3
3
Data Embedding Dimension
8
Layer type
GCN
GCN
GCN
GCN
32

---

Table 10: Hyperparameters for real-world experiments for the STG-NCDE model.
tgbn-trade
igbn-genre
twitter-tennis
england-covid
Number of Recurrent Layers
3
3
Recurrent Hidden Dimension
16
32
32
Number of GNN Layers
2
1
2
3
GNN Hidden Dimension
16
32
32
Data Embedding Dimension
32
8
Chebychev Order
Node Embedding Dimension
5
5
5
Table 11: Runtime per training epoch (s) for GN-CDE and PENG-CDE across different node counts in Section 4.1
Model
128
256
512
1024
2048
Pre Mult GN-CDE 48
0.45
0.56
1.18
1.62
9.15
PENG-CDE (ours)
0.42
0.49
0.57
0.90
1.50
Rel. Improv. (%)
7.1
14.3
107.0
80.0
510.0
H Statistical analysis of Experiments
In this section, we conduct statistical verification of the relevance of our experiments. Note that in Section 4 whenever possible, we reported means along with 95% confidence intervals. This applies all synthetic experiments and the PyTorch Geometric Temporal datasets reported in Tables T and in the main text, For the Temporal Graph Benchmark datasets, baseline results were only available as mean NDGC@ 10 scores with standard deviations, so we adopted the same format for consistency,
To address statistical significance testing, we employ critical difference diagrams, based on a two-stage procedure:
A global Friedman test was used to determine whether any model differences were statisti- cally significant.
2. If the null hypothesis was rejected, we performed pairwise comparisons using the aligned Friedman post-hoe
We present the critical difference diagrams for the dynamical systems, Pytorch Geometric Temporal and Temporal Graph Benchmark datasets in Figures 3 and respectively.
We can see that in all synthetic experiments (Figure 3), our PENG-CDE consistently ranks within the top non-significant clique, which demonstrates its strong and stable performance. For the PyTorch Geometric Temporal datasets (twitter-tennis. england-covid; Figure 4 the global Friedman test was inconclusive, meaning the test did not reject the null hypothesis that model (even the baselines) statistically different from the others. On the Temporal Graph Benchmark, the critical difference analysis (Figure 5) shows that PENG-CDE performs significantly better than both GN-CDE and STG-NCDE.
I Runtime analysis
Runtime, To highlight the computational benefits of our approach, repeat the experiments from Section 4.1 with varying numbers nodes and record the time per training epoch in Table [T] Our approach exhibits significantly better runtime performance scaling, as the number of learnable parameters in the fusion matrices W, the Pre Mult GN-CDE scales quadratically with the number of nodes, whereas the parameter count of PENG-CDE is independent of graph size.
33

---

Loss
5
PermEquiiKraagteeecloffffffl
12377
TreekutFiusiorr@raaWeeeerrrrr
GNODEEFicoVecdoFiidd
22131
5,005
ConstVectorFleld
GNODEVectorFiel
23037
GraphVectorField
(a) All dynamical systems.
Loss
5
GNODEVectorFiel
PraMuuFFuuäonrrppWWeettraaad
PermEuivGragtVedorrfild 20000
ConstVectorFiele
GNCDEFFoooVeetorFieed
23071
GraphVeclorField
(b) Heat diffusion dynamics.
Loss
GNODEVectorFiel
18356
PermEqulKrraniVeettriied
ConstVactorField
GNODEFRoorVecooFiid
23571
3.7143
GraphVectorField
Gene regulation dynamics.
Loss
5
6
PermEquivrragiieecooFedd GNODEFFooWectoriide
14736
PreMulFuusionen PronkulFFusionrrapWVettoriia
2,320
50002
ConstVectorFiet
GNODEVectorField
GraphVeetorField
(d) Consensus opinion dynamics.
Loss
PemEEgiiGrapNVectofiid GNDDEFRooNeedoFFied
17215
TreMuutFisionGrappNeeeiee ConstVectorField
19421
GNODEVectorFide
15786
9M1? GraphVectorField
(e) Personal capital dynamics.
Figure 3: Critical difference diagrams for the synthetic experiments in Section 4.1 and Tables 1 and
Miscellaneous
J.1
Societal impact
This work constitutes foundational research and is not tied to any specific application or deployment. As such, it shares the general risks and benefits inherent novel machine-learning architectures. For example, dynamic graph representation learning models have traditionally been applied to tasks such traffic forecasting [1T [10 and molecular modelling (58), and hope that the advances presented here will further drive progress in these and related areas. Overall, we assess the societal impact of this work be predominantly positive.
34

---

Loss
1
2
5
6
ConstVectorFiel Fraph
1.700
astgon
5.6833
STCraphVeettrFieed
GNODEFloor #NoDEFllooVecttoFieel dornn
1.00
4,2550
stid_gen
(a) Both PGT datasets.
Loss
2
5
6
FiaphVectorField 35754
astgon
GNODEFlcoVVectorFii 33371
5,142
stid gon
3
5,1428
STGrapiVectorField
FonstVectorField
4.0000
PermEqqiivGnappheectrrrii
(b) Twitter-tennis dataset.
Loss
5
stid
astgon
ConstVectorFFeld
3ams
6.2030
STGraphVectorField
GNODEFlaarVectorFiel
4200
PermEquivvGaap Vecteeiilir
domn 42000
4.000
GraphVeetorf
(c) England-covid dataset.
Figure 4: Critical difference diagrams for the Pytorch Geometric Temporal (PGT) real-world experi- ments in Section 4.2 and Table
35

---

Loss
2
3
7
PermE PeemEguiGrraNNVecoofeeee himmEpuihhiiGnnpheeotoriiiid
T 2,0005
STGraph/ectorfied
23300
ConstvectorField
PermE:uuNGrapiieetooff GraphiVectorField
$3000
19902
PemnEqukDDDGGegeVVeeeooaaalll
3.200
Both TGB datasets.
Loss
2
PermEquivGrapiieettofiiee 1000
2.6424
SeGrorh
GraphVectorField 2.1178
(b) Both TGB dataset (reduced set of models).
Loss
2.000
37000
STGraphVectoriiel
2320 GraphVectorField
6.3002
ConstVractoFFiel
32200
(c) Trade dataset.
Loss
PemfEuiiGrapNeecoofidddt 2.000t
2.7000
STGraphNaactoFiees
Samm@juiiDi@@mmphetteeiiids 2.100
6.3302
ConstVeectoFFled.
PermEquii@raphieeeoorre GraphiVettorFeedd
(d) Genre dataset.
Figure 5: Critical difference diagrams for the Temporal Graph Benchmark (TGB) real-world experi-
ments in Section 4.2 and Table
36


Reviews:
[{"review_id": "FyBIy0m8QE", "reviewer": "Reviewer_G4dS", "summary": "This paper introduced a permutation equivariant graph neural controlled differential equation (PENG-CDEs) by leveraging 2nd-order linear mapping between the adjacency matrix (Equation 5). In synthetic and real-world datasets, it showed improvements over previous methods like GN-CDEs.", "questions": "1. For experiment 4.1, there's only one change in the graph structure in the validation and test phase. Is it enough to test the generalization?\n\n2. In what scenarios is time-warp equivariance important? And why?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**:\n1. Clarity. The method described in the paper is clear and sound.\n2. Experiments. Experiments are relatively solid, with diverse settings and dataset and a strong choice of baselines. The improvement after adding the full linear equivariant functions over original GN-CDE is notable. \n\n**Weakness**:\n1. Significance: It is a pretty standard application of the well-known linear equivariant function w.r.t. $S_n$ to the dynamic graph setting. It only considered a simplification (Eq. 4) of the original graph neural CDE formulation (Eq. 3) and makes it equivariant w.r.t. node relabeling. This downgraded the significance and novelty of the proposed PENG-CDE.", "was_revised": true, "final_justification": "The questions I raised have been resolved. I still maintain my point about the paper; it's an improvement in graph neural CDE, however, the way it improves the original GN-CDE is a straightforward application of permutation-equivariant theory from standard graph neural networks. Given the potential impact this work might bring to the field of dynamic graph learning, I will weakly recommend accepting this paper, and I maintain my score of \"4 Borderline Accept\".", "rebuttal": "We thank the reviewer for their constructive and helpful review. \n\n**[W1]: Limited Significance**\n\nAlthough the permutation-equivariant linear map we employ is well-known, we do not believe our contribution is limited to a routine application. Theorem 3.1 proves that projecting the GN-CDE vector field onto the equivariant subspace is the optimal choice, establishing new theory for dynamic-graph CDEs. Empirically, this projection delivers large practical gains: on our synthetic benchmarks, PENG-CDE uses only 2% more parameters than GN-CDE (see Table 1 below) yet reduces average MSE by between 30.44% and 73.84%. Compared to the Pre-Mult GN-CDE variant, PENG-CDE reduces the parameter count by over 99%, while still achieving 8.56× to 27.65× lower average MSE. Finally, our method belongs to a family of models that are unique in their approach to tackling graph dynamics. Most other approaches are permutation equivariant due to handling the graph structure using an existing graph neural network structure. Therefore, introducing the first permutation equivariant graph neural CDE represents an important step for this class of models. We believe this represents a meaningful advancement in the design of continuous-time models for dynamic graphs.\n\n**Table 1: Comparison of number of parameters for GN-CDE, its Pre Mult version and PENG-CDE on our synthetic experiments**\n| Model           | No. of parameters |\n| --------------- | ----------------- |\n| GN-CDE          | 1,741             |\n| PENG-CDE (ours) | 1,771             |\n| Pre Mult GN-CDE | 641 k             |\n\n\n**[Q1]:  Change in Graph Structure during Validation**\n\nIn all synthetic experiments in Section 4.1, the underlying graph topology changes at 12 uniformly random snapshots out of the total 120 time steps. This implies that the expected number of topology changes in the validation window (20 snapshots for interpolation validation, 20 for extrapolation) is 40 / 120 × 12 = 4. As shown in Figure 1, where vertical gray dashed lines indicate the timing of topology changes, 3 changes occur within the extrapolation regime of the validation set in this setting. We consider this number sufficient to assess the model’s ability to generalise to evolving graph structures. For testing, we generate entirely new time series realisations, each containing 12 randomly sampled topology changes. This ensures that the test set evaluates generalisation to unseen sequences and topological dynamics. We have modified Section 4 to make this clearer.\n\n**[Q2]:  Importance of time-warp equivariance**\nTime-warp equivariance is important in time series modeling when the task depends on the sequential structure of the signal rather than the absolute timing of events. For example, in classifying whether a trajectory draws the digit “6”, it is irrelevant how fast or slowly it is traced, as long as the shape is preserved. Similarly, if you consider classifying the nodes of a social media network as to whether they are friends with a specific node or friends-of-a-friend with that node. The important characteristic is the order in which the edges appeared (you can't connect with a friend-of-a-friend without first connecting with the friend) and not the time in-between those connections appearing. We will emphasise the motivation for this type of equivariance more clearly in the camera-ready version, as well as detailing how you can avoid it by simply including time as a channel in the driving path when you believe that your problem is not time-warp equivariant."}, {"review_id": "yo4pUm0y8R", "reviewer": "Reviewer_GJRF", "summary": "This paper introduces Permutation Equivariant Neural Graph Controlled Differential Equations (PENG-CDEs), a novel framework that extends Neural CDEs to dynamic graph settings while preserving permutation equivariance. The key innovation lies in projecting Graph Neural CDEs onto permutation equivariant function spaces, significantly reducing parameters without compromising expressiveness. The method demonstrates superior performance on both synthetic benchmarks and real-world tasks, particularly excelling in interpolation and extrapolation scenarios across various graph dynamics.", "questions": "How does the computational complexity of PENG-CDEs scale with graph size compared to traditional Graph Neural CDEs, especially in large-scale real-world applications?\n\nCould you elaborate on the theoretical guarantees regarding the approximation power of the proposed equivariant function space projection?\n\nWhat are the practical limitations when applying this method to highly sparse or rapidly evolving graph structures?\n\nHave you considered extending this framework to handle discrete-time graph sequences or hybrid continuous-discrete dynamics?", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The paper presents a theoretically grounded permutation-equivariant neural CDE framework that demonstrates strong performance across synthetic and real-world benchmarks, achieving up to 73.84% improvement in predictive accuracy while maintaining computational efficiency through its parameter-reduced design. While the methodology shows impressive empirical results (outperforming baselines on TGB benchmarks by 5-67%), its reliance on dense adjacency matrices limits scalability to very large graphs, and the theoretical guarantees remain confined to linear cases. The work makes significant conceptual contributions by bridging geometric deep learning with neural differential equations, though broader impacts would benefit from validation on industrial-scale problems. The clear exposition of degeneracy modes and mitigation strategies enhances reproducibility, while the modular design enables straightforward extension to other equivariant architectures.", "was_revised": true, "final_justification": "PENG-CDE delivers consistent performance gains across benchmarks and provides clear theoretical grounding, although questions persist about its behavior on highly sparse or rapidly evolving graphs.", "rebuttal": "We thank the reviewer for their time invested in reading the paper and for their constructive and helpful review. \n\n**[Q1] Computational complexity compared with original Graph Neural CDEs**\n\nWe appreciate you highlighting this aspect of our model, since it was one of the key motivations for introducing PENG-CDEs.\n\n**Parameters:**  The number of parameters in the PENG-CDE is independent of the number of nodes  $n$  or edges  $e$, whereas in the original GN-CDE, the parameter count scales quadratically with  $n$. This makes the original formulation unsuitable for large-scale graph problems and negatively impacts runtime complexity.\n\nAs shown in Table 1 below, even for a modest graph size of 2048 nodes, our model trains  **6.1× faster per epoch** on the experiments in section 4.1 compared to the original GN-CDE. We have now clarified these differences in Section 3.2.\n\n**Table1: Runtime per training epoch (s) for the GN-CDE and our PENG-CDE, for experiments in Section 4.1 for different graph sizes.**\n|Model\\Num. of nodes | 128  | 256  | 512   | 1024 | 2048 |\n|--------|------|------|--------|------|-------|\n| (Pre Mult) GN-CDE    | 0.45 | 0.56 | 1.18  | 1.62 | 9.15  |\n| PENG-CDE (ours)   | 0.42 | 0.49 | 0.57| 0.90 | 1.50  |\n\n\n\n**[Q2] Theoretical guarantees for approximation power of equivariant projection**\n\nIn terms of the statement of Theorem 3.1, we know that projecting an element in a vector space onto a subspace will give us the closest element in that subspace (see Appendix A for a more formal treatment of this). We use this notion to argue in Theorem 3.1 that (assuming the absence of non-linearities) PENG-CDEs are the best possible permutation equivariant approximation to the GN-CDE. Furthermore, [1] show that GN-CDEs are universal.\n\n**[Q3] Handling of rapidly evolving and sparse graph structures**\n\nBefore training and inference, the provided graph snapshots are interpolated into a continuous driving path (which depends on the number of observations in the sequence). Once this preprocessing has been done, both the complexity of the forward and backward passes are decoupled from the number of input observations. Indeed, the number of vector field evaluations depends on the ODE solver rather than the sampling rate. This makes our model inherently robust to oversampled input data. For more details and experiments on this, we kindly refer the reviewer to Section 4.1 and Figure 2.\n\nWhen the graph structure itself evolves rapidly, any time series model will require a large number of observations to perform well. However, since the PENG-CDE is formulated as a controlled differential equation, it offers a distinct advantage over conventional time series models: it allows us to tap into the differential equations literature to understand how rapidly changing data will impact our model, by analysing the bounded variation norm of the input path.\n\nRegarding sparsity, we acknowledge that in its current implementation, just like the original GN-CDE, PENG-CDEs have quadratic memory complexity in the number of nodes due to the need to store dense adjacency matrices. This limitation, which we outline in the Limitations section, restricts scalability to problems with a large number of nodes. This constraint arises primarily from  JAX’s limited support for sparse matrix operations. With future versions of JAX moving its sparse support beyond the experimental stage, we will be able to address this  scalability bottleneck  is an important direction for future work. As shown in Table 1, once memory allocation constraints are met, PENG-CDE scales much more favorably in runtime compared to GN-CDE - demonstrating its potential for efficient execution, even at larger scales.\n\nWe emphasise that for a fixed number of nodes, the memory footprint of PENG-CDEs is independent of the number of edges. This enables us to scale (within the same order of magnitude) to the largest dataset in the Temporal Graph Benchmark [2] in terms of the number of edges. For an overview of the edge counts in these datasets, we refer the reader to Figure 1 in [2].\n\n**[Q4] Extension to discrete-time graph sequences or hybrid continuous-discrete dynamics**\n\nBy construction, CDEs - and by extension, PENG-CDEs - are well-suited to handle all these scenarios. While Sections 2 and 3 present the framework assuming continuous observations, discrete-time graph sequences can be naturally incorporated by interpolating the observations, for example using cubic splines. If the evaluations of the control path are ought to remain in a discrete domain, one can employ rectilinear interpolation, as discussed in [3].\nWe note that all real-world experiments in Section 4.2 operate on discrete-time graph dynamics, which we handle precisely in this way, as described in Section 2.2. Hybrid continuous-discrete settings can also be accommodated straightforwardly: discrete-time regimes or features can be interpolated, while continuous ones are used directly. \nWe agree that this point could be emphasised more clearly in the main paper and will add further details in Section 4.\n\n--- \nWe hope to have addressed the reviewer’s questions and concerns, and we would be happy to engage in further discussion or respond to any follow-up questions.\n\n---\n\n[1] Tiexin Qin, Benjamin Walker, Terry Lyons, Hong Yan, and Haoliang Li. Learning dynamic\ngraph embeddings with neural controlled differential equations, 2023.\n\n[2] Shenyang Huang, Farimah Poursafaei, Jacob Danovitch, Matthias Fey, Weihua Hu, Emanuele Rossi, Jure Leskovec, Michael Bronstein, Guillaume Rabusseau, and Reihaneh Rabbany. Temporal graph benchmark for machine learning on temporal graphs. Advances in Neural Information Processing Systems, 2023\n\n[3] James Morrill, Patrick Kidger, Lingyi Yang, and Terry Lyons. On the choice of interpolation\nscheme for neural CDEs. Transactions of Machine Learning Research, 2022."}, {"review_id": "Vzm7NtPB33", "reviewer": "Reviewer_Dvhd", "summary": "This paper proposes Permutation Equivariant Neural Graph Controlled Differential Equations (PENG-CDE), an advancement over existing Neural Graph CDE methods. PENG-CDE ensures permutation equivariance to enhance generalization capabilities. Experiments conducted on synthetic and real-world datasets demonstrate PENG-CDE's performance advantages over other dynamic graph representation methods.", "questions": "The Questions are listed in Weaknesses.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1. **Novel Approach:** The idea of integrating a permutation equivariant map into Neural Graph CDE for handling edge-valued signals is innovative.\n\n2. **Robust Theoretical Foundation:** The paper includes rigorous and detailed theoretical proofs confirming the permutation equivariance of the proposed model.\n\n3. **Clear Presentation:** The manuscript is well-organized, clearly written, and easy to follow.\n\nWeaknesses:\n\n1. **Limited Real-world Performance:** According to Tables 1, 2, and 3, while PENG-CDE performs strongly in synthetic tasks, it demonstrates only average performance in real-world scenarios (e.g., england-covid, twitter-tennis, and tgbn-trade). The marginal superiority of PENG-CDE over other methods on these tasks raises concerns about the practical effectiveness of the proposed approach.\n\n2. **Missing Relevant Baselines:** The absence of critical baselines such as STG-NCDE [1], DCRNN [2], and STIDGCN [3] in the synthetic experiments (heat diffusion and gene regulation) weakens the validity and comprehensiveness of these results.\n\n3. **Minor Grammatical Errors:** A grammatical mistake was found at line 173: \"Projecting an the flow of an ODE\" should be corrected to \"Projecting the flow of an ODE.\"", "was_revised": true, "final_justification": "The feedback fulfils my concerns. I will increase my score.", "rebuttal": "We appreciate the reviewer’s concerns and thank them for their thoughtful feedback. We will address the points raised one by one.\n\n**[W1] Limited real-world performance**\n\nFirst, we would like to highlight that across all tested real-world datasets - twitter-tennis, england-covid, tgbn-trade, and tgbn-genre - PENG-CDE achieves the best mean performance  among all baselines in terms of mean losses and NDCG@10, respectively. On tgbn-genre, a widely used benchmark, our method sets a new state-of-the-art result.\n\nRegarding the reviewer's concern about marginal performance differences: in our response to  [Q1] of Reviewer _FW39_, we perform a statistical significance analysis using Friedman tests followed by  critical difference diagrams. We kindly refer the reviewer to Tables 1–6 in our reply, which show that while none of the models (including the baselines) on twitter-tennis and england-covid are statistically distinguishable by this test, PENG-CDE's superior performance over both GN-CDE and STG-NCDE on the Temporal Graph Benchmark (TGB) and our dynamical systems datasets are statistically significant.\n\nSecondly, beyond strong empirical results, PENG-CDE offers substantial practical advantages over the original GN-CDE:\n-  PENG-CDE uses only ~2% more parameters than GN-CDE, yet reduces the average MSE by 0.7% and 5.4% on twitter-tennis and england-covid datasets, respectively, and improves NDCG@10 by 2.5% and 13.7% on the TGB datasets.\n-   Compared to the Pre-Mult GN-CDE variant, PENG-CDE reduces its parameter count by over 99%, while still reducing the mean MSE by a factor of 8.56× to 27.65× on the synthetic datasets.\n\n**[W2] Missing baselines in synthetic experiments**\n\nWe thank the reviewer for suggesting the inclusion of additional baselines in the synthetic experiments. Please refer to Tables 1 and 2 below for updated results incorporating the requested baselines. As shown, our method consistently outperforms these baselines, often by an order of magnitude. For further synthetic experiments on wealth formation and opinion dynamics including these baselines, we kindly refer to our response to Reviewer _FW39_.\n\n**Table 1: Comparison of GN-CDE variants and extended baselines on the heat diffusion tasks. Mean MSEs with 95% confidence intervals are reported, with the best mean highlighted in bold.**\n| Model                        | Community         | Grid              | Power Law            | Small World          |\n| ---------------------------- | ----------------- | ----------------- | -------------------- | -------------------- |\n| **DCRNN**                        | 0.722 ± 1.145     | 70.392 ± 108.473  | 1.690 ± 1.456        | 84.690 ± 105.696     |\n| **STIDGCN**                      | 0.554 ± 0.393     | 4.297 ± 0.975     | 0.907 ± 0.268        | 1.826 ± 0.240        |\n| **ASTGCN**                       | 2.188 ± 0.656     | 15.480 ± 1.757    | 3.849 ± 0.832        | 8.269 ± 1.094        |\n| **STG-NCDE**     | 2.091 ± 0.645     | 11.989 ± 1.090    | 3.518 ± 1.105        | 6.902 ± 1.303        |\n| Constant                     | 1.936 ± 0.550     | 11.155 ± 0.669    | 3.147 ± 0.850        | 6.286 ± 1.056        |\n| Graph Neural ODE (GNODEs)    | 0.237 ± 0.322     | 1.001 ± 0.751     | 0.270 ± 0.310 | 0.311 ± 0.268 |\n| Adjacency GN‑CDE             | 0.208 ± 0.240     | 0.691 ± 0.887     | **0.258 ± 0.288**    | 0.248 ± 0.240 |\n| Pre Mult GN‑CDE              | 1.968 ± 0.548     | 12.262 ± 1.437    | 7.440 ± 4.458        | 6.829 ± 0.644        |\n| Original GN‑CDE (Qin et al.) | 0.366 ± 0.400     | 1.324 ± 0.630     | 0.417 ± 0.314 | 0.552 ± 0.470        |\n| **PENG‑CDE (ours)**          | **0.096 ± 0.051** | **0.481 ± 0.195** | 0.290 ± 0.265 | **0.247 ± 0.215**    |\n| **Relative Improvement**     | 73.84 %           | 63.70 %           | 30.44 %              | 55.20 %              |\n\n\n**Table 2: Comparison of GN-CDE variants and extended baselines on the gene regulation tasks. Mean MSEs with $95\\%$ confidence intervals are reported, with the best mean highlighted in bold.**\n| Model                        | Community         | Grid                 | Power Law            | Small World          |\n| ---------------------------- | ----------------- | -------------------- | -------------------- | -------------------- |\n| **DCRNN**                        | 159.095 ± 149.970 | 28.784 ± 16.926      | 77.469 ± 28.178      | 27.300 ± 11.641      |\n| **STIDGCN**                      | 14.579 ± 2.815    | 0.633 ± 0.163        | 4.828 ± 0.694        | 0.611 ± 0.162        |\n| **ASTGCN**                       | 13.366 ± 3.302    | 0.695 ± 0.178        | 4.722 ± 0.644        | 0.579 ± 0.116        |\n| **STGNCDE**                  | 88.354 ± 15.126   | 8.753 ± 0.898        | 20.498 ± 2.633       | 6.760 ± 0.787        |\n| Constant                     | 36.307 ± 2.609    | 1.390 ± 0.168        | 7.099 ± 0.382        | 0.772 ± 0.126        |\n| Graph Neural ODE (GNODEs)    | 8.548 ± 3.212     | **0.167 ± 0.191**    | **0.372 ± 0.398**    | 0.294 ± 1.308 |\n| Adjacency GN‑CDE             | 8.909 ± 6.311     | 1.476 ± 2.504        | 0.498 ± 0.126 | 0.195 ± 0.046 |\n| Pre Mult GN‑CDE              | 153.084 ± 149.609 | 2.553 ± 0.251        | 6.978 ± 3.045        | 1.591 ± 0.146        |\n| Original GN‑CDE (Qin et al.) | 10.717 ± 7.079    | 0.457 ± 0.167        | 0.822 ± 0.299        | 0.323 ± 0.154 |\n| **PENG‑CDE (ours)**          | **4.566 ± 2.780** | 0.247 ± 0.090 | 0.526 ± 0.220 | **0.186 ± 0.484**    |\n| **Relative Improvement**     | 67.06 %           | 45.90 %              | 39.71 %              | 54.14 %   \n\n**[W3] Minor grammatical error**\n\nThank you for highlighting this. We have adjusted the paper accordingly."}, {"review_id": "kVAPALEAGk", "reviewer": "Reviewer_FW39", "summary": "The paper introduces Permutation Equivariant Neural Graph Controlled Differential Equations (PENG-CDEs), extending Graph Neural CDEs to enforce permutation equivariance, reducing parameter count while preserving representation power. It proves optimality within permutation-equivariant function spaces (Theorem 3.1) and equivariance under time reparametrizations and node permutations (Proposition 3.2). Evaluated on synthetic (heat diffusion, gene regulation, SIR model) and real-world tasks (TGB-genre, tgbn-trade, twitter-tennis, england-covid), PENG-CDEs outperform baselines like DCRNN, STG-NCDE, and GN-CDE, setting a state-of-the-art on TGB-genre node affinity prediction.", "questions": "1. Metrics are averaged over 10 seeds but lack statistical tests. Can you include t-tests or confidence intervals to confirm PENG-CDEs’ superiority, especially for small margins (e.g., tgbn-trade)?\nSuggestion: Add statistical tests in Section 4. Significant improvements could raise the Quality.\n\n2. Evaluation is limited to moderate-sized graphs. Can you test PENG-CDEs on large-scale dynamic graphs (e.g., million-node networks) to demonstrate scalability?\nSuggestion: Include scalability experiments in Section 4.4. Demonstrated scalability could raise the Significance.\n\n3. Synthetic tasks focus on SIR and diffusion models. Can you evaluate PENG-CDEs on diverse graph dynamics (e.g., economic networks, social interactions)?\nSuggestion: Add diverse synthetic tasks in Section 4.1. Broader applicability could raise the Significance.\n\n4. Theorem 3.1 assumes simplified conditions (e.g., linear maps). How robust is PENG-CDEs to complex, non-linear graph dynamics? Can you provide analysis?\nSuggestion: Analyze robustness in Appendix D or Section 3.2.\n\n5. Hyperparameters and fusion details are in Appendix F. Can you include key implementation details in Section 3 to improve readability?\nSuggestion: Add a paragraph in Section 3 summarizing the architecture and training.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\nTheorem 3.1 (Section 3.2) proves PENG-CDEs’ optimality in permutation-equivariant spaces, and Proposition 3.2 ensures equivariance, with complete proofs in Appendices D–E. Assumptions are clear.\nEvaluated on synthetic tasks (heat diffusion, gene regulation, SIR model, Section 4.1) and four real-world datasets (Section 4.4), PENG-CDEs outperform baselines (e.g., DCRNN, STG-NCDE, Table 9). Ablations (Appendix H) analyze fusion weights.\nWhile PENG-CDEs set a state-of-the-art on TGB-genre, gains over GN-CDE are small in some tasks (e.g., tgbn-trade, twitter-tennis, Section 4). No statistical significance tests (e.g., t-tests) confirm superiority, despite averaging over 10 seeds.\nTheorem 3.1 assumes simplified conditions (e.g., linear equivariant maps, finite group, Appendix D), potentially unrealistic for complex dynamic graphs. The proof relies on standard projection theory (Appendix B), offering limited novelty.\nReal-world datasets are limited to four, with no large-scale graphs (e.g., million-node networks). Scalability is untested despite dynamic graph complexity. Synthetic tasks (SIR, heat diffusion) are controlled, lacking diversity in graph dynamics (e.g., non-epidemic models).\nBaselines like DCRNN and STG-NCDE use grid-searched hyperparameters (Table 5), but modern temporal GNNs (e.g., GraphMixer [14]) are not fully compared, potentially inflating PENG-CDEs’ performance.\n\nClarity:\nThe paper is organized with clear sections on background, methodology, theory, and experiments. Appendices A–H provide detailed proofs and implementation.\nHyperparameters (e.g., GCN layers, learning rate) and fusion operation specifics are in Appendix F, reducing standalone clarity. The MLP in source-target identification (Appendix F.5) lacks architectural detail.\nTerms like “permutation equivariant function spaces” and “Haar measure” (Appendix C) are mathematically dense, lacking intuitive explanations for non-experts.\nSection 4 reports metrics but lacks analysis of why PENG-CDEs excel on TGB-genre but not others. Ablation insights (Table 9) are descriptive, not mechanistic.\n\nSignificance:\nDynamic graph learning is critical for traffic forecasting, molecular modeling, and epidemiology (Appendix G.1). PENG-CDEs’ efficiency could benefit these fields.\nGains over GN-CDE are modest, and broader applicability (e.g., large-scale graphs) is untested. The practical advantage is limited without statistical validation.\nThe focus on node/graph-level tasks excludes other dynamic graph problems (e.g., link prediction, clustering).\nAppendix F.1 notes moderate compute (8 GPU hours for TGB tasks), but scalability to massive graphs is unaddressed, critical for real-world applications.\nAppendix G.1 claims positive impact but lacks specific negative impact analysis (e.g., misuse in surveillance, Checklist Q10), weakening assessment.\n\nOriginality:\nPENG-CDEs extend GN-CDEs with permutation equivariance, a new approach reducing parameters while preserving power (Section 3).\nHowever, PENG-CDEs build on GN-CDEs [10] and Neural ODEs [7], with equivariance concepts from Maron et al. [44]. The contribution is an adaptation, not a groundbreaking paradigm. Theoretical results rely on standard tools (e.g., projection theorem, Haar measure, Appendix B–C), limiting novelty.", "was_revised": true, "final_justification": "There’s still room for improvement in these areas, but your response has addressed some of the issues, so I’ll give a score.", "rebuttal": "We sincerely thank the reviewer for their detailed and constructive feedback. We begin by responding to the specific questions and will then address your remaining concerns.\n\n**[Q1]: Statistical significance tests**\n\nWhenever possible, we reported means along with 95% confidence intervals. This applies to all synthetic experiments and the PyTorch Geometric Temporal datasets reported in Tables 1 and 2 of the main paper. For the Temporal Graph Benchmark datasets, baseline results were only available as mean NDGC scores with standard deviations, so we adopted the same format for consistency.\n\nTo address statistical significance testing, we employ critical difference diagrams, based on a two-stage procedure:\n\n1.  A global Friedman test  was used to determine whether any model differences were statistically significant.\n    \n2.  If the null hypothesis was rejected, we performed pairwise comparisons using the aligned Friedman post-hoc test.\n\nWe present the results in Tables 1-6 below and will include the full CD diagrams in the camera-ready version of the manuscript. \n\nTable 1: Critical difference diagram for heat diffusion synthetic experiment.\n|Mean Rank|Model|Non‑significant clique(s)|\n|----------:|------------------|--------------------------:|\n|1.93|Graph Neural ODE|A|\n|2.00|PENG‑CDE|A|\n|2.36|Adjacency GN‑CDE|A|\n|3.71|GN‑CDE|A|\n|5.00|Constant|B|\n|6.00|Pre Mult GN‑CDE|B|\n\nTable 2: Critical difference diagram for gene regulation synthetic experiment.\n|Mean Rank|Model|Non‑significant clique(s)|\n|----------|------------------|--------------------------|\n|1.73|PENG‑CDE|A|\n|2.45|Adjacency GN‑CDE|A|\n|2.55|Graph Neural ODE|A|\n|3.45|GN‑CDE|A|\n|5.18|Constant|B|\n|5.64|Pre Mult GN‑CDE|C|\n\nTable 3: Critical difference diagram for opinion dynamics synthetic experiment.\n|Mean Rank|Model|Non‑significant clique(s)|\n|----------|------------------|--------------------------|\n|1.47|PENG‑CDE|A|\n|2.35|Adjacency GN‑CDE|A|\n|2.35|Graph Neural ODE|A|\n|3.82|GN‑CDE|B|\n|5.00|Constant|C|\n|6.00|Pre Mult GN‑CDE|D|\n\nTable 4: Critical difference diagram for wealth formation synthetic experiment.\n|Mean Rank|Model|Non‑significant clique(s)|\n|----------|------------------|--------------------------|\n|1.79|PENG‑CDE|A|\n|1.84|Adjacency GN‑CDE|A|\n|2.58|Graph Neural ODE|A|\n|3.89|GN‑CDE|B|\n|4.89|Constant|C|\n|6.00|Pre Mult GN‑CDE|D|\n\nTable 5: Critical difference diagram for PGT experiments.\n|Mean Rank|Model|Non‑significant clique(s)|\n|----------|----------------------|---------------------|\n|3.75|Constant|A|\n|3.92|GN-CDE|A|\n|4.00|Adjacency GN-CDE|A|\n|4.00|DCRNN|A|\n|4.08|STIDGCN|A|\n|4.25|PENG-CDE|A|\n|5.58|STG-NCDE|A|\n|6.42|ASTGCN|A|\n\nTable 6: Critical difference diagram for TGB experiments.\n|Mean Rank|Model|Non-significant clique|\n|----------|----------------|---------------------|\n|1.00|PENG-CDE|A|\n|2.15|GN-CDE|B|\n|2.85|STG-NCDE|C|\n\nWe can see that in all synthetic experiments (Tables 1-4), our PENG-CDE consistently ranks within the top non-significant clique which demonstrates its strong and stable performance. For the PyTorch Geometric Temporal datasets (twitter-tennis, england-covid; Table 5), the global Friedman test was inconclusive, meaning the test did not reject the null hypothesis that no model (even the baselines) is statistically different from the others. On the Temporal Graph Benchmark, the critical difference analysis (see Table 6) shows that PENG-CDE performs significantly better than both GN-CDE and STG-NCDE.\n\n**[Q2]: Evaluation on larger graphs**\n\nThe main scope of our paper is the derivation of a new mathematical formalism for dynamic graph representation learning. Our method belongs to a family of models that are unique in their approach to tackling graph dynamics. Most other approaches are permutation equivariant due to handling the graph structure using an existing graph neural network structure. \n\nOur implementation relies on JAX and, in particular, the Diffrax library. As both currently lack full support for sparse matrix operations, PENG-CDE, like the original GN-CDE, exhibits quadratic memory complexity in the number of nodes. This constrains scalability beyond moderately sized graphs, as discussed in the Limitations section. We view this as an exciting direction for future work. As sparse support in JAX matures beyond its experimental stage, we expect to overcome this scalability bottleneck. Importantly, as shown in Table 1 in our response to Reviewer GJRF, once memory constraints are addressed, PENG-CDE exhibits significantly better runtime scaling than GN-CDE, highlighting its potential for efficient execution even at larger scales.\n\n**[Q3]: More diverse synthetic experiments**\n\nWe are happy to provide the results of testing our PENG-CDE, the baselines included in the original submission, as well as the  additional baselines suggested by Reviewer _Dvhd_, in the two tables below.\n\nAs suggested by the reviewer, we have added two more dynamical systems to our synthetic experiments: an economic network model and a social interaction network model.\n\n1.  Capital Gains Wealth Dynamics\n\t\n\t$\\frac{dx_i}{dt} = \\delta x_i + s_i x_i^k + \\sum_{j} A_{ij} (x_j - x_i)$\n   \n    \n2. Nonlinear Opinion Dynamics\n    \n\t$\\frac{dx_i}{dt} = -x_i + \\sigma\\left( \\sum_{j} A_{ij} \\cdot x_j \\right)$    \n    where  $\\sigma$  is a thresholding function. \n    \nFor the results see Tables 7 and 8. We can see that PENG-CDE outperforms the original GN-CDE by 35–90%, and achieves the overall lowest mean MSE in 6 out of 8 dataset configurations.\n\n**Table 7: Comparison of GN-CDE variants and extended baselines on the wealth dynammics tasks. Mean MSEs with $95\\%$ confidence intervals are reported, with the best mean highlighted in bold.**\n\n|Model|Community|Grid|Power Law|Small World|\n|----------------------------|-----------------|--------------------|--------------------|--------------------|\n|DCRNN|2.639±1.981|22.619±17.115|113.564±236.126|200.486±280.387|\n|STIDGCN|1.862±0.947|7.296±2.547|2.124±1.012|3.133±1.286|\n|ASTGCN|3.431±1.224|19.694±2.986|5.153±1.184|9.246±2.204|\n|STG-NCDE|36.087±2.454|44.677±3.755|37.148±2.742|41.046±2.205|\n|Constant|2.504±0.473|14.157±1.746|3.682±0.873|7.445±1.200|\n|Graph Neural ODE (GNODEs)|0.878±0.388|**1.898±1.342**|0.950±0.660|0.940±0.185|\n|Adjacency GN-CDE|0.904±0.493|2.193±1.721|1.075±0.847|1.297±0.765|\n|Pre Mult GN-CDE|7.740±1.122|24.522±6.594|39.715±32.698|16.568±6.902|\n|Original GN-CDE|1.577±0.781|6.582±3.273|2.406±0.965|3.076±1.193|\n|**PENG-CDE (ours)**|**0.522±0.316**|4.273±3.818|**0.863±0.330**|**0.813±0.627**|\n|**Relative Improvement (%)**|66.89%|35.09%|64.13%|73.57%|\n\n\n**Table 8: Comparison of GN-CDE variants and extended baselines on the opinion dynammics tasks. Mean MSEs with $95\\%$ confidence intervals are reported, with the best mean highlighted in bold**.\n|Model|Community|Grid|Power Law|Small World|\n|----------------------------|--------------------|--------------------|--------------------|--------------------|\n|DCRNN|20.659±32.149|92.165±132.026|91.687±81.638|173.554±183.913|\n|STIDGCN|2.434±1.562|6.977±2.451|2.724±1.530|3.933±1.567|\n|ASTGCN|22.001±1.519|34.855±2.370|22.517±1.490|27.439±1.554|\n|STG-NCDE|57.495±9.245|85.405±6.479|59.446±9.294|68.092±8.988|\n|Constant|22.989±1.372|33.124±1.275|23.775±1.480|27.239±1.929|\n|Graph Neural ODE (GNODEs)|0.665±0.620|2.338±1.277|3.073±6.114|**0.779±0.417**|\n|Adjacency GN-CDE|0.777±0.798|1.912±0.566|0.802±0.652|0.909±0.416|\n|Pre Mult GN-CDE|38.717±6.229|64.825±14.066|61.226±47.714|88.503±74.608|\n|Original GN-CDE (Qin et al.)|1.060±0.436|6.963±4.698|6.781±11.159|5.223±4.240|\n|**PENG-CDE (ours)**|**0.525±0.581**|**1.851±1.488**|**0.674±0.606**|1.161±6.011|\n|**Relative Improvement (%)**|50.52%|73.41%|90.07%|77.77%|\n\n\n**[Q4]: Non-linear graph dynamics in Theorem 3.1**\n\nThe reviewer is correct to point out that Theorem 3.1  is stated under the assumption of a  _linear_  vector field, i.e. the absence of non-linearities. The purpose of this result is to theoretically motivate the architectural modifications we introduced when transitioning from GN-CDEs to PENG-CDEs. Specifically, it formalises the idea that  _“PENG-CDE is the closest equivariant version of GN-CDE we can construct in the linear setting.”_\n\nIn practice, however, we train all CDE-based models with non-linearities in the vector field. This enhances their expressive power and enables them to learn the non-linear dynamical systems in Section 4.1 and Tables 7 and 8 above.\n\n**[Q5]: More information on architectures and hyperparameters in main text**\n\nWe agree with the reviewer the paper would benefit from more details in the main manuscript and will include this with the camera-ready version.\n\n**[W1]: Lack of novelty**\n\nSee our reply to [W1] of reviewer _G4dS_.\n\n**[W2]: Hyperparams of baselines**\n\nTo ensure a fair comparison, we followed the hyperparameter settings from the original papers as closely as possible for all baselines, in particular DCRNN and GraphMixer.\n\n**[W3]: Ablation in Table 9 is descriptive, not mechanistic**\n\nWe thank the reviewer for pointing this out. We will include additional explanations in the final version. For example, summing over rows or columns of the adjacency matrix corresponds to computing node degrees, indicating that our model can utilise global information such as degree normalisation.\n\n**[W4]: No evaluation of link prediction or clustering**\n\nTo ensure a fair comparison with the main baseline GN-CDE, we focused on the setup evaluated in the original paper, which included both graph-level and node-level tasks. This is a promising avenue for future work.\n\n---\n\n[1] Weilin Cong, Si Zhang, Jian Kang, Baichuan Yuan, Hao Wu, Xin Zhou, Hanghang Tong, and\nMehrdad Mahdavi. Do we really need complicated model architectures for temporal networks?,\n2023. \n\n[2] Shenyang Huang, Farimah Poursafaei, Jacob Danovitch, Matthias Fey, Weihua Hu, Emanuele Rossi, Jure Leskovec, Michael Bronstein, Guillaume Rabuseau, and Reihaneh Rabbany. Temporal graph benchmark for machine learning on temporal graphs, 2023"}]

Author rebuttal:

Meta-review: The paper introduces a permutation equivariant model for controlled differential equations to enable representation learning on dynamic graphs. The key idea of the model is to leverage permutation equivariant function spaces to reduce the number of parameters while preserving model capacity, leading to more efficient training and better generalization. The work has several clear merits highlighted by the reviewers, including the solid formulation of the model and the detailed experimental results. During the discussion period, the authors provided clarifications and addressed several of the reviewers' concerns. Overall, the paper makes a good contribution to the field. I would encourage the authors to take into account the reviewers' comments to further improve clarity.
Decision: Accept (poster)