Uncertainty Modeling in Graph Neural Networks via Stochastic Differential Equations
Authors: Richard Bergna
Sergio Calvo Ordoñez
Felix Opolka
Pietro Lio
José Miguel Hernández-Lobato
Venue: ICLR 2025 Spotlight (2025)
Keywords: Graph Neural Networks
Stochastic Differential Equations
Uncertainty Quantification
Bayesian Machine Learning
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
TL;DR: We introduce Latent Graph Neural SDEs (LGNSDEs), a model that quantifies uncertainty in graph-structured data while maintaining robustness to input perturbations, supported by theoretical guarantees and empirical evaluation.

Abstract: We propose a novel Stochastic Differential Equation (SDE) framework to address the problem of learning uncertainty-aware representations for graph-structured data. While Graph Neural Ordinary Differential Equations (GNODEs) have shown promise in learning node representations, they lack the ability to quantify uncertainty. To address this, we introduce Latent Graph Neural Stochastic Differential Equations (LGNSDE), which enhance GNODE by embedding randomness through a Bayesian prior-posterior mechanism for epistemic uncertainty and Brownian motion for aleatoric uncertainty. By leveraging the existence and uniqueness of solutions to graph-based SDEs, we prove that the variance of the latent space bounds the variance of model outputs, thereby providing theoretically sensible guarantees for the uncertainty estimates. Furthermore, we show mathematically that LGNSDEs are robust to small perturbations in the input, maintaining stability over time. Empirical results across several benchmarks demonstrate that our framework is competitive in out-of-distribution detection, robustness to noise perturbations, and active learning, underscoring the ability of LGNSDEs to quantify uncertainty reliably.

Published as a conference paper at ICLR 2025
UNCERTAINTY MODELING IN GRAPH NEURAL NET- WORKS VIA STOCHASTIC DIFFERENTIAL EQUATIONS
Richard ;ergna'; Sergio Calvo-Ordonezz",, Felix L. Opolka",
Pietro Lid", Jose Miguel Hernandez-Lobato
l *Dearrmment of Engineering. University of Cambridge
2 *Mathematical Institute, University of Oxford
3 "Oxford- Man Institute of Quantitative Finance, University of Oxford 'Department of Computer Science and Technology, University of Cambridge
ABSTRACT
We propose a novel Stochastic Differential Equation (SDE) framework to address the problem of learning uncertainty-awar representations for graph-structured data. While Graph Neural Ordinary Differential Equations (GNODEs) have shown promise in learning node representations, they lack the ability to quantify uncer- tainty. To address this, we introduce Latent Graph Neural Stochastic Differential Equations (LGNSDE), which enhance GNODE by embedding randomness through a Bayesian prior-posterior mechanism for epistemic uncertainty and Brownian motion for aleatorie uncertainty. By leveraging the existence and uniqueness of solutions to graph-based SDEs, we prove that the variance of the latent space bounds the variance of model outputs, thereby providing theoretically sensible guarantees for the uncertainty estimates. Furthermore, we show mathematically that LGNSDEs are robust to small perturbations in the input, maintaining stability over time. Empirical results across several benchmarks demonstrate that our frame- work is competitive in out-of-distribution detection, robustness noise, and active learning, underscoring the ability of LGNSDEs to quantify uncertainty reliably.
INTRODUCTION
Before the widespread of neural networks and the boom in modern machine learning, complex systems in various scientific fields were predominantly modelled using differential equations. Stochastic Differential Equations (SDEs) were the standard approach to incorporating randomness. These methods were foundational across disciplines such as physics, finance, and computational biology (Hoops al., 2016 Quach al. 2007 Mandelzweig & Tabakin [001]] Arroyo et al. 2024 Moreno-
Pino et al. 2024 Cardelli 2008
Buckdahn et al.) 2011 Cvijovic et al. 2014).
In recent years, Graph Neural Networks (GNNs) have become the standard for graph-structured
data due to their ability to capture relationships between nodes. They are widely used in social network analysis, molecular biology, and recommendation systems. However, traditional GNNs cannot reliably quantify uncertainty. Both alcatoric (inherent randomness in the data) and epistemic (model uncertainty due limited knowledge) are essential for deciisionmmaknng risk assessment, and resource allocation, making GNNs less applicable in critical applications.
To address this gap, we propose Latent Graph Neural Stochastic Differential Equations (LGNSDE), a method that perturbs features during both the training and testing phases using Brownian motion noise, allowing for handling noise and aleatoric uncertainty. We assume a prior SDE on the latent space and learn a posterior SDE using GNN as the drift function. This Bayesian approach to the latent space allows us to quantify epistemic uncertainty. As a result, our model can capture and quantify both epistemic and alcatoric uncertainties. More specifically, our contributions are as follows:
We introduce a novel model class combining SDE robustness with GNN flexibility for handling complex staph-structured data, which quantifies both epistemic and aleatoric uncertainties.
We provide theoretical guarantees demonstrating our model's ability to provide meaningful uncer- tainty estimates and its robustness to perturbations in the inputs.
"Corresponding Author. Email: rsb630cam. ac.uk

---

Published as a conference paper at ICLR 2025
We empirically show that Latent GNSDEs demonstrate exceptional performance in uncertainty guantification, outperforming Bayesian GCNs (Hasanzadeh et al. 2020), GCN ensembles (Lin
ct al. 2022 and Graph Gaussian Process (Borovitskiy et al. 20211.
H(t)
0
T
Figure 1: The diagram shows the evolution of one of the nodes of the input graph in latent space, (!),, through an SDE, with sample paths (purple) and confidence bands representing variance. At three timesteps, we visualize graph embeddings, where nodes (white and orange) become more separable over time due to the influence the vector field. The inset axes represent latent dimensions, while the purple and yellow background highlights the magnitude and direction of the vector field guiding the latent dynamics.
2
BACKGROUND
Graph Neural Ordinary Differential Equations (GNODE). Introduced by Poli et al. (2019). GNODEs extend Graph ResNets by modelling node representations continuously over time. typical Graph ResNet, the node features evolve according to the update rule
H(t 11 = H(t) + Fg( t,
where represents the layer index, H(1) and H + 1) are the input and output node embeddings, and (V,E) is graph with node set V and edge set E. The function fe, parameterized by 0, defines the transformation applied to node features. Here, the input features are X,, H(0), and the final
node embeddings HT)) yield the model's 's output predictions, denoted Y. Now, consider an update with small time step R
H(t + c) H(t) c. (.(((
which leads to the continuous limit as c c
dH(t)
= ())),,t,,))
dt
This differential equation represents the continuous evolution of node features over time, transforming the discrete depth of layers into continuous variable t. The solution this equation given by
H(t) H(0) J. Fg (H((), u, f) du.
In practice, Fg is modelled by neural network, and operates continuous depth parameter. The solution to the integral approximated numerically, making GNODE a continuous-deptt analogue of graph residual networks.
2

---

Published as a conference paper at ICLR 2025
3 METHODOLOGY
Inspired by Graph Neural ODEs (Poli et al. 2019) and Latent SDEs Li et al. 2020). we now introduce our model: Latent Graph Neural SDES LGNSDEs (Figure I which use SDES define prior and approximate posterior stochastic trajectories for H(!) (Xu et al. 2022). Furthermore, LGNSDEs can be viewed as the continuous representations of existing discrete architectures (A.4).
3.1 MODEL DEFINITION
LGNSDEs are designed capture the stochastic latent evolution of Ht) on graph-structured data. use an Ornstein-Uhlenbeck (OU) prior process, represented by
(H) = F3 (H((), dr + Go(H(t), ())
where we the drift and diffusion functions, Fg and Gg. to constants and consider them hyperpa- rameters. Moreover, dW (t) ) is a Wiener process. The approximate posterior is defined as
dH = F (H((),, Φ) dt + t)
where Fg is now parameterized by GCN with é representing the learned weights of the neural network. The drift function mainly determines the dynamics of the evolution of the latent state, while the diffusion term Gg (t) (t) introduces stochastic elements. With the need to keep the Kullback-Leible (KL) divergence bounded, we set the diffusion functions, Gg, of the prior and posterior to be the same [Calvo-Ordonez et al. 2024 Archambeau al. 2007].
Let Y be collection of target variables, e.g., class labels, for some of the graph nodes. Given Y train our model with variational inference, with the ELBO computed as
J | (H()H 0. 0, du
C
log (og(((((()))
where the expectation is approximated over trajectories of H(t)) sampled from the approximate posterior SDE, and Go(H(tt)-- FF. (H((), u) F. u)].
sample H() by integrating the SDE in Eq. The analytical solution
H(t) = H()) + [ Fe.o uH)
f." Gg Gg(H()),, () (W)(),,
where H(0) are the node-wise features X the graph G. We numerically solve this integral with standard Stochastic Runge-Kung Kutta method (Rößler 2010). We then use Monte Carlo approximation get the expectation of H(t) and approximate the posterior predictive distribution
N
p(Y Y)
Σ p H.
N
where (t).. H are samples drawn from the approximation to p(H(t) Xin. 9).
Following Poli et al. (2019), we use a similar encoder-decoder setup. Our encoding focuses solely on the features of individual nodes, while the graph structure remains unchanged. Finally, remark that the memory complexity when using the stochastic adjoint sensitivity method is O(1) and the time complexity is OL log L(Eld + [Vd)), where L is the number of SDE solver steps, is the number of edges in the graph, V is the number of nodes, and d is the dimension of the features (see
Appendix C.3). For a runtime comparison with other models see 12 and T5|
Note that in our framework, model depth is inherently tied to the evolution of the latent space, where depth determined by the number layers corresponding to the sampling steps of the SDE solver. As the SDE solver dictates the number of steps, it effectively controls the number of layers in the model. Thereby the SDE solver chooses the number layers, dynamically changing the model's complexity based on task difficulty. For more complex tasks, the solver will generate additional steps (layers), while simpler tasks will require fewer layers. Exploring optimal adaptive SDE solvers will remain part of future work.
3

---

Published as a conference paper at ICLR 2025
4 THEORETICAL GUARANTEES
To establish the theoretical foundations of our framework, we begin by acknowledging the existence and uniqueness results for graph-based stochastic differential equations, as proven in Lin et al. (2024). This result guarantees that, under certain assumptions (Appendix A..2). there exists a unique mild solution the Graph Neural SDE, ensuring the well-posedness of the model's dynamics and that the solution behaves in a stable and predictable manner, i.e. small changes in the initial conditions or input data lead small changes in the solution. We borrow this result in the following theorem:
Theorem (Lin et al. (0244). If & is linear operator with complete orthonormal basis set and eigenvalues 0 O, the continuous operator Gg satisfies the Lipschitz condition, and the initial node representation (0) is square-integrable and Fo-measurable, then there exists unique mild solution (I) on [0, for any T and E V, such that
(1( HH(0) S: ( G (c(H.()))WW(s))
where e is the semigroup generated by V. Furthermore, there exists constant CT >> 0 such that sup
H+] s |
<E(O.T]
This theorem confirms that graph-hased SDEs have well-posed solution trajectory over time that is well-behaved and bounded, ensuring that our LGNSDE model can maintain stability across varying graph structures, meaning that the solution does not exhibit erratic unbounded behaviour even under small changes in graph structure or input features.
Leveraging this result, proceed to present key results the stability and robustness of our framework.
We derive bound that proves that proposed models provide meaningful uncertainties.
We demonstrate the robustness of our framework under small perturbations in the initial conditions.
By showing that the variance of the latent representation bounds the model output variance, we highlight the ability of LGNSDEs to capture and quantify inherent uncertainty in the system. The model's s output given by the trajectory the latent representation since y = H(( T), therefore the uncertainty in the latent space directly influences the uncertainty in predictions. We formalize this in the following proposition:
Proposition 1. Under assumptions and given Theorem 1 there exists unique mild mild solution to an LGNSDE of the form
dH = Fg((( f, Ot + Gg(H(f), dW
whose variance bounds the variance of the model output (t)
Var(y(t)) & L2 (a)(H
where L2 is the Lipschitz constant of the readout layer. This ensures that the output variance is bounded by the prior variance of the latent space, providing controlled measure of uncertainty.
Now, by deriving explicit bounds on the deviation between the perturbed and unperturbed solutions over time, we show that the model's output remains stable.
consider two initial conditions Ho and H, = H, + (H. where (0) E I aad is small perturbation in the initial node features with [[H(()]]e= that Ho taken from a compact set H. S Rexd Then, the deviation between the solutions H(t) and Ht) of the LGNSDE with these initial conditions remains bounded across time specifically
Proposition 2. Under assumptions)
Assume
E[[ - H(t)]l)) S t
'A mild solution SDE is expressed via integral equation involving the semigroup generated the linear operator and represents weaker notion of the solution.
Noote that while the bound is exponential in f, in practice, the time horizon is usually constrained to a limited range, such as i E [0, 1]. Within this interval, the exponential factor does not grow excessively. ensuring that the deviation between the perturbed and unperturhed solutions remains under control.
4

---

Published as a conference paper at ICLR 2025
In summary, we show analytically that our framework effectively quantifies uncertainty and maintains robustness under small perturbations of the input. First, we confirm that the model's 's output variance is controlled and directly linked to the variance of the latent state. Second, provide bound on the deviation between solutions with perturbed initial conditions, ensuring stability over time. The proofs can be found in Appendix A
5
EXPERIMENTS
Entropy Distribution: Correct vs. Incorrect Predictions
GCN
GNODE
LGNSDE (Our)
BGCN
Ensemble
21
2.0
Density
1.5
Density
Density Ca
5 bn
Density
Density
12
0.0
0.0
1.50 Entropy
0.D
1.0
1.65 1.50
Entropy
Entropy
Entropy
Entropy
Entropy Distribution: In-Distribution vs. OOD
GCN
GNODE
LGNSDE (Our)
BGCN
Ensemble
10
Olec IOBE
Densiyy
OOD
Denstyy
2.4
Denety
Densly
Densiy N
Foe Minan
OCD Me
1.6
0.8
0.4
0.8
1.2
1.6
0.0
0.0
015
1.1
155
2.0
0.0
0.0
0.6
1.2
1.
0.4
18
1.2
1.6
1.2
1.4 Entropy
1.6
1.8
Entropy
Entropy
Entropy
Entrapy
Figure Top: Entropy distributions comparing correct and incorrect model predictions on the CORA dataset. Higher entropy is expected for incorrect predictions. Bottom: Entropy distributions comparing OOD samples with in-distribution samples in the CORA dataset.
Metric
Model
Cora
Citesger
Computers
Photo
Pubmed
GNN
0,9654 0.0050
0.9173 + 0.0068
0.9680 0.0016
0.9905 0.0003
0,9006 0.0139
GNODE
0,9664
0.0051
0.9146 1 0.0063
0.9569
0.0067
0,9885
0.0007
0.8857 1 0.0203
AUROC (1)
BGCN
0,9571
0.0092
0.9099 + 0.0090
0.9421
0.0097
0.9489 = 0.0189
0.7030 + O. 1331
ENSEMBLE
0,9635 0.0031
0.9181 1 0.0062
0.0025 0.9669
0,9886
0.0004
0,8785 0.0163
GRAPH GP
0,8970 0.0055
0.8877 + 0.0062
00M
0OM
0OM
LGNSDE (Ours)
0,9659 0.0038
0.9111 1: 0.0072
0.9691 0.0032
0,9909 a 0.0004
0,9004 1: 0.0087
GNN
10009:0.000
0.1626 + 0.0100
00045500:0533
0.0199 0.0007
0.1 : 0.0192
GNODE
0.0628 0.0095
0.1609 1 0.0141
0.1055 a 0.0165
0,0219 = 0.0015
0,1427 + 0.0214
AURC (1)
BGCN
0.0858 0.0219
0.1764 = 0.0215
0.16340..044
0,1218 = 0.0577
0,4152 + 0,1723
ENSEMBLE
0,0789 0.0061
0.1722 1 0.0179
0.0077000037
0,0244 = 0.0012
0.1722 1: 0,0285
GRAPH GP
0,1869 0.0084
0.2328 + 0.01188
0OM
0OM
0OM 0.1378 0.0118
LGNSDE (Ours)
0,0702 0.009S
0.1686 0.0146
0.0687 0.0114
0.0186 0.0007
GNN
0,8105 0.0173
0.7258 + 0.0137
0.8098 = 0.0048
0,9116 = 0.0021
0.7570 0.0229
GNODE
0,8202 0.0149
0.7235 4 0.0159
0.7911
0.0098
0.9053 + 0.0032
0.7577 + 0.0231
Avcuracy
BGCN
0,7897 0.0261
0.7013 0.0196
0.7114
0.0333
0,7124 0.0968
0.4581 O. 1846
ENSEMBLE
0,8038 0.0105
0.7108 1 0.0166
0.8070 a 0.0055
0,9070 a 0.0019
0.7299 a 0,0218
GRAPH GP
0.6491 0.0116
0.6675 +
0OM
00M
0OM
LGNSDE (Ours)
0,8079 0.0154
0.7120 & 0.0119
0.8247 0.0103
0,9169 0.0021 0.7589 0.0161
Table 1: Performance comparison of models across five datasets (Cora, Citeseer, Computers, Photo, Pubmed) based on AUROC, AURC, and Accuracy (mean + standard deviation). Red indicates the best-performing model, while blue indicates the second- hert-peroooninn model for each metric. Some results for the Graph GP model are unavailable due to out-of-memmon (OOM) errors.
We evaluate LGNSDE on the following datasets: Cora (Sen et al. 2008). CiteSeer (Giles et al. 1998 PubMed (Sen et al. 22088. and the Amazon co-purchasing graphs Computer (McAuley al. 2015
and Photo (Shchur et al. 2018). We compare its performance against GNODE (Poli et al.) 2019).
GCN (Kipf Welling 2016), Bayesian GCN (BGCN) (Hasanzadeh al., 2020), an ensemble of GCNs (Lin et al.
2022 and Graph GP (Borovitskiy et al. 2021).
'We were unable to run the Graph GP model on certain datasets due to its high memory requirements and scalability issues, resulting in out-of-memory (OOM) errors. Consequently, report these cases 00M in our experimental results tables.
5

---

Published as a conference paper at ICLR 2025
In conducting our experiments, used the setup outlined in Shchur al. (2018). This involved using 20 random weight initializations for datasets with fixed Planetoid splits and implementing 100 random splits for other datasets. The hyperparameters that achieved the highest validation accuracy were chosen, and their performance was evaluated on a test set. For further details on our hyperparameter grid search refer Appendix B
5.1 STANDARD SETTING
The results in Table I demonstrate that LGNSDE consistently ranks as either the best or second-best model across most datasets in terms of Micro-AUROC (Area Under the Receiver Operating Character- istic), AURC (Area Under the Risk Coverage), and accuracy. This indicates that LGNSDE effectively handles model uncertainty, successfully distinguishing between classes (AUROC), maintaining low risk while ensuring confident predictions (AURC), and delivering high accuracy.
The top of F the entropy distributions the models for correct and incorrect predictions. Note that most models display similar mean entropy for both correct and incorrect predictions. Notably, our model stands out with the largest difference in entropy, with incorrect predictions having 35% more entropy (more uncertainty see Subsection 5.2 compared to correct predictions, larger gap than observed in other models.
5.2
OUT OF DISTRIBUTION DETECTION
Metric
Model
Cora
Citeseer
Computers
Photo
Pubmed
GNN
0.7063 0.0569
0.7937 0.0366
0.77%600:027
0.8578 060136
0.6127 = 0.035T
GNODE
0.739% 0.0677
0.7828 + 0.0465
0.7753 + 0.0795
0,8473 + 0.0158
0,5813 + 0.0242
AUROC (t)
BGCN
0.7193+ 0.0947
0.8287 # 0.0377
0.7914001233
0.7910 0.0464
0,5310 + 0.0472
ENSEMBLE
0.7031 0.0696
0.8190 4: 0.0375
0.8292 4 0.0338
0,8352 0.0059
0.6130 + 0.0311
LGNSDE (Ours)
0.7614 + 0.0804
0.8258 + 0.0418
0.7994 + 0.0238
0,8707 = 0.0099
0.6204 + 0.0162
GNN
0.0220 0.0049
0.0527 1 0.0075
0.0072
0.0076 0.0006
0.3227
GNODE
0.0184 0.0053
0.0545 : 0.0110
0.0070 0.0029
0,0097 0.0015
0.3357 1 0.0309
AURC (1)
BGCN
0,0208 0.0091
0.0458 1 0.0071
0.0064 0.004
0.0108 1: 0.0034
0,3714 4: 0.0317
ENSEMBLE
0,0215 0.0061
0.0487 0.0072
0.0041
0.0081 0.0001
0.3277 0.0265
LGNSDE (Ours)
0.0168 0.0070
0.0479 a 0.0109
4
0,0068 a 0.0008
0,3205 + 0.0135
GNN
0,9470 0.0004
0.8614 = 0.0071
0.0000
0,9558 = 0.0002
0.6180 = 0.0153
GNODE
0,9469 0.0002
0.8603 0.0086
0.9788 0.0004
0,9557 0 0.0000
0.6084 0.0120
Accuracy (t)
BGCN
0,9472 # 0.0004
0.8711 # 0.0133
0.9797 = 0.0010
0,9558 = 0.0002
0.6039 + 0.0074
ENSEMBLE
0,9470 0.0003
0.8699 +
0.9788
0,9560
0.0001
0.6216
LGNSDE (Ours)
0,9471 0.0003
0,8729 + 0,0108
0.9788 0.0000
0,9560 0.0002
0.6243 0.0094
Table 2: Performance comparison of models for OOD detection across five datasets (Cora. Citeseer, Computers, Photo. Pubmed). Metrics reported are AUROC, AURC, and Accuracy (mean + standard deviation). Red indicates the best-performing model, while blue indicates the second eccond-best-pefoommin model for each metric.
We evaluate the models' ability to detect out-of-distribution (OOD) data by training them with one class left out of the dataset. This introduces an additional class in the validation and test sets that the models have not encountered during training. The goal is to determine if the models can identify this class OOD. We analyze the entropy
C Σ p(ù -
H(JXX.) =
c(X.) logp(ú)
where p(ù = c| X.) represents the probability of input X, belonging to class e. Entropy quantifies the uncertainty in the model's predicted probability distribution over C' classes for given input X,,
The bottom of Figure shows the test entropy distribution for in-distribution (blue) and out-of- distribution (red) data. For each test sample, predictions were made over C' -1 classes, excluding the left-out class. The OOD class exhibits higher entropy, indicating greater uncertainty. While most models show similar entropy distributions for both data types, our LGNSDE model achieves clear separation, with a 50% higher mean entropy for OOD data compared to in-distribution data. Other models show less than 10% difference between the two distributions.
Table 2 presents the AUROC and AURC scores for OOD detection across multiple datasets. AU- ROC evaluates the model's ability to differentiate between in-distribution and out-of-distribution

---

Published as a conference paper at ICLR 2025
(OOD) samples, with higher scores indicating better discrimination. AURC measures the risk of misclassification as coverage increases, where lower values are preferred. LGNSDE consistently achieves the best AUROC and AURC scores across most datasets, indicating its superior performance in accurately identifying OOD samples and minimizing the risk of misclassification.
The accuracy was determined by applying an entropy-basee threshold. Predictions with entropy above this threshold were classified as out-of-distribution (OOD), while those below were considered in-distribution. The optimal threshold was identified using a validation dataset, where it was selected to maximize overall classification performance.
5.3
NOISE PERTURBATION
Metric
Model
Cora
Citeseer
Computers
Photo
Pubmed
GCN
0,9610 + 0.0045
0.9096 0,0056
0.9682 + 0.0029
0.9909 000002
0.8466 + 0.0214
GNODE
0.9649 # 0.0054
0.9077 # 0.0062
0.9593 # 0.0033
0,9892 = 0.0007
0.8727 # 0.0183
AUROC (T)
BGCN
0,9606
0.9069
0.0076
0,9547
0,9845 0.0028
0.7643
ENSEMBLE
0,9581 + 0.0051
0.9166 0.0048
0,9701 + 0.0011
0,9893 = +00004
0,8093 + 0,0511
GRAPH GP
0.8979 +: 0.0050
0.8889
0.0037
OOM
0OM
0OM
LGNSDE (Our)
0,9634 0.0065
0.9172 0.0070
0,9698 0.0015
0.9911 0.0004
0.8636 0,0310
GCN
0,0782 = 0.0076
0.1 0,0079
0,0661 = 0.0099
0.0185
0000005
0.209% = 0.0397
GNODE
0.0625 + 0.0109
0.1676
0.00K5
0.1006 + 0.0114
0.0209
0.0018
0. 061655 + 0.0260
AURC (1)
BGCN
0.0799 + 0.0075
0,1687 0.0122
0.1400 0.0438
0.0359 0.0059
0,3026 0.0928
ENSEMBLE
0,0862 # 0.0067
0.1690
0.0099
0.0718 + 0.0037
0.0230 = 0.0008
0.2601 + 0.0528
GRAPH GP
0.1827 1 0.0081
0,2294 0,0088
0OM
0OM
00M
LGGNDDE (Our)
0.0731 4 0.0128
0.1612 0.0145
0.0642 a 0.0059
0.0184 a 0.0010
0.1908 0.0519
GCN
0,8054 + 0.0112
0.7162 + 0.0145
0.8169 # 0.0026
0.9139 + 0.0013
0.6874 + 0,0346
GNODE
0.8255 = 0.0134
0.7213 0.0116
0.7907 = 0.0126
0,9074 0.0061
0.7402 = 0.0286
Accuracy
BGCN
0.7946 0.0115
0.7034 0,0223
0.7401 + 0.0472
0.8754 0.0180
0.5848 0.0973
ENSEMBLE
0.7916 1 0.0156
0.7199 1 0.0139
0.8160 + 0.0038
0.9091 = 0.0013
0.6390 + 0,0577
GRAPH GP
0.6486
0.6697 0.0070
0OM
0OM
LGNSDE (Our)
0.8101 1 0.0179
0.7214 + 0.0178
0,8263 a 0.0098
0.9165 0.0022
0.7100 a 0.0422
Table 3: AUROC (Mean + Std), AURC (Mean + Std), and Accuracy (Mean + Std) for all datasets with noise perturbations. Red denotes the best-performing model, and blue denotes the second-best- performing model.
We evaluate the models with noise added during testing to assess their robustness to input perturbations. No noise is introduced during training validation. At test time, Gaussian noise is applied to the input feature vectors. Specifically, the noisy inputs are defined as Xnew where (O or) represents element-wise Gaussian noise with mean 0 and standard deviation o, independently applied to each input feature. Here, a is computed as the standard deviation of X i... o std (Lex). By adding noise scaled by 0.5 times the standard deviation of the test set, we ensure that the perturbations are proportional to the feature distribution across all datasets.
X + 0.5 (O0 o),
Table presents the results under the noisy perturbation setting, where our model, LGNSDE, consistently ranks among the top two across all datasets and metrics (AUROC, AURC, and Accuracy). This demonstrates its robust performance under noise, frequently outperforming other models. These results also align with the guarantees provided by Proposition 2 which predict bounded deviations under input perturbations, supporting the observed robustness of LGNSDE.
5.4 ACTIVE LEARNING
We delve into decision-making under uncertainty in the context of an active learning setup, where the model selects its own training data. The experiments begin with the same set of observed labels as in the previous experiments (see Table(8). In each active learning round, 5 additional labels/nodes are incrementally included using an acquisition function, selected from either the validation or test dataset. After each new label is added, the models trained for 25 epochs, and this process continues until the number of newly added labels has doubled.
Figure 3 shows the active learning experiment conducted the Cora dataset nodes using two acquisition functions. In the right plot, labels are selected based on the highest predictive entropy, while in the left plot, they selected randomly.
7

---

Published as a conference paper at ICLR 2025
Random Acquisition Function
Max. Entropy Acquisition Function
0,88
0.85
0.86
Accuraty
0.84
Accuracy
0.81
0.82
0.82
Models
Models
GANDD (Oxe)
GNCFF Ersonble
I GASDD (Our)
GNCDF
0.78
0.78
Essontle
BOON
BGCN
10
Active Learning Round
Active Learning Round
Figure 3: Active learning on the Cora dataset using two acquisition functions. The left plot shows random selection of labels, while the right plot shows selection based on maximum predictive entropy
Metric
Model
Cora
Citeseer
Computers
Photo
Pubmed
GCN
0,9814 0.0019
0.9447 0.0017
0.9823 0.0017
0,9933 0.0006
0,9313
GNODE
0.9732
0.0030
0.9149 + 0.0080
0.9830 + 0.0028
0,9923 = 0.0008
0.9108
0.01
AUROC (t)
BGCN
0,9813 0.0024
0.8339 1 0.0341
0.9777
0.0028
0.9896 a 0.0023
0,7359 1 0.0406
ENSEMBLE
0,9712 0.0016
0.9139 0.0047
0.9830
0.0013
0,9926 0.0008
0.9243 0,0054 OOM
GRAPH GP
0,9434 a 0.0050
0.9060 1 0.0040
00M
0OM
LGNSDE (Ours)
0,9812 0.0016
+
0.9846
0,9933 0.0004
0.0050
GCN
0,0392 0.0037
0.1084 0.0065
0.0442
0:0020
0.0148 = 0.0017
0.0937 0.0066
GNODE
0.0439 0.0034
0.1396 # 0.0159
0.0512
0.0163 = 0.0028
0,1 + 0.0235
AURC
BGCN
0,0399 0.0049
0.2522 0.0795
0.0704
0.0102
0.0204 0.0069
0,3574 0.0695
ENSEMBLE
0,0543 0,0038
0.1410 = 0.0111
0.0027 0.0375
0,0143 0,0013
0,1007 + 0,0076
GRAPH GP
0,1027 0.0101
0.1975 + 0.0108
OOM
00M
OOM
LGNSDE (Ours)
0,0387 0.0033
0.1249 + 0.0106
0,0031
0,0148 = 0.0013
0.1030 0.0094
GCN
0,8626 0.0072
0.7720 + 0.0051
0.8734 + 0.0054
0,9315 = 0.0042
0,8130 + 0L0077
GNODE
0,8450 0.0088
0.7308 1 0.0269
0,8731 0.0069
0,9247 =: 0.0041
0,7888 0.0207
Accuracy
BGCN
0,8556 0.0115
0.5530 4 0.0992
0.8274 a 0.0272
0,9049 0.0150
0.4996 0.0721
ENSEMBLE
0,8384 0.0057
0.7253 + 0.0115
0.8844 + 0.0055
0,9280 = 0.0039
0.7911 + 0.0109
GRAPH GP
0.7727 0.0114
0.7148 1 0.0141
OOM
0OM
0OM
LGNSDD (Ours)
0.8641 0.0061
0.7534 + 0.01 16
0.8830 + 0.0058
0,9318 = 0.0012
0.8097 + 0.0111
Table 4: Results of the active learning experiments using a random acquisition function. The table reports AUROC, AURC, and Accuracy (Mean + Std) for all datasets. Red indicates the best- performing model, and blue indicates the second-best-perfone bert-performing model.
5.4.1 RANDOM ACQUISITION FUNCTION
To provide a baseline comparison with the maximum entropy acquisition function, we first evaluate the models using random acquisition function. At each active learning round, random labels are selected to be included the training set. Table illustrates the performance of the models under this setup, showing the AUROC, AURC, and accuracy metrics across all datasets.
This setup allows us to contrast the effectiveness of random label selection with more informed selection strategies like the maximum entropy acquisition function, which we explore next.
5.4.2 TOTAL ENTROPY ACQUISITION FUNCTION
Now, we evaluate the models using total entropy acquisition function, where labels with the highest uncertainty are selected at each active learning round. 3 the AUROC, AURC, and accuracy metrics for all datasets. Our model, LGNSDE, performs significantly better with the total entropy acquisition function compared to the random acquisition function (Table It achieves the highest AUROC on Cora, Computers, Photo, and Pubmed, showing that it effectively selects the most informative points when uncertainty is used as guide. In contrast, the random function led to more mixed results across these datasets. For the AURC metric, where lower is better, LGNSDE shows clear improvements with the entropy strategy, particularly on Cora, Computers, and Pubmed. This suggests that model is better reducing classification errors when it focuses uncertain points, compared to the random selection method. terms accuracy, LGNSDE also sees gains under the total entropy function. For example, accuracy on Cora improves from 0.8641 to 0.8889, and on
8

---

Published as a conference paper at ICLR 2025
Metric
Model
Cora
Citeseer
Computers
Photo
Pubmed
GCN
0,9831 0.0013
0.9434 1 0.0053
0.9881 0.0008
0.9936 1 0.0004
0,9248 1 0.01 29
GNODE
0,9789
0.0027
0.9419 + 0.0035
0.9838
0.0017
0,9914 0.0006
0.9113 0.0140
AUROC (t)
BGCN
0,9830 1 0.0026
0,8073 a: 0.0495
0.9809 A 0.0029
0,9915 a 0.0014
0,7169 + 0.0529
ENSEMBLE
0,9755 0.0023
+ 0.0049
0.9867 0,0005
0.916 0.0004
0.9236
GRAPH GP
0,9520 0,0026
0.9122 1: 0.0030
0OM
00M
OOM
LGNSDE (Ours)
0,9850 + 0.0010
0.9426 + 0.0057
0.9892 a 0.0001
0,9922 = 0.0002
0,9338 + 0.0050
GCN
0.0391 1
0.1104 1 0.0065
0.0425
0:0024
0.0153 0.0010
0.1051 1 0.0215
GNODE
0,0428 0,0050
0.1 1.36 + 0.0056
0.0658 = 0.0080
0,0206 = 0.0022
0.1 + 0.0199
AURC (1)
BGCN
0,0383 0.0045
0.2849 + 0.0595
0.0733 0.0176
0.0200 = 0.0053
0,3716 + 0,0681
ENSEMBLE
0,0524 0.0080
0. 1.1402+ 0.0081
0.0353+00000
0,0188 0,0010
0.1053 0.0089
GRAPH GP
0,1007 0.0039
0.2022 + 0.0043
0OM
OOM
OOM 0.0980 0.0094
LGNSDE (Ours)
0,0353 0.0027
0.1199 + 0.0110
0.0378
0.0010
0,0139 = 0.0007
GCN
0,8806 0.0065
0.7877 4 0.0077
0.8850 a 0.0045
0,9355 0.0020
0.7995 0.0238
GNODE
0.8710 0.0091
0.7836 + 0.0086
0.8708
0.0048
0.9302 = 0.0047
0.7799 + 0.0280
Accuracy (t)
BGCN
0.8728 0.0089
0.5167 1 0.0727
0.8208
0.0246
0,9104 0.0199
0.4788 0.0707
ENSEMBLE
0.8627 0.0063
0.7530 + 0.0075
0.8996
+ 0.0020
0,9410 + 0.0010
0.7924 + 0.01 19
GRAPH GP
0,7868 0.0061
0.7129 1 0.0051
OOM
0OM
OOM
LGNSDDE (Ours)
0,8889 0.0067
0.7826 + 0.0099
0.8984 0.0021
0,9381 0.0017 0.8208 0.0111
Table 5: Results of the active learning experiments using a maximum entropy acquisition function. The table reports AUROC, AURC, and Accuracy (Mean + Std) for all datasets. Red indicates the best-performing model, and blue indicates the recond-best-peffone
model.
Pubmed from 0.8097 to 0.8208, compared to the random acquisition setup. This shows that using uncertainty to guide label selection leads to better overall performance.
In summary, the total entropy acquisition function helps LGNSDE perform more effectively, particu- larly by selecting more informative data points than random selection, resulting in higher accuracy and better uncertainty management. For details on the effects of the hyperparameters and additional benchmark experiments, please refer to Appendix C
6 RELATED WORK
Uncertainty quantification in GNNs has recently gained attention, with contributions from Bayesian GNNs (Hasanzadeh al.) 2020). ensemble-based methods (Lin et al. (022), and Gaussian Processes on graphs (Borovitskiy et al. 2021 Sāez de Ocáriz Borde et al. 2024). We benchmark our LGNSDE framework against these methods and show improved performance across tasks, demonstrating more flexibility in capturing both aleatoric and epistemic uncertainty through a dynamic stochastic framework.
The work of Bishnoi et al. (023) introduces stochastic elements into graph learning but is restricted to learning scalar parameters for an SDE, unlike our method, which models the graph evolution directly as an SDE. Similarly, Poli al. (202I) attempt to propose Graph Neural SDEs with different formulation that uses a iinte-ddimeniionll KL divergence in the ELBO instead of an infiniteddimensional version. They also use prior distributions instead of prior processes, which better suited for modelling continuous dynamical systems. Hence, constructing a different method. Moreover, they do not provide theoretical analysis or thoroughly explore the model's uncertainty estimation capabilities beyond limited toy experiments.
The framework proposed by Lin et al. (2024) employs stochastic partial differential equations (SPDEs) to model message passing for uncertainty estimation, leveraging a novel Q- Q-Wiener process to propagate uncertainty directly within the graph diffusion process. While their approach emphasizes diffusion-basee uncertainty, our method adopts Bayesian framework with SDEs, focusing on quantifying uncertainty in the latent space instead. Furthermore, the work by [Stadler et al. (2021) on Graph Posterior Networks (GPN) takes a Bayesian approach to uncertainty estimation, focusing on interdependent nodes in graph-structured data. Their model explicitly performs posterior updates for nove-level classification by leveraging Dirichlet distributions and performs well in uncertainty- sensitive tasks. However, GPNs do not capture the dynamic evolution of node embeddings over time our LGNSDE does. Lastly, (Zhao al., (020) work uncertainty-awarc semi-supervised learning for graphs introduces a method which uses belief theory to quantify uncertainty types like vacuity and dissonance. Their focus semi-supervised learning and belief theory- uncertainty

---

Published as a conference paper at ICLR 2025
contrasts with our SDE-based approach, where uncertainty is captured through variance in latent representations and modelled using Bayesian updates.
7 CONCLUSIONS AND FUTURE WORK
We have introduced Latent Graph Neural Stochastic Differential Equations (LGNSDE), novel framework designed to quantify uncertainty in graph-structured data, By leveraging both epistemic and aleatoric uncertainty through a Bayesian prior-posterior mechanism and Brownian motion, LGNSDE provides meaningful uncertainty estimates. Theoretical guarantees demonstrate that our model ensures well-posedness, variance bounds, and robustness small perturbations in inputs. Empirically, LGNSDE performs competitively across range of benchmarks, excelling in out-of- distribution detection, noise robustness, and active learning tasks. Future directions include exploring higher-order SDEs, optimizing computational efficiency, and applying the model to real-world systems requiring reliable uncertainty quantification such as recommendation systems, drug discovery or dynamic networks.
REFERENCES
Cédric Archambeau, Manfred Opper. Yuan Shen, Dan Cornford, and John Shawe-taylor. Variational inference for diffusion processes. In J. Platt, D. Koller, Y. Singer, and S. Roweis (eds.), Advances in Neural Information Processing Systems, volume 20. Curran Associates, Inc., 2007.
Alvaro Arroyo, Alvaro Cartea, Fernando Moreno-Pino, and Stefan Zohren. Deep attentive sur- vival analysis in limit order books: Estimating fill probabilities with convolutional-taasffomme Quantitative Finance, 24(1):35-517 2024.
Suresh Bishnoi, Sayan Ranu, NM Krishnan, al. Graph neural stochastic differential equations for learning brownian dynamics. arXiv preprint arXiv:2306. 1435, 2023.
Viacheslav Borovitskiy, Iskander Azangulov, Alexander Terenin, Peter Mostowsky, Mare Deisenroth, and Nicolas Durrande. Matérn gaussian processes graphs. In International Conference Artificial Intelligence and Statistics, pp. 2593-2601, PMLR, 2021.
Rainer Buckdahn, Boualem Djehiche, and Juan Li. general stochastic maximum principle for sdes of mean-field type. Applied Mathematics & Optimization, 64(2):197-216, 2011.
Sergio Calvo-Ordonez, Matthieu Meunier, Francesco Piatti, and Yuantao Shi. Partially stochastic infinitely deep bayesian neural networks, 2024.
Luca Cardelli. From processes to odes by chemistry. In Fifth Ifip International Conference On Theoretical Computer Science-Tes 2008, pp. 261-281. Springer, 2008
Marija Cvijovic, Joachim Almquist, Jonas Hagmar, Stefan Hohmann, Hans-Michael Kaltenbach, Edda Klipp. Marcus Krantz, Pedro Mendes, Sven Nelander, Jens Nielsen, et al. Bridging the gaps in systems biology. Molecular Genetics and Genomics, 289:727-734, 2014.
C. Lee Giles, Kurt D. Bollacker, and Steve Lawrence. Citeseer: An automatic citation indexing system. Proceedings of the third ACM conference on Digital libraries, pp. 89-98, 1998.
Arman Hasanzadeh, Ehsan Hajiramezanali, Shahin Boluki, Mingyuan Zhou, Nick Duffield, Krishna Narayanan, and Xiaoning Qian. Bayesian graph neural networks with adaptive connection sampling. In International conference machine learning, pp. 4094 4104, PMLR, 2020.
Stefan Hoops, Raquel Hontecillas, Vida Abedi, Andrew Leber, Casandra Philipson, Adria Carbo, and Josep Maassaganya-Riess Ordinary differential equations (odes) based modeling. In Computational Immunology, pp. 63-78. Elsevier, 2016.
Thomas Kipf and Max Welling. Semi-supervee classification with graph convolutional networks. arXiv preprint a 1609 2016.
10

---

Published as a conference paper at ICLR 2025
Xuechen Li, Ting- Kam Leonard Wong, Ricky TQ Chen, and David Duvenaud. Scalable gradients for stochastic differential equations. In International Conference on Artificial Intelligence and Statistics, pp. 3870-3882. PMLR, 2020.
Qi Lin, Shuo Yu, Ke Sun, Wenhong Zhao, Osama Alfarraj, Amr Tolba, and Feng Xia. Robust graph neural networks via ensemble learning. Mathematics, 1308):1300, 2022.
Xixun Lin, Wenxiao Zhang, Fengzhao Shi, Chuan Zhou, Lixin Zou, Xiangyu Zhao, Dawei Yin, Shirui Pan, and Yanan Cao. Graph neural stochastic diffusion for estimating uncertainty in node classification. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Conference Machine Learning. volume 235 of Proceedings of Machine Learning Research, pp. 30457-30478. PMLR, 21-27 Jul 2024.
VB Mandelzweig and F Tabakin. Quasilinearization approach to nonlinear problems in physics with application to nonlinear odes. Computer Physics Communications, 141 -2,281 2001.
Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton Van Den Hengel. Image-based recommendations on styles and substitutes. In Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval, pp. 43-52, 2015.
Fernando Moreno-Pino, Álvaro Arroyo, Harrison Waldon, Xiaowen Dong, and Álvaro Cartea. Rough transformers: Lightweight continuous-time sequence modelling with path signatures. arXiv preprint XXiv:24455227999 2024.
Michael Poli, Stefano Massaroli, Junyoung Park, Atsushi Yamashita, Hajime Asama, and Jinkyoo Park. Graph neural ordinary differential equations. arXiv preprint
2019.
Michael Poli, Stefano Massaroli, Clayton M Rabideau, Junyoung Park, Atsushi Yamashita, Hajime Asama, and Jinkyoo Park. Continuous-depth neural models for dynamic graph prediction. arXiv preprint arXiv:2 10t.:2106..111818 2021.
Minh Quach, Nicolas Brunel, and Florence d' Alché Buc. Estimating parameters and hidden variables in non-linear state-space models based on odes for biological networks inference. Bioinformatics, 23(23):3209-324 2007.
Andreas RöBler. Runge-kutta methods for the strong approximation of solutions of stochastic differential equations. SIAM Journal on Numerical Analysis, 48(3):922-952, 2010.
Haitz Sáez de Ocáriz Borde, Alvaro Arroyo, Ismael Morales, Ingmar Posner, and Xiaowen Dong. Neu- ral latent geometry search: product manifold inference via gromov-hausdorfs sondddrrff--fomeed bayesian optimization. Advances in Neural Information Processing Systems, 36, 2024.
Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad.
Collective classification in network data. In Al magazine, volume 29, pp. 93, 2008
Oleksandr Shchur, Maximilian Mumme, Aleksandar Bojchevski, and Stephan Günnnemann. Pitfalls of graph neural network evaluation. arXiv preprint 1..i::181..05550 2018.
Maximilian Stadler, Bertrand Charpentier, Simon Geisler, Daniel Zügner, and Stephan Günnemann. Graph posterior network: Bayesian predictive uncertainty for node classification, 2021.
Winnie Xu, Ricky TQ Chen, Xuechen Li, and David Duvenaud. Infinitely deep bayesian neural net- works with stochastic differential equations. In International Conference on Artificial Intelligence and Statistics, pp. 721-738 PMLR, 2022.
Xujiang Zhao, Feng Chen, Shu Hu, and Jin-Hee Cho. Uncertainty aware semi-supervised learning on graph data, 2020.
11

---

Propusition 1. Under assumptions 3 and given Theorem there exists a unique mild solution to Published as a conference paper at ICLR 2025
A THEORETICAL REMARKS
A.1 NOTATION
Let G = (V,E) denote a graph with node set V and edge set & The node feature matrix at time H(t)) E Ba where is the number of nodes and d is the feature dimension. The evolution of H()) described by Graph Neural Stochastic Differential Equation, with drift function ( t, and diffusion function (H((),) t). Here, Fg depends the graph G, the node features Htt), time t, and parameters O. The diffusion function Gg depends on & and Ht but not on 0, as in practice, this is usually constant function. The randomness is introduced through the Brownian motion (()..
The constants f and Ty are Lipschitz constants for the drift and diffusion functions, respectively, ensuring the existence and uniqueness of the solution to the GNSDE. The linear growth condition controlled by a constant K, preventing unbounded growth in Fg and Gg. Finally, (ar(H(()) represents the variance of the node features, capturing the aleatoric uncertainty in the system, which also reflected the variance of the model output ý(t) h(H(t)).
A.2 TECHNICAL ASSUMPTIONS
Assumption 1. The drift and diffusion functions Fg and Gg satisfy the following Lipschitz condi-
tions:
( ()), - L H (1) H2(t) (H,(()()) Gg((H(().))l S T Hi H2(1)]
(3)
for all H1, H2 E Raxd, e [), and some constants Lg and Lg- Assumption 2. The drift and diffussio functions F& and Gg satisfy a linear growth condition:
(H)), O) + (H))( t)ll S K(1+ {
for all H E Raxd EE [0, and some constant K.
Assumption 3. The variance of the initial conditions, H()) Ho, of the dynamical system is bounded: E[] Holl?] < oc.
A.3 PROOFS
an LGNSDE of the form
dH(() = FG + (HH)
whose variance bounds the variance of the model output (t) as:
Var(S(1)) L Var (()()
where L2, is the Lipschitz constant of the readout layer. This ensures that the output variance is bounded by the prior variance of the latent space, providing controlled measure of uncertainty.
Proof. Using the result in [ et al. 2024
follows that the Lipschitz conditions of Fg and Gg ensure the existence and uniqueness of mild solution H(t to the GNSDE.
Now, consider the stochastic part of the variance of the solution. By applying the Itô isometry, we compute the expectation of the Frobenius norm of the stochastic integral:
I. Go (G(H()) udd((w)
(H)
'A mild solution to an SDE is expressed via integral equation involving the semigroup generated by the linear operator and represents weaker notion of the solution.
12

---

Published as a conference paper at ICLR 2025
Under the Lipschitz condition Gg, we can bound the variance of H(t) follows:
Var(H( = L. [Gg(H(u), dd..
If Gg is bounded, i.e., | ([Gg(H(u) w)llF M for some constant M, then Var(H(t)) S M This shows that the variance of the latent state H(t) is bounded and grows linearly with time, capturing the aleatoric uncertainty introduced by the stochastic process.
Finally, assuming that the model output (t) is a function of the latent state Htt.. = ((H(t),,
where h Raxd + Raxp is smooth function, we can apply Itō's Lemma follows:
dy(()) (H) (H(H,)) + (H(H(())))))))) (W) + (H()))
For the variance of (t), we focus on the term involving ()(H))))))dW))
(H((H) V / (H((H) (H) u) ( (H())) du.
Using the Cauchy-Schward incquality for matrix norms, we can bound this as follows: tr (H(()) Gg (H(u), u) G)((w)), h'(H()))) <
Therefore, if h is Lipschitz continuous with constant T,,,, then:
(I)) Lh / (H(((()),,)) u) LA Var V)r(H())).
Hence, under the Lipschitz continuity and boundedness assumptions for the drift and diffusion functions, the solution to the GNSDE exists and is unique, and its output variance serves meaningful measure of aleatoric uncertainty.
Proposition 2. Under assumptions consider two initial conditions Ho and H. = Ho + (0),
where (0) E aa a small perturbation the initial node features with [0][ = e. Assume that Ho is taken from compact set H C Rexd, Then, the deviation between the solutions H() and
H(t of the LGNSDE with these initial conditions remains bounded across time specifically
((] H(tt]l=) S (.r
Proof. Consider solutions H (!) and H2( of the GNSDE with different initial conditions. Define the initial perturbation as (0) where (0) = H. + (H) and H2( Hy. with [|0H(()]|==
The difference between the two solutions at any time t is given by öH(() = (t) Ht((). The dynamics of (t) are:
d() [F (t,, Fg (t)( (H, Gg (H)
Applying Itô's lemma to tr(SH(t) (H())), (t)), we obtain:
d(tr(oH(t) (oH(t)) = 2tr Fg - (, dt
2tr (1H( (H,(() (HH () dW(()
+ tr [GG (H,( Gg(HH(t), (t, Gg (Hz()), dt.
'Note that while the bound is exponential in f, in practice the time horizon is usually constrained to a limited range, such as f E [0, 1]. Within this interval, the exponential factor does not grow excessively, ensuring that the deviation between the perturbed and unperturhed solutions remains under control.
13

---

Published as a conference paper at ICLR 2025
Taking the expected value, the stochastic integral term involving (t) has expectation of zero due to the properties of the Brownian motion. Thus, we have:
E[d(tr(oH((t) (H()))) E (t)
+ G H (t). Gg tt 0))) dt dt.
Here, the second term arises from the variance of the diffusion term, as captured by Itō's Lemma. Using the Lipschitz bounds for F and Gg, we obtain:
((d(t)(&H((t) (H() s (2(H(((()H(())) (HH())) (())(( (H))))) L (H)()) dt.
Rewriting this as differential inequality:
(()()) (H(+)) & (2L, + (() (H())))
Solving this using Gronwall's inequality gives:
(t) (H({))) S (e)
Since |[ conclude that:
[[][)(()]]e] S (e
Hence, the deviation in the output remains bounded under small perturbations to the initial conditions, providing robustness guarantees.
A.4 LGNSDE AS CONTINUOUS REPRESENTATION OF GRAPH RESNET WITH STOCHASTIC NOISE INSERTION
Consider Latent Graph Neural Stochastic Differential Equation (LGNSDE) represented as
dH((t FFH t)dt + (G((),
where H(t) Bax Fg((((), t), and Gg (H(t),) Brownian motion. The numerical Maruyaaa discretization of this GNSDE can be expressed as
-valudd functions, and W(t) is
H(G++) H(4) E)) + ()) At
At
which simplifies to
H H H, F GG(HH
Here, At represents fixed time step and AW, is Brownian increment, normally distributed with mean zero and variance At. This numerical discretization is analogous to a Graph Recurrent Network (Graph ReNet) with specific structure, where Brownian noise injected each recurrent layer. Therefore, the Graph Neural SDE can be interpreted as a deep Graph ReNet where the depth corresponds to the number of discretization steps of the SDE solver.
'Noe that the second (stochastic part) he omitted the first dominates.
14

---

Published as a conference paper at ICLR 2025
B DETAILS OF THE EXPERIMENTAL SETUP
B.1 HYPERPARAMETER SEARCH
Table 6: Hyperparameter Grid Search Configuration
Hyperparameter
Values
Learning Rate
0.001, 0.005, 0.01, 0.T}
Weight Decay
0.011 0.001, 0.0005, 0.0001)
Epoch
[15, 100, 200, 300}
Dropout
0.0. 0.1, 0.3, 0.5)
Hidden Dimension
16, 32, 64, 128, 256)
Step Size
(0.01, 0.05, O.T, 0.2}
Table Hyperparameters left out of the grid search for all models and used for all datasets.
Parameter
GNSDE
GNODE
Other
f
N/A
Optimizer
Adam
Adam
Adam
Method
SRK
RK4
N/A
Early Stop
20
20
20
Diffusion Gg
1.0
N/A
N/A
B.2 ACTIVE LEARNING
Table 8: Dataset statistics before and after active learning.
Dataset
# Nodes
Links
Training/aliddationn/ess Split
Initial Training Labels
Final If Training Labels
Cora
2,708
5,429
140/500/1000
140
280
Citeseer
3,327
4,732
120/500/1000
120
240
Computers
13,752
245,861
200/500/1000
200
400
Photo 7,650
119,081
160/500/1000
160
320
Pubmed
19,717
44,338
60/500//000
60
120
B.3 LGNSDE HYPERPARAMETERS
Table 9: Accuracy vs. prior drift in Cora.
Table 10: Accuracy vs. prior diffusion in Cora.
Prior drift
Accuracy (%)
Sigma (o)
Accuracy (%)
-100.0
81.29
0.1
79.45
-1.00
82.21
0.2
80.13
5500
80.52
-1.0
82.17
0.5
81.32
-0.5
83.66
0.8
80.52
0.0
82.74
1.0
80.79
0.5
82,58
1.5
78.93
1.0
82,41
2.0
78.47
5.0
80.48
5.0
76.85
10.0
80.80
20.0
82.74
7.0
68.78
100.0
82.17
10.0
60.54
15

---

Published as a conference paper at ICLR 2025
C MORE EXPERIMENTS AND BENCHMARKS
C.I LGNSDE BACKBONE MODEL. GCN VS GAT
Metric
Model
Cora
Citeseer
LGNSDE-GAT
0,8249 0.0321
0.8003 = 0.0488
AUROC (t)
LGNSDE-GCN
0.7614 0.0804
0.8258 = 0.0418
LGNSDE-GAT
10.001
0.0536 0.0080
AURC (1)
LGNSDE-GON
0.0168 0.0070
0.0479 0.0109
LGNSDE-GAT
0.9474 0.0002
0.8642 = 0.0090
Accuracy (1)
LGNSDE-GCN
0.9471 = 0.0003
0.8729 = 0.0108
Table 11: AUROC (Mean + Std) and AURC (Mean Std) for OOD Detection across datasets. Red denotes the best-performing model, and blue denotes the second-best best-performing model.
Entropy Distribution: In-Distribution vs. OOD
LGNSDE-GCN
LGNSDE-GAT
GNODE
Denity
Densty
0.8
0.0
0.0
3.5
1.2
0.75
1.00
1.25
1.30
1.73
0.0
0.5
1.0
1.5
2.0
Entropy
Entropy
Entropy
Figure Illustrating two Drift function backbone the standard GCN Graph Attention Network.
C.2 COMPUTATIONAL COST COMPARISON
Model
Time per Epoch (s)
GCN
0.19
GAP
0.23
BGCN
0.36
GNODE
0.38
GRAPH
0.72
GNSD
0.92
ENSEMBLE
1.13
LGNSDE (Ours) 1.23
Table 12: Time Taken per Epoch for Different Models.
C.3 FURTHER BENCHMARKING AND COMPLEXITIES
Table 13: OOD detection performance comparison Cora with different OOD constructions.
Label leave-out
Feature perturbation
AUROC
AUPR in
AUPR out
FPR95
DET ACC
AUROC
AUPR in
AUPR out
FPR95
DET ACC
75.40
GAT
90.81
71.97
96.50
44.91
83.65
89.39
80.19
92,80
70.37
76.16
GRAND
88.58
75.72
94.28
63.59
34
87.04
74.81
93.42
55.35
76.43
GREAD
88.40
73.80
95.01
57.35
80.74
86.49
70.59
93.01
57.10
75.30
MSP
91.58
80.59
96.72
44.03
84.51
90.58
82.31
92.02
50.33
80.45
ODIN
49.24
24.27
75.45
100.00
49. 95
80
26.92
72.95
100.00
49 98
Mahulanobis
66.83
36.69
85.92
82.96
59.62
60.46
40.65
74.52
99.59
58.52
GNNsafe
92.68
82.03
97.48
31.54
83.48
93.28
88 16
96.35
43,43
83.71
GCN-Fasemble
90.97
80.35
97.37
29.92
85.73
89.01
79.71
94.57
61.74
74.57
BGCN
91.16
79.41
96.60
46.30
84.40
84.82
75.40
91,20
75.20
78.09
GKDE
82.80
62.
92.62
70.59
71.64
80.22
63.05
90.01
75.01
64.74
GPN
90.10
79.98
96.13
50.71
82.81
91.89
82.62
96.01
42.32
81.56
GNSD
94.76
88.45
97.73
27.38
89.74
91.77
94.41
88.23
26.27
86.92
LGNSDE (Ours)
93.14
85.01
96.65
49.03
81.42
90.61
92.77
31.63
83.05
16

---

Published as a conference paper at ICLR 2025
Table 14: OOD detection comparison on Amazon-Computers with different OOD constructions.
Model
Label leave-out
Feature perturbation
AUROC
AUPR
AUPR out
ACC
AUROC
AUPR
AUPR out
FPR95
DET ACC
GCN
82.35
56.46
93.67
56.06
74.72
80.55
78.53
78.55
80.67
75.09
GAT
80.66
53.19
93.05
53.91
72.65
73.69
78.00
65.61
97.41
75.76
GRAND
80.27
52.51
92.84
54.81
71.99
84.93
81.29
87.33
54.98
65.34
80.56
92.70
87.60
MSP
74,88
47,53
89.64
75.52
68.85
72.86
74.50
67.73
95.70
70.81
ODIN
71.78
70.54
77.09
83.09
Mahalanobis
71.87
37.76
89.87
70.24
50.18
74.47
67.54
76.28
82.48
50.04
90.50
95.05
84.01
75.62
GCN- Fasemble
79.53
52.39
91.99
69.
73.51
71.71
79:45
72.60
94.20
77.51
82.19
57.52
93,30
57.43
73.55
82.93
81.50
72.49
75.78
GKDE
76.46
48.18
90.64
73.36
64.35
71.09
71.40
69.04
90.83
69.70
GNSD
94,06
82.27
97.06
31.47
88.76
95.95
94.76
94,62
15.69
91.28
LGNSDE
Table 15: Time and Memory Complexity of Models
Model
Time Complexity
Memory Complexity
Graph Posterior Network (GPN)
O(N K)
O(N C)
Graph Gaussian Processes (GGP)
O(N
O(NM)
Graph Neural ODEs (GNODE)
O F2, NFEs)
O(1)
Latent Graph Neural SDEs (LGNSDE)
OLL L( d (Vid)
0(1)
Table 16: Explanation of Time and Memory Complexity Components
Component
Description
N
Number of nodes in the graph
K
Average degree of the nodes (used in GPN)
M
Number of inducing points (used in GGP to reduce complexity)
E.
Number of edges the graph
F
Feature dimensionality for nodes or edges
NFES
Number function evaluations during ODE solving (for GNODE)
L
Number of steps required by the SDE or ODE solver (for LGNSDE)
C
Number of classes in classification tasks (for GPN)
17


Reviews:
[{"review_id": "K08DRBlfhW", "reviewer": "Reviewer_SZ1C", "summary": "This paper generalizes GNODE to its stochastic counterpart, using SDEs to derive uncertainty-aware representations for graph-structured data. Theoretical and experimental characterization of the framework, named LGNSDE, showed its robustness and uncertainty quantification capability.", "questions": "- Have you tried varying the backbones of the GNN and is there any performance change? I think rewiring the graph structure might have a huge impact.", "limitations": "", "rating": 10, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "- The idea is sound and innovative.\n- The mathematical framework is simply elegant.\n- The robustness of this model is clearly demonstrated by theoretical and experimental results.\n- This framework can be of high utility to the community.", "weaknesses": "- I would imagine the result is heavily dependent upon the integration methods. See the extensive study conducted in GRAND: Graph Neural Diffusion, Chamberlain et al. 2021. \n- The accuracy of this model is not as performant as many cheaper variants.\n- I would love to see its speed and memory benchmark, as I imagine it to be quite expensive.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "p1x9y4hjKs", "reviewer": "Reviewer_kdL1", "summary": "The paper proposes a new GNN architecture version based on stochastic differential equations to improve uncertainty estimation.", "questions": "1. Can you elaborate on how the 'framework effectively quantifies uncertainty' based on Proposition 1? It is not immediately clear to me how this bounded output variance translates to effectively quantified uncertainty.\n\n\nWhile the topic and approach are interesting and the method appears potentially promising, I have concerns about the evaluation and the presentation, particularly the lack of differentiation from existing methods. If additional data and clarifications are provided, I would be willing to reconsider my rating.\n\n\n--- \nPost rebuttal: experimental evaluation seems good now.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The paper demonstrates good empirical performance.\n2. The authors evaluate uncertainty estimation through OOD detection, noise perturbation, and active learning.", "weaknesses": "1. The paper lacks comparisons with SOTA works. Some strong methods, such as GPN (Stadler et al., 2021) and GNSD (Lin et al., 2024), are mentioned but not evaluated, despite their better empirical performance compared to the baselines used. Additionally, other high-performing methods that utilize energy variants are not tested.\n2. The method shows significant similarities to Lin et al. (2024), who also propose an SDE-based GNN. While the authors acknowledge the difference with one sentence in the related work section, I believe the similarities and differences should be further discussed in detail, such as by comparing the frameworks mathematically and conducting a specific study (even on a synthetic dataset) to highlight the unique merits of their approach. But the method is not compared against at all.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MQ1YgSO6qD", "reviewer": "Reviewer_9vdg", "summary": "This paper introduces a new model (LGNSDE) for learning on graphs with uncertainty quantification. It leverages an Ornstein-Uhlenbeck prior and a posterior with drift parameterized by a GCN model that is trained through variational inference. This SDE approach is latent as it operates on node feature embeddings. Two novel theoretical results are shown, providing a bound on model output variance and a bound on solutions under node feature perturbations. Six graph models are then compared in various experiments on five standard graph datasets, including studies of OOD detection, test noise, and active learning. The proposed model performs favorably across these experiments.", "questions": "- Is it possible to allow a non-constant drift function in the prior OU process? Would a similar method or extension allow this?\n- What is the effect upon training and convergence for different choices of hyperparameters, and in particular, for different choices of constant drift and diffusion functions? Has an ablation study been performed for this proposed model?\n- How does the walltime of learning with an LGNSDE compare to walltime for the other tested methods?\n- Are there any interesting limitations on the practicality of this method, and in your experience, what can allow for learning the best results?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The LGNSDE model description is clear, concise, and intuitive.\n- The two theoretical results are important and original, providing a good deal of strength to the proposed methodology.\n- The experiments clearly show this model has significant potential in delivering upon the promise of uncertainty quantification for graph-structured learning problems.", "weaknesses": "- Some of the cited works (particularly Calvo-Ordoñez et al., 2024, and Xu et al., 2022) leverage SDEs in a very similar way, but do not consider the problem of learning on graphs. While the application to graphs is creative, the model definition is somewhat limited in novelty.\n- The choice of a constant drift and diffusion function in the OU prior is not sufficiently explored. It would be great if there was mention of why this is a reasonable restriction if this is indeed the case.\n- The experimental section would benefit from increased clarity and specificity, especially in regard to the structure and setup of experiments. Choices such as number of epochs and early stopping criteria, if any, are not explicitly stated.\n- It is stated in Section 5 that hyperparameters that achieved the highest validation accuracy were chosen. It is unclear to me which validation accuracy is used here, and knowing the exact grid search setup would be ideal for improved reproducibility.\n- Table 7 shows that uniform hyperparameters were chosen across all models; if the hyperparameter choice was performed via search for each model independently, the comparison results might be more fair.\n- The most impactful perceived weakness of this paper was the lack of experimental comparison with the models referenced as related works, particularly those referenced in Section 6. The other methods used in comparison do not follow a stochastic approach, limiting the ability for a fair comparison. I would be extremely interested to see a comparative study against the referenced GNSD (Lin et al., 2024) method, for example.\n- For OOD detection, comparison against methods built specifically for this task on graphs, like GNNSafe (Wu et al., 2023) would be enlightening.\n- One of the theoretical claims was a resistance to small perturbations in graph structure. This might lead to a compelling experiment, but this claim is not tested.\n\nThere are two small writing nitpicks:\n- In the first line of the introduction, \"Before the widespread of neural networks\" is missing a word. Perhaps use \"widespread success\" or similar?\n- In Section 6, \"Hence, constructing a different method.\" does not read as a complete sentence, and should probably be folded into the line before it.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Spotlight)