Energy-based Epistemic Uncertainty for Graph Neural Networks
Authors: Dominik Fuchsgruber
Tom Wollschläger
Stephan Günnemann
Venue: NeurIPS 2024 spotlight (2024)
Keywords: graph neural network
uncertainty estimation
epistemic uncertainty
energy-based model
Primary area: graph_neural_networks
TL;DR: We introduce a graph-based EBM that achieves state-of-the-art epistemic uncertainty estimation performance and can be applied post-hoc to any GNN.

Abstract: In domains with interdependent data, such as graphs, quantifying the epistemic uncertainty of a Graph Neural Network (GNN) is challenging as uncertainty can arise at different structural scales. Existing techniques neglect this issue or only distinguish between structure-aware and structure-agnostic uncertainty without combining them into a single measure. We propose GEBM, an energy-based model (EBM) that provides high-quality uncertainty estimates by aggregating energy at different structural levels that naturally arise from graph diffusion. In contrast to logit-based EBMs, we provably induce an integrable density in the data space by regularizing the energy function. We introduce an evidential interpretation of our EBM that significantly improves the predictive robustness of the GNN. Our framework is a simple and effective post hoc method applicable to any pre-trained GNN that is sensitive to various distribution shifts. It consistently achieves the best separation of in-distribution and out-of-distribution data on 6 out of 7 anomaly types while having the best average rank over shifts on *all* datasets.

Energy-based Epistemic Uncertainty for Graph Neural Networks
Dominik Fuchsgruber, Tom Wollschläger, and Stephan Günnemann School of Computation, Information and Technology & Munich Data Science Institute
Technical University of Munich, Germany
{d. fuchsgrubb,,, tom. wollschlaeger, .gggnnemannn @ttum
Abstract
In domains with interdependent data, such graphs, quantifying the epistemic uncertainty of a Graph Neural Network (GNN) is challenging as uncertainty can arise different structural scales. Existing techniques neglect this issue only distinguish between structure-awan and structure structure-agnostie uncertainty without combining them into single measure. We propose GEBM, an energy-based model (EBM) that provides high-quality uncertainty estimates by aggregating energy at different structural levels that naturally arise from graph diffusion. In contrast to logit-based EBMs, we provably induce an integrable density in the data space by regularizing the energy function. We introduce evidential interpretation of our EBM that significantly improves the predictive robustness of the GNN. Our framework is simple and effective post hoc method applicable any pre-trained GNN that is sensitive to various distribution shifts. It consistently achieves the best separation of in- n--distribution and of-distribuution data on out of anomaly types while having the best average rank over shifts on all datasets.
Introduction
Quantifying and understanding uncertainty is crucial to developing safe and reliable machine learning systems. Many applications such as Reinforcement Learning 42]. Active Learning 31 or Out-
of-Distribution detection [60] benefit from disentangling different facets of uncertainty 54 17 Typically, one distinguishes between an irreducible aleatoric component and reducible epistemic factors [29]. The former is inherent to the data, while the latter is rooted in lack of knowledge. One way quantify epistemic uncertainty is to define classifier-dependen density over the data domain 55 High values indicate similarity to the training data and thus imply low epistemic uncertainty. Energy-based models (EBMs) induce this density by defining an energy function that assumes low values near the training data (38) A common choice is the logits of the classifier as their magnitude supposed to correlate with the model confidence [40]. However, many architectures have been shown to produce arbitrarily overconfident predictions far from the training data (28). The negative
46)
implications for logit-based EBMs are only scarcely discussed in the literature B5 36
While substantial effort has been directed toward uncertainty estimation for independent and identi- 49 37 22), graphs have only recently received attention within the community [71 64 67 21. There, uncertainty can arise at different structural scales, e.g., from only single node, clusters, or global properties. Previous work only accounts for this implicitly by applying techniques from i.i.d. domains to Graph Neural Networks (GNNs) 67 49 A recent ap-
cally distributed (i.i.d)) problems 24
proach distinguishes only between structure-aware and structure-agnostic uncertainty combining them into single measure, which is often required in downstream applications [2] (4).
64 while not
Consequently, estimates are often only sensitive to shifts that match their structural resolution. They may overfit certain anomaly types and not be reliable in general.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Graph with Anomalies
Epistemic Uncertainty
Group
Softmin
Diflusion
f.
Graph Diffission
fo
Local
Softmin
ft)
Independent
Gaussian Regularization
Softmin
Figure I: Overview the Graph Energy-based Model (GEBM). Graph-agnostic energy (uncertainty) of trained GNN fG( is first regularized to mitigate overconfidence and then aggregated at local, cluster, and tructure-independenr scale by interleaving energy marginalization and graph diffusion. While group energy marginalizes before diffusion, local energy is fine-grt ane-graieed and can pick up conflicting evidence. GEBM assigns high uncertainty to several anomaly types simultancously.
We address these shortcomings and propose a novel graph-based EBM (GEBM) that is structure- aware different levels. The core idea behind GEBM is define energy functions at different structural abstractions and facilitate the flexibility of the EBMs to aggregate them into a single, theoretically grounded measure. Interestingly, we find that interleaving a graph diffusion process with energy marginalization gives rise to energy functions that naturally capture patterns of different granularity. As depicted Figure [ we utilize three energy types: Group energy corresponds to evidence smoothing and emphasizes anomalies clusters within the graph while not distinguishing between their type locally. (ii) Local energy more granular and sensitive evidence disagreements in the neighborhood of node. (iii) Independent energy is fully structure-agnont
and describes patterns that are limited to individual nodes. Aggregating them using soft maximum selection induces single energy measure that assigns high uncertainty to anomalies at various scales. A Gaussian regularizer provably ensures that GEBM converges to low confidence far from the training data.
We evaluate GEBM over an extensive suite of datasets, distribution shifts, and baselines It consis- tently exhibits ootte-of-thearr performance in detecting out-of-distribution (o.o.d.) instances, while other approaches are only effective in a subset of settings. On all datasets, GEBM ranks the best on average over all distribution shifts. Beyond o.o.d. detection, we discover novel theoretical connection between EBMs and evidential models. This interpretation of GEBM enables the GNN backbone to provide accurate predictions even under severe distribution shifts.
In summary, we tackle three main deficiencies of energy-based uncertainty for graphs by:
Proposing GEBM, an EBM that is aware of interdependence and provides single uncer- tainty estimate incorporating different structural scales.
2. Formally showing that Gaussian regularizer mitigates the overconfidence problem of logit-based EBMs and enables GEBM to provably induce an integrable data density.
3. Showing how to interpret GEBM as an evidential approach which considerably improves the predictive robustness of the classifier under distribution shifts.
2 Background
Semi-SSpeervieed Node Classification. We consider (semi-supervised) node classification on an attributed graph G (A, X) with n nodes and edges & that is represented by an adjacency matrix A {0. 1 and node feature matrix X E xx / Each node has label y: E {1....C) C). Given the labels of training nodes, the task is to infer the labels of the remaining nodes. Lastly, we focus homophilic graphs, i.c. edges are predominantly present between nodes of the same class.
We provide code ss..ci.um.deeddamllgeh

---

Uncertainty in Machine Learning. Usually, uncertainty regarding the prediction of model is disentangled into aleatoric and epistemic factors =d and uth, respectively. The former encompasses irrducible sources of uncertainty like measurement noise and inherent ambiguities. The latter is commonly defined as the non-aleatoric components of the overall uncertainty and can in general, be reduced, for example by acquiring additional data.
Energy-based Models. Epistemic uncertainty is commonly defined to be anti-correlated
classifierddependnnt density pe (a) over the input space [55]. We study Energy-based Models (EBMs) 38] which define joint energy Ez( y) over features and labels (a( y) x exp (-Eg ())) Marginalization over the labels induces feature density with normalizer Zg.
po(x) Zz² Σ exp (-E (ax v))) ZØ¹ exp exp(-Eo(aa) (=E)(x))
(1)
Intuitively, the energy Eg (x( = - log E, exp ( Ex, y)) can be interpreted as a soft minimum of the joint energies. is low in regions of high confidence and high elsewhere. The measure of
epistemic uncertainty implied by an EBM therefore is: u (x) ) = - -log po = Ez( + const. We
can also write the conditional class probabilities as:
Po(v x) = exp (- Eg (x. u)))
Σ exp (- Ea (x.
This is exactly the softmax distribution applied to - Foo y) which directly connects the logits fe(x) E IR predicted by a classifier to the joint energy by defining Es(æ, y) == - fe(()) [40].
Evidential Deep Learning. In contrast to first-order methods that directly predict p(y æ), evidential
methods [62 43 instead parameterize the corresponding second-order distribution from which first- sampled [32]. In classification, this second-orde distribution is a Dirichlet
order distributions distribution with parameters a >> O called evidence. They indicate the confidence of the classifier. For inference, a first-order prediction can be obtained by taking the expectation:
E Ep-Ditta) [p(y x)] oz Σ Oly
(3)
3 Related Work
Uncertainty Estimation for i.i.d. Data. Disentangling uncertainty has been approached from various perspectives. A family of sampling based approaches uses a Bayesian information-theoree framework (29] that relies on stochastic predictions derived from a posterior over model weights. The total uncertainty is defined on the mean predictor, epistemic uncertainty as the deviation of each
sample from that mean, and aleatoric uncertainty as the difference both estimates 1 The posterior
either be explicitly modeled by Bayesian Neural Networks (BNNs) 66
13
15
18] Monte-Carlo
Dropout (MCD) (22). or implicitly realized by ensemble methods [37 77 Test-time augmentation
and Stochastic Centering [66] also provide samples from this posterior. Sampling free approaches are deterministic and estimate uncertainty with a single forward pass. Evidential methods [62
72 73
68 predict second-order distribution from which epistemic uncertainty is derived. Distance- based approaches quantify epistemic uncertainty as similarity to the training data [46] 48 20, 65 and are closely related to density-based uncertainty estimation. (Deep) Gaussian Processes (GPs) use a kernel function to measure this similarity but do not disentangle epistemic and
52
45 39
aleatoric uncertainty. Posterior Networks combine evidential and density-based approaches by predicting density-based updates to the evidence
Uncertainty Estimation for Graphs. Many of these approaches transfer graphs: DropEdge (59 83
applies MCD to edges and GPs can utilize a structure-aware kernel 53
41
60).
SGCN 82
proposes evidential student-teacher approach while G-AUQ (67 applies Stochastic Centering to a GNN and improves on calibration. Graph Posterior Network (GPN) 34 diffuses the density-
based evidence of Posterior Network but provides separate measures for structure-aware and structure-agnen agnostic uncertainty, each of which is only effective on some distribution shifts.
Energy-basse based Models. EBMs 38 3 2
are typically employed in generative modelling [14] but
have also been applied to uncertainty estimation [16] and anomaly detection [40 gT. To address
the overconfidence of logit-baser based EBMs far from training data, Gaussian regularization term has
3

---

recently been proposed [35] that we employ in our framework as well. While their work empirically validates this adjustment, we formally prove the overconfidence of logit- baseed EBMs to happen with high probability and Gaussian regularization to mitigate the issue. The HEAT framework (36] learns multiple corrected EBMs via stochastic gradient Langevin dynamics [76 and composes them into a single measure. In contrast, our approach does not require additional training and aggregates energy that emerges naturally at different scales in the graph from a single logit- joint energy model. Lastly, GNNSafe 77 diffuses the logit-based energy of GNN to improve its out-of-distribution detection, In contrast, our model considers energy beyond the cluster scale.
Method
We develop a simple yet effective Traph-Energy- Model (GEBM) for post hoc epistemic uncertainty that is sensitive variety of distribution shifts from any pre-trained GNN.
4.1 Energy at Different Scales
Previous work either does not distinguish between uncertainty at different structural resolutions at all or only disentangles structure-aware and structure-agnostic factors without combining them (64] However, for many practical purposes, epistemic uncertainty must be quantified with a single measure We address these issues by proposing EBM-based uncertainty that incorporates
79
42 5 60)
patterns on different natural abstractions of a graph. The density induced by GEBM, pe (æ serves as a singular uncertainty measure that sensitive multiple distribution shifts simultancously.
Inspired by the success of graph diffusion 2 23 we propose energy on different structural levels: (i) Graph-agnostic. for node features in isolation. (ii) Based the evidence in the local neighborhood of a node. (ii) Within clusters in the graph. Note that defining global energy on the whole graph induces no differences at the node level and therefore we do not consider it in this work. We point to appropriate measures for a potential extension of GEBM in the existing literature [84).
We make the intriguing observation that interleaving a diffusion process PA R + R with the marginalization of structure- aggnostcc joint energy Eg y) as in Equation functions that describe the aforementioned natural abstraction levels. Based on this insight, we can derive definitions for three types of energy functions:
11 induces energy
Independent Energy. On the finest scale, we consider energy independent of structural effects by omitting the diffusion operator. This term captures uncertainty regarding node features in isolation.
(x) log Σ exp exp (-Fo(xx
Local Energy. On a coarser scale, we diffuse the joint energy before marginalization. This retains local information like conflicting feature-based evidence within the neighborhood of node as the class-specific information is marginalized after propagation:
Fo,z (x)) = log Σ
exp [PA (=Fo(x,))) y))
Group Energy. By interchanging marginalization and diffusion, we effectively smooth the marginal evidence Fx)). Therefore, energy gets propagated predominantly within clusters of the graph.
Fa, = - PA log Σ exp (-Fo Fx v))
3
Since marginalization is done before propagation, less local information is preserved: The energy of a node will increase when its cluster is anomalous, regardless of whether the type of anomaly matches its own. As can be seen exemplary in Figure [ this loss of local information comes at the benefit of being less exposed to local variability within coarser clustered patterns.
Each energy type captures anomalies that affect the corresponding structural scale. We provide synthetic experiments Appendix C.7 as an additional intuitive explanation for the aforementioned energy terms. In practice, GEBM enables practitioners to augment our framework with further, potentially tpec-fp,, energy functions. We empirically find however that the combination of the three naturally arising energy terms already detects broad range of distribution shifts.

---

4.2 Regularizing Logit-based EBMs
Following Equation
I EBMs imply density over the data domain po(x) with normalization
constant:
Zo / Σ exp (-, v)) dx 8
When quantifying epistemic uncertainty with this density, the energy must be low regions of the input feature space close to the training data, while high values should be assumed far away. To that end. (x) must be integrable, that is, have a finite normalizer Zg. However, previous work has shown that piecewise affine classifiers which are the backbone of modern GNNs will converge to overconfident logits far from training data (28). We remark that previous studies found GNNs to be underconfident in-distribution data 774
75 while the aforementioned issue of overconfidence arises from high distance to training data induced by a distribution shift (see Appendix C.8). This overconfidence is problematic, logit-based based energy can suffer from the same issue and Egg (as) may assume arbitrarily small values far from the training data. This contradicts the aforementioned desideratum of low confidence and the implied uncertainty measure breaks under severe distribution shifts. The normalizer of the logit-based joint energy Zo is not finite with a high probability and therefore the EBM not induce integrable density Po (=)).
Proposition 4.1. Let fe B. RC be a piecewise affine function and Rh - Qi be the disjoint set of polytopes on which Sa is affine, i.e. fa(æ) w + b(0) for x E Qt. Assuming the direction of the rows of each W be uniformly distributed, the probability that Zo converges decreases exponentially in the number of non-closed linear regions L' and classes C.
Pr Zo < oo] = =01221/))
We provide proofs for all claims in Appendix A Intuitively, since fe behaves like an affine function in the limit || co, its predicted logits may diverge toward co. If for any class the model produces overconfident predictions in one of its affine regions, the marginal energy Eg (x) will diverge toward maximal confidence. As classifiers the actual probability of a well-hehaved energy function to be even lower than our theoretical bound that assumes uniform weight directions. In practice, this pathological behavior of logit-based EBMs has been observed in previous work [64][3] 35). We mitigate this issue by augmenting the logit-based energy and we prove that this regularization induces an integrable density.
trained to output high values for one of the classes, expect
4.3 Our Model
Similarly to recent work on EBMs [55,, we employ a class-conditional Gaussian prior ((f((x) My Σy) regularizer for the logit-based energy. We learn the parameters {Hy Enyy=1 as the maximum likelihood estimates from the training instances of each class.
Ex(x, y) = Eg (x. y) - y Nof My: En)
(8)
The regularization strength y > 0 and the choice of the diffusion operator PA the only hyperpa- rameters GEBM. Each of them has an intuitive interpretation: 7 controls the trade-off between the predictive dependency and distance-awarenes, of the EBM while PA encapsulates how information propagated over G. Regularization ensures that the corresponding marginal density po (x) has a finite normalizer Zo and is therefore integrable.
Theorem 4.2. For a piecewise affine classifier fe as in Proposition 4.1 Po is well-defined.
Zo
Σ exp(-Eo(ax ()) pa y < 0
This regularized joint energy diverges toward maximal uncertainty in the limit. Consequently, its induced density provides an uncertainty estimator that is reliable even far away from the training data.
Corollary 4.3. For a piecewise affine classifier fo as in Proposition 4.1 and any E Rd almost surely:
lim po(ax)) =
a-+oc
5

---

evidential classifier We then combine the regularized energy Eg (2x, y) at different structural scales (Equations (4 to (6).
Ee (x) - log exp (Eo, + exp (Eo..(a)) + exp (Eo,c(x))
(9)
Since energy defined per node, most diffusion processes (Appendix (.6) act on individual nodes as an affine function with positive coefficient (see Appendix A.2). Therefore, each individual energy and their aggregate, GEBM. induce an integrable density when using a linear diffusion operator PA.
Theorem 4.4. For a linear diffusion operator (a(x)- = aæ + const, a > 0 and the regularized energy Fg( GEBM induces an integrable density:
J exp (-Ro,GERM (a:) dæ < 00
GEBM is lightweight and can be applied post hoc to any logit-based GNN without additional training. Its induced density is a proxy for epistemic uncertainty: (x)) log (x) and we compute aleatoric uncertainty as the entropy of the predictive distribution of the GNN [29]. Following 79.. we realize the diffusion operator as a repeated application of label-propagation scheme (see Appendix C Intuitively, this operator is a smoothing process which is only appropriate when assuming the in-distribution data be homophilic. In the case of non-smooth (heterophilic) training data, the proposed energy would be high for in-distribution data which is undesired behaviour for an EBM-based uncertainty estimator. To disentangle effects at different structural scales, we define each of GEBM's components on structure-agnostr regularized joint energies Fo y). To that end, we compute the outputs of the classifier by omitting the structure for the computation of Fgg only by evaluating fo (I, X), i.e. setting A I. These outputs depend only the node features.
4.4 EBMs as Evidential Models
Beyond using epistemic uncertainty to detect anomalies, we also show how the joint density induced by GEBM enables predictions that robust against distribution shifts. First, we discover a cor- respondence between first-order predictions of logit-based classifiers (Equation (2)) and evidential predictions (Equation This connects the joint energy F0 y) of an EBM to the evidence ar:
C exp - Ex y)) pa(x, 3)
(10)
For an integrable density (a, y). the evidence a will vanish the limit far away from training data. Previous work on Posterior Networks [1] I0] fits normalizing flow to obtain class-conditional densities Po (a: y) and uses them to compute Bayesian update prior evidence prior, Its extension graphs, GPN (64), diffuses these structure-agnostic updates with operator PA- Similarly, our GEBM framework induces normalized joint density (a y) through its regularized energy function at no additional cost. Therefore, it can also be interpreted
that enables inference according to Equation
GPN 64 (Evidential)
GEBM (ours)
Q post
prior
prior
PA (N po(x y))
post
a prior + PA Lg exp(- Eg 8)
prior
Here, N is called an uncertainty budget and it roughly corresponds to the normalizer of the joint energy. This interpretation of EBMs recovers desirable properties of density-based evidential methods IT 64]: Predictions will converge toward a prior otoor far from the training distribution. They will be less affected by anomalies in the neighborhood of node and therefore be more robust against distribution shifts. We remark that while our framework enables this evidential inference scheme, it also possible to instead use the backbone classifier as-is and preserve its predictive performance.
5 Experiments
We evaluate the efficacy of GEBM by extending the evaluation proposed in 64 and expose it to a suite of 7 distribution shifts from three families that cover broad range of anomaly types.

---

5.1 Setup
Datasets and Distribution Shifts. We use seven common benchmark datasets for node classification: The five citation datasets C ConaML-LLM 78) (4), Citeseer (511 PubMed [5]], Coauthor- Physies and Coauthor- Computers 633, and the co-purchase graphs Amazon Photos and Amazon Computers 44 We expose all methods to three families of distribution shifts:
Structural. We select nodes with the lowest homophily as o.o.d. and train homophilic nodes only. This induces a shift in the local connectivity pattern of the nodes. Furthermore, we include setting in which nodes with low Page-Rank (PR) centrality (56] considered o.o.d.
(ii) Leave-out-Class. We withhold nodes belonging to a subset of classes during training and reintroduce them during inference. In practice, this might, for example, correspond to new type of user joining a social network. We either select the held-out classes randomly or pick those with the lowest average homophily to make them more dissimilar to the retained classes.
(iii) Feature Perturbations. We randomly choose a subset of nodes and perturb their features by replacing them with noise. Since most datasets have categorical bag-of-wordh features, we have fine-grained control over the severity of the shift in this setting. We generate near-o.o.d. data by sampling from a Bernoulli distribution p fitted on the node features of the dataset X. A stronger shift is induced by fixing the success probability at value of p 0.5. Drawing node features from (O, 1) constitutes a domain shift (ar-o..od.) for categorical data.
Existing work concerns transductive node classification which enables leakage of o.o.d. information during training [64] (9). Instead, we study the inductive setting and remove o.o.d. nodes and their All results were
edges during training, and provide results for the transductive setting in Appendix C averaged over 5 splits and initializations each (for standard deviations, see Appendix C
We compare GEBM to different baselines for (epistemic) uncertainty es- timation: Ensembles (GCN-Ens), MC-dropout (GCN-MCD), DropEdge (GCN-DropEdge), combination of MC-dropout and DropEdge (GCN-MCD+DropEdde) and a Bayesian GCN (BGCN) are sampling-based approaches. We also compare against the logit-based EBM (GCN-Energy), GNNSafe, and HEAT as EBM baselines. Lastly, we consider GPN and SGCN as evidential methods. For all models. including GEBM, we use the same backbone architecture (see Appendix [.2).
Model and Baselines.
Metrics. We evaluate epistemic uncertainty by detecting distribution shifts. That is, we report the AUC-ROC and AUC-PR metrics for the binary out-of-disttriuuion detection problem of separating in-distribution (i.d.) from diitr--dittiittionn (o.o.d.) nodes. Additionally, we report how well uncertainty correlates with erroneous (misclassification detection) in Appendix C.3] Since proposed method does not alter the softmax predictions of the backbone model, it will affect neither the aleatoric estimates nor their calibration. Consequently, our framework is open additional post hoc calibration methods such as temperature scaling [26). For completeness, we report the Expected Calibration Error (ECE) (50) and the Brier score
in Appendix C.4 for the GNN backbone.
5.2 Results
Dittr--Distribution Detection. Table [ shows the performance of different aleatoric and epistemic uncertainty methods on various distribution shifts (full results in T). Across all datasets and distribution shifts, our model provides the best or bestnd-best separation of o.o.d. data from i.d. data. In practice, an estimator that is effective all distribution shifts simultaneously is desirable. Therefore, we rank (Q) all estimators individually for each shift and dataset and report its average rank over all distribution shifts for each dataset. To not favor one distribution shift family, we adjust the weight such that Leave-out-Claas same amount. We rank all epistemic uncertainty proxies both against other epistemic measures and aleatoric uncertainty separately. In both cases, Tables [ and 7 show that our proposed EBM- epistemic uncertainty is the only estimator that is effective under different distribution shifts at the same time. On all datasets, GEBM improves the AUC-ROC scores by 5.8 to 10.9 percentage points on average (16%-32% relative improvement) over vanilla EBM, the second best-ranked estimator (Appendix (.2). Since all 1BM-based approaches (GCN-EBM) GCNSafe, GEBM) are post hoc methods, they share the aleatoric uncertainty and high predictive accuracy of the backbone.
structural shifts, and feature perturbations contribute the
Out of all 7 distribution shifts, our framework is only less sensitive to the centrality shift. Many basclines only perform well because of their symmetric diffusion operator that biases any arbitrary
7

---

LoC (laxt)
(cn(
(O 1) (aa)
Homophily
Model
AUC-ROCT
Ace.1
AUC-ROCT
Acc.1
AUC-ROCT
Ace.1
AUC-ROCT
Acc.1
(Alca. / Epi.)
(Alca. / (pp.))
(Alca. / Epi.)
(Alca. / (Ep.)
GCN-DE
86. 2673/73.0
87.2
47. 1/68.8
71.9
26.8/57.8 8
71.9
70.8/54.5
85.6
GCN-Ens
89.8/73.9
90.4
59. 19.4/64.1
75.9
30.4/52.7
75.9
71.5/68.5
91.5
CoraML
GPN
85.3/88.1 1
88.5
53.4/52..4
72.8
51.4/55.5
72.8
66.9/51.5
87.6
GCN-EBM
89.7/89.9
90.3
59.9/58.6
75.8
31.7/25.7
75.8
71.3/71.2
91.2
GCN-HEAT
89. 89.7/87.1
90.3
59.9/64.4
75.8
31.7/76.0
75.8
71.3/70.0
91.2
GCNSafc
89.7/91.6
90.3
59.9/53.4
75.8
31.7/35.3
75.8
71.3/73.1
91.2
GCN-GEBM
89.7/91.6
90.3
59.994455
75.8
7/87/66.4
75.8
71.3/76.7
91.2
GCN-DE
76.4/61.2
91.7
52.8/5144
88.5
41.8/51.5
88.5
62.7/52.0
96.6
Amazon Photo
GCN-Ens
74.7/80.3
91.9
52.3/51.3
89.9
42.6/55.0
89.9
62.0/6.22
GPN
74.5/87.3
88.8
55.3/54.9
86.5
54. 2452555.5
86.5
65.9/47.0
97.1
GCN-EBM
74.6/75.2
91.8
52.1/52.1
89.8
24.2/12.6
89.8
61.9/60.0
97.4
GCN-HEAT
74.6/77.3
91.8
12.1/51.8
80.8
44. 2/57.3
89.8
61.9/62.6 6
97.4
GCNSafe
74.6/75.6
91.8
52. 1/5199
89.8
44.2/46.4
89.8
61.9/60.5
97.4
GCN-GEBM
74.6/83.6
91.8
52. 1764.2
89.8
4/.2/92.4
89.8
61.9/68.3
97.4
GCN-DE
89.2/73.7
91.3
64.3/67.9 9
88.8
7/60.3
88.8
62.9/55.7
97.1
GCN-Ens
89.7/87.3
92.4
69.6/67.88
90.0
32.8/6U.7
90.0
63.9/69.1
97.8 .9
Couuthor-CS
GPN
73.7/88.3
86.6
1/59.7
81.7
53.6/59.9
81.7
61.3/45.5
97.9
GCN-EBM
89.7/89.9
92.3
69. 09.3/71.66
89.8
34.7/28.8
89.8
63.8/60.5
97.8
GCN-HEAT
89.7/88.6
92.3
69.3/70.0
34.7/69.8
89.8
63.8/66.6
GCNSafe
89.7/92.1
92.3
69.3/ .9.3/63.0
S0.8
7437/36.9
89.8
63.8/62.4
97.8
GCN-GEBM
89.7/92.8
92.3
69.3/99.5
89.8
34. 7/90.0
89.8
63.8/69.5
97.8
Table 1: Out-of- of-tsssributtin detection AUC-ROC (1) using aleatoric or epistemic uncertainty (best and runner-up). Our epistemic measure achieves the strongest performance most datasets and shifts while maintaining the classification accuracy of the GCN backbone.
CoraML LLM
Amazon Computers
Amazon Photo
Coauthor CS
Coauthor Physics
Model
CoraML
Citeseer
PubMed
GCN-DE
6.3/14.2
8.0/16.9
7.8/16.2
7.3/13.6
7.2/15.9
7.1/15.8
6.5/13.2 2
8.1/16.8
GCN-Ens
7.8/16.1
6.3/15.0
6.4/12.9
6.2/12.4
5.7/11.6
4.7/9.0
4.8/10.4
6.2/13.3
GPN
7.0/15.2
5.4/11.2
6.6/12.9
5.8/12.4
5.7/11.8
5.3/11.7
7.2/13.9
6.1/14.5
GCN-EBM
4.5/7.9
4.0/8.2
5.2/10.3
5.2/12.0
4.2/10.7
4.9/10.9
4.1/7.1
4,4/8.4
GCN-HEAT
4,4/10.4
5.4/12.9
3.8/9.2
5.6/12.8
3.5/8.6
4.5/10.2
4.2/8.3
4.7/10.4
GCNSafe 5.3/9.1
4.4/8.1
5.2/9.5
5.4/8.9
5.9/13.2
6.4/12.9
5.5/10.7
5.8/11.7
GCN-GEBM 2.7.4.5
2.974.
3.77
3.8/7.4
2.5
2.774.5
2.774.3
3.0/5.0
Table 2: Average o.o.d. detection rank (L) of epistemic uncertainty versus other epistemic measures all uncertainty measures over all distribution shifts (best and runner-up), GEBM has the best average performance rank over all distribution shifts and epistemic (and aleatoric) uncertainty estimators.
signal toward high-degree nodes regardless of the quality of the uncertainty (see Appendix C.6). While this bias could be explicitly incorporated into GEBM by defining an additional energy term, the goal of our work is not to engineer measure toward certain downstream settings. GEBM is already sensitive to many distribution shifts as-is. Even after accounting for this setting, our estimator achieves the best average rank over all shifts. We defer further discussion to Appendix [
Misclassification Detection. alignment with previous work (64, we observe aleatoric uncertainty to be more effective for misclassification detection than epistemic estimates. Our method performs competitively among other epistemic measures as shown in Table [6] As previously discussed, improving aleatoric uncertainty for GNNs is beyond the scope of this work.
Robust Evidential Inference. As described in Section 4.4 GEBM enables evidential inference. We expose our model to feature shifts sampled from (0, 1) and increase the fraction of perturbed nodes. Figure shows the predictive performance of the classifier induced by an evidential interpretation of the EBM at different regularization weights 7 compared to vanilla EBM and the evidential model, GPN. We maintain high accuracy like an evidential model, while the performance of the baselines rapidly deteriorates. The hyperparameter y can be seen as interpolating between an unregularized overconfident logit-based EBM and an uncertainty-aware evidential method. While GPN requires explicit evidential training, our method enables robust inference at negligible additional cost from a pre-trained GNN. Furthermore, GPN notably sacrifices predictive capability on clean data, while GEBM enables more control over the trade-off between accuracy clean and perturbed data.

---

80
GCN GCN-EBM
Logits fx(x)
77
GPN
0
log.V Σ)
GEBM (oun))
AUC-ROC -E(x))
Ex(x)
Accuracy (56)
80
==1
5.0
with A
60
2.0
40
1.0
1
20
0.5
0
0
50
100
2.5
5.0
7.5
10.0
Fraction of Perturbed Nodes (%)
Magnitude ||>||
Figure 2: Accuracy of evidential inference at Figure 3: Logit-based, Gaussian, and regularized different regularization y. GEBM performs on- energy different magnitudes. Logit-EBMs be- par with evidential methods at increasingly se- come overconfident while the corrected energy vere perturbations.
eventually dominated by its regularizer.
The advantage of this evidential perspective is that the evidence approaches zero under increasingly severe distribution shifts. Therefore, fixed number of diffusion steps suffices effectively counteract the influence of anomalous neighbors when making predictions for node. In contrast, using diffusion at, for example, the predictive level is inadequate: As shown Proposition 4.1 the energy is likely to diverge and, therefore, the number of diffusion steps necessary mitigate arbitrarily severe distribution shifts unbounded. Consequentially. we find that even compared to models like APPNP that heavily rely on graph diffusion at the predictive level, the evidential interpretation of GEBM notably improves robustness (see Figure 6).
5.3 Ablations.
Energy at Different Scales, In Table 3 we compare GEBM to energy specific structural scales and vari- ant without regularization (y 0). Each of the former corresponds one of the energies that the GEBM frame- work is composed of. Group energy is equivalent to regularized logit-based EBM which is therefore also ablated implicitly. As expected, each component is sen- sitive to different shifts. This confirms that interleaving marginalization and diffusion captures patterns at dif- shown in Table 26, a variant of
Model
LoC
A/(D 1)
Homo.
Rank(
EBM
87.5
24.7
67.9
4.1
Indep.
79.2
99.7
63.9
3.7
Local
86.9
Group
91.7
62.8
68.9
3.3
GEBM-E.
90.1
1.4
71.1
3.1
GEBM
Table 3:
-dete--dttt AUC-ROC(T)
using different EBMs (best, runner-up). GEBM combines the benefits of energy at different scales and ranks best over splits.
ferent resolutions. the aggregate GEBM achieves the best rank on of 8 datasets. This shows that scale-awareness is the key ingredient for effective uncertainty estimation on graphs which can be further improved with energy regularization. In far-o.o.d. settings, this regularization is crucial to obtain reliable estimates as it mitigates overconfidence issues.
Energy Regularization. We also ablate the effect of reg- ularizing the joint logit-based energy in Figure creasing distance from the training data, logit-based energy contrast, the regularized Eg
Model
LoC
Ber(
W/t0, I)
Homo.
in-
GCN
EBM
89.9
67.1
26.4
71.2
Safe
91.6
56.9
36.1
73.1
becomes overconfident. follows the logit-based energy near the training data and dominated by the regularizer far away. This is reflected in the clear separation between perturbed and unperturbed nodes. When computing structure-awee energy, feature corruptions affect unperturbed nodes through the diffusion operation, making a clean separation difficult. This justi- fies our choice compute structure-an structure-agmostic joint energy (z. y) and factor in the graph afterward by applying PA-
GFBBM
91.6
77.1
86.6
76.7
GAT
EBM
90.2
35.6
11.1
72.1
Safe
91.5
53.2
45.4
70.3
GEBM
69.3
76.5
72.6
EBM
76.5
52.3
42.8
53.2
GIN
Safe
49.3
GEBM
80.7
51.4
$3.6
65.4
SAGE
EBM
74.0
52.7
42.2
533.2
Safe
77.3
49.2
46.6
51.1
GEBM
77.3
51.6
54.7
62.7
Table 4: O.o.d. detection AUC-ROC(1)
using different backbones. Our method is effective on all architectures.
Backbone Architecture. Our method can be applied post hoc to any logit-based GNN. detection performance of our EBM framework using differ-
evaluates the o.o.d.-

---

ent commonly used GNN backbones: GCN [4), GAT (v2) 70 8 GIN 80 and GraphSAGE [27].
Standard deviations and AUC-PR are reported Appendix C.5 Over different distribution shifts, our proposed approach consistently outperforms the logit-based EBM and the graph-specific GNNSafe variation. This shows the broad applicability of our framework that enables reliable high-quality epistemic uncertainty estimation from a large family of logit-based GNNs.
Limitations and Broader Impact
Limitations. As GEBM is post hoc epistemic estimator, it does not improve alcatoric uncertainty or its calibration. In particular, the GCN backbone used in this work does not consistently achieve the strongest performance in both tasks. While the structural scales that arise naturally from graph diffusion cover many distribution shifts as-is, specific applications may require augmenting GEBM also observe for centrality-basee shifts. While GEBM enables
with additional energy terms, robust evidential inference, future work may build upon its paradigm of aggregating different structural scales in the graph for fully evidential methods. Lastly, we study homophilic node classification problems and leave an extension of GEBM beyond this setting to future work.
Broader Impact. GEBM enables cheap and simple uncertainty quantification for GNNs which we believe to contribute to the development of more reliable AI. Nonetheless, we encourage practitioners to actively reevaluate measure of uncertainty to mitigate risks in safety-critical domains.
7 Conclusion
We propose GEBM, simple and efficient EBM for post-hoc epistemic uncertainty estimation for GNNs. To the best of our knowledge, we are the first to consider uncertainty at different structural scales and address this gap by proposing a model that aggregates energy that naturally arises from graph diffusion. It consistently outperforms existing approaches over an extensive suite of datasets at detecting various distribution shifts. We formally and empirically confirm that logit-based EBMs suffer from overconfidence and prove that the regularized GEBM mitigates this issue by inducing an integrable data density. We exploit this property by discovering link evidential methods that enables the backbone to provide accurate predictions even under severe distribution shifts.
Acknowledgements
We want to give special thank to I Leo Schwinn and Franz. Rieger for giving helpful suggestions on an early draft of the manuscript. The research presented has been performed in the frame of the RADELN project funded by TUM Georg Nemetschek Institute Artificial Intelligence for the Built World (GNI). It is further supported by the Bavarian Ministry of Economic Affairs, Regional Development and Energy with funds from the Hightech Agenda Bayern.
References
[1] Moloud Abdar, Farhad Pourpanah, Sadiq Hussain, Dana Rezazadegan, Li Liu, Mohammad Ghavamzadeh, Paul Fieguth, Xiaochun Cao. Abbas Khosravi, U Rajendra Acharya, et al. A review of uncertainty quantification in deep learning: Techniques, applications and challenges. Information fusion, (6:243-2977 2021.
[2] David H. Ackley, Geoffrey Hinton, and Terrence J. Sejnowski. A learning algorithm for boltzmann machines. Cognitive Science, (1):147-169, 1985.
[3] Michael Arbel, Liang Zhou, and Arthur Gretton. Generalized energy based models. arXiv preprint arXiv:2003.050033 2020.
[4] Sanghamitra Bandyopadhyay, Ujjwal Maulik, Lawrence B Holder, Diane Cook, and Lise Getoor. Link-based classification. Advanced methods for knowledge discovery from complex data, pages 189-207, 2005.
[5] William H. Beluch, Tim Genewein, Andreas Numbeerger, and Jan M. Kohler. The power of ensembles for active learning in image classification. In 2018 JEEF/CVE Conference on Computer Vision and Pattern Recognition, pages 9368-9377, 2018
10

---

[6] Charles Blundell, Julien Cornebise, Koray Kavukeuoglu, and Daan Wierstra. Weight uncertainty neural networks, 2015.
[7] Glenn W Brier. Verification of forecasts expressed in terms of probability. Monthly weather review, 78(1):1-3, 1950.
[8] Shaked Brody, Uri Alon, and Eran Yahav. How attentive are graph attention networks? arXiv preprint arXiv:2105. /4491, 2021.
[9] Ben Chamberlain, James Rowbottom, Maria Gorinova, Michael Bronstein, Stefan Webb, and Emanuele Rossi. Grand: Graph neural diffusion. In International Conference Machine Learning, pages 1407-1418. PMLR, 2021.
[10] Bertrand Charpentier, Oliver Borchert, Daniel Zügner, Simon Geisler, and Stephan Günnemann. Natural posterior network: Deep bayesian uncertainty for exponential family distributions. arXiv preprint arXiv:2 2021.
[11] Bertrand Charpentier, Daniel Zügner, and Stephan Günnemann. Posterior network: Uncer- tainty estimation without ood samples via density-based pseudo-counts. Advances in neural information processing systems, 33:1356-1367, 2020.
[12] Fan RK Chung. Spectral gruph theory, volume 92. American Mathematical Soc., 1997.
[13] Stefan Depeweg, José Miguel Hernández-Lobato, Finale Doshi-Velez, and Steffen Udluft. Learning and policy search in stochastic dynamical systems with bayesian neural networks. arXiv preprint
2016.
[14] Yilun Du and Igor Mordatch. Implicit generation and modeling with energy based models. In H. Wallach, H. Larochelle, A. Beygelzimer, F. F d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
[15] Michael Dusenberry, Ghassen Jerfel, Yeming Wen, Yian Ma, Jasper Snoek, Katherine Heller, Balaji Lakshminarayanan, and Dustin Tran. Efficient and scalable bayesian neural nets with rank-1 factors. In International conference on machine learning, pages 2782-2792. PMLR, 2020.
[16] Sven Elflein, Bertrand Charpentier, Daniel Zügner. and Stephan Günnemann. On out-of- distribution detection with energy-based models, 2021.
[17] Derek Everett, Andre T Nguyen, Luke E Richards, and Edward Raff. Improving out- of-distribution detection via epistemic uncertainty adversarial training. arXiv preprint arXiv:2209.03/44 148, 2022.
[18] Sebastian Farquhar, Lewis Smith, and Yarin Gal. Liberty or depth: Deep bayesian neural nets do need complex weight posterior approximations. Advances in Neural Information Processing Systems, 33:4346-4357, 2020.
[19] Matthias Fey and Jan E. Lenssen. Fast graph representation learning with PyTorch Geometric. In ICLR Workshop on Representation Learning Graphs and Manifolds, 2019,
[20] Stanislav Fort, Jie Ren, and Balaji Lakshminarayanan. Exploring the limits of out-of-distriburi detection. Advances in Neural Information Processing Systems, 34:7068-7081, 2021.
[21] Dominik Fuchsgruber, Tom Wollschläger, Bertrand Charpentier, Antonio Oroz, and Stephan Günnemann. Uncertainty for active learning on graphs, 2024.
[22] Yarin Gal and Zoubin Ghahramani. Dropout bayesian approximation: Representing model uncertainty in deep learning, 2016.
[23] Johannes Gasteiger, Stefan Weißenberger, and Stephan Günnemann. Diffusion improves graph learning, Advances in neural information processing systems, 32, 2019.

---

[24] Jakob Gawlikowski, Cedrique Rovile Njieutcheu Tassi, Mohsin Ali, Jongseok Lee, Matthias Humt, Jianxiang Feng, Anna Kruspe, Rudolph Triebel, Peter Jung, Ribana Roscher, al. A survey of uncertainty in deep neural networks. Artificial Intelligence Review, 56(Suppl 1):1513-158,, 2023.
[25] C Lee Giles, Kurt D Bollacker, and Steve Lawrence. Citeseer: An automatic citation indexing system. In Proceedings of the third ACM conference on Digital libraries, pages 89-98, 1998.
[26] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks. In Doina Precup and Yee Whye Tch, editors, Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 1321-1330. PMLR, 06-11 Aug 2017.
[27] Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in neural information processing systems, 30, 2017.
[28] Matthias Hein, Maksym Andriushchenko. and Julian Bitterwolf. Why relu networks yield high-confidence predictions far away from the training data and how to mitigate the problem. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages
41-50, 2019.
[29] Eyke Hüllermeier and Willem Waegeman. Aleatoric and epistemic uncertainty in machine learning: An introduction to concepts and methods. Machine learning. 110(3):457-50,, 2021.
[30] Ahmet Iscen, Giorgos Tolias, Yannis Avrithis, and Ondrej Chum. Label propagation for deep semi-superrised learning. In Proceedings of the IEEE/CVF conference on computer vision and pattem recognition, pages 5070-5079, 2019.
[31] Ajay Joshi, Fatih Murat Porikli, and Nikolaos Papanikolopoulos. Multi-class active learning for image classification. 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 2379-2379, 2009.
[32] Mira Jürgens, Nis Meinert, Viktor Bengs, Eyke Hüllermeier, and Willem Waegeman. Is epistemic uncertainty faithfully represented by evidential deep learning methods?. 2024.
[33] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
[34] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv: S0kiv:66090099000 2016.
[35] Mare Lafon, Clément Rambour, and Nicolas Thome. Energy correction model in the feature space for tittof-diistibutton detection. arXiv preprint arXiv:2403. 104000033.000,, 2024.
[36] Mare Lafon, Elias Ramzi, Clément Rambour, and Nicolas Thome. Hybrid energy based model in the feature space for out-of-distribution detection, 2023.
[37] Balaji Lakshminarayanan Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles, 2017.
[38] Yann LeCun, Sumit Chopra, Raia Hadsell, M Ranzato, and Fujie Huang. A tutorial on energy- based learning. Predicting structured data, 1(0), 2006.
[39] Jeremiah Liu, 7i Lin, Shreyas Padhy, Dustin Tran, Tania Bedrax Weiss, and Balaji I Lakshmi- narayanan. Simple and principled uncertainty estimation with deterministic deep learning via distance awareness. Advances in neural information processing systems, 33:7498-7511 2020.
[40] Weitang Liu, Xiaoyun Wang, John D. Owens, and Yixuan Li. Energy-based distri-dittiitttinn detection, 2021.
[41] Zhao- Yang Liu, Shao- Yuan Li, Songean Chen, Yao Hu, and Sheng-Jun Huang. Uncertainty aware graph gaussian process for semi-supervised learning. In Proceedings of the AAAI Conference Artificial Intelligence, volume 34, pages 4957-4964, 2020.
12

---

[42] Owen Lockwood and Mei Si. A review of uncertainty for deep reinforcement learning. In Pro- ceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment. volume 18, pages 155-162, 2022.
[43] Andrey Malinin and Mark Gales. Predictive uncertainty estimation via prior networks. In
S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018.
[44] Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton Van Den Hengel. Image- based recommendations styles and substitutes. In Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval, pages 43-52, 2015.
[45] Pablo Morales-Alvarez. Daniel Hernández-Lobato Rafael Molina, and José Miguel Heránnde-- Lobato. Activation-level uncertainty deep neural networks. International Conference on Learning Representations, 2020.
[46] Jishnu Mukhoti, Andreas Kirsch, Joost van Amersfoort, Philip HS Torr, and Yarin Gal. Deep deterministic uncertainty: A simple baseline. arXiv preprint arXiv:2102. 11582, 2021.
[47] Jishnu Mukhoti, Andreas Kirsch, Joost van Amersfoort, Philip H.S. Torr, and Yarin Gal. Deep deterministic uncertainty: A new simple baseline. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 24384-24394, June 2023.
[48] Jishnu Mukhoti, Joost van Amersfoort, Philip HS Torr, and Yarin Gal. Deep deterministic uncertainty for semantic segmentation. arXiv preprint arXiv:2111.00079 2021.
[49] Sai Munikoti, Deepesh Agarwal, Laya Das, and Balasubramaniam Natarajan. general framework for quantifying aleatoric and epistemic uncertainty in graph neural networks. Neuro- computing, 521:1-10, 2023.
[50] Mahdi Pakdaman Nacini, Gregory Cooper, and Milos Hauskrecht. Obtaining well calibrated probabilities using bayesian binning. In Proceedings of the AAAI conference on artificial intelligence, volume 29, 2015.
[51] Galileo Namata, Ben London, Lise Getoor, Bert Huang, and U Edu. Query-driven active surveying for collective classification. In 10th international workshop on mining and learning with gruphs, volume 8, page 1, 2012.
[52] Yin Cheng Ng, Nicolò Colombo, and Ricardo Silva. Bayesian semi-supervised learning with graph gaussian processes. Advances in Neural Information Processing Systems, 31, 2018.
[53] Yin Cheng Ng. Nicolò Colombo, and Ricardo Silva. Bayesian sumi-supervised learning with graph gaussian processes. Advances in Neural Information Processing Systems, 31, 2018.
[54] Vu-Linh Nguyen, Sébastien Destercke, and Eyke Hüllermeier. Epistemic uncertainty sampling. In Discovery Science: 22nd International Conference, DS 2019, Split, Croatia, October 28-30, 2019, Proceedings 22, pages 72-86. Springer, 2019.
[55] CAZHAOW SALIH OAZA. Bayesian error bars for regression. 1996.
[56) Lawrence Page, Sergey Brin, Rajeev Motwani, Terry Winograd, et al. The pagerank citation ranking: Bringing order to the web. 1999.
[57] John Palowitch, Anton Tsitsulin, Brandon Mayer, and Bryan Perozzi. Graphworld: Fake graphs bring real insights for gnns. In Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining, pages 3691-3701, 2022.
[58] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary De Vito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. In NIPS-WS W, 2017.
[59] Yu Rong, Wenbing Huang, Tingyang Xu, and Junzhou Huang. Dropedge: Towards deep graph convolutional networks on node classification. arXiv preprint 1rXiv:1907.10903 2019.
13

---

[60] Adrian Schwaiger, Poulami Sinhamahapatra, Jens Gansloser, and Karsten Roscher. Is uncer tainty quantification in deep learning sufficient for out-of-distribun detection? Aisafery@ ijeai, 54, 2020.
[61] Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi- Rad. Collective classification in network data. Al magazine, 29:93-93, 2008.
[62] Murat Sensoy, Lance Kaplan, and Melih Kandemir. Evidential deep learning to quantify classification uncertainty. Advances in neural information processing systems, 31, 2018.
[63] Oleksandr Shchur, Maximilian Mumme, Aleksandar Bojchevski, and Stephan Günnemann. Pitfalls of graph neural network evaluation. arXiv preprint 1811 1.05868, 2018.
[64] Maximilian Stadler, Bertrand Charpentier, Simon Geisler, Daniel Zügner, and Stephan Gün- nemann. Graph posterior network: Bayesian predictive uncertainty for node classification. Advances in Neural Information Processing Systems, 14:18033-18000 2021.
[65] Yiyou Sun, Yifei Ming, Xiaojin Zhu, and Yixuan Li. duitt--dist detection with deep nearest neighbors. In International Conference on Machine Learning, pages 20827-20840.
PMLR, 2022.
[66] Jayaraman Thiagarajan, Rushil Anirudh, Vivek Sivaraman Narayanaswamy, and Timo Bre- mer. Single model uncertainty estimation via stochastic data centering, Advances in Neural Information Processing Systems, 35:8662-BB 15:8662-8674, 2022.
[67] Puja Trivedi, Mark Heimann, Rushil Anirudh, Danai Koutra, and Jayaraman J Thiagarajan. Accurate and scalable estimation of epistemic uncertainty for graph neural networks. arXiv preprint arXiv:2401.0333 2024.
[68] Dennis Thomas Ulmer. survey on evidential deep learning for single-pass uncertainty estimation. 2021.
[69] Joost Van Amersfoort, Lewis Smith, Andrew Jesson, Oscar Key, and Yarin Gal. On fea- ture collapse and deep kernel learning for single forward pass uncertainty. arXiv preprint arX1v:2/02.11409, 2021.
[70] Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, Yoshua Bengio, al. Graph attention networks. stat, 1050(20): 2017.
[71] Fangxin Wang, Yuqing Liu, Kay Liu, Yibo Wang, Sourav Medya, and Philip S Yu. Uncertainty in graph neural networks: A survey, arXiv preprint arXiv:2403.071885, 2024.
[72] Guotai Wang, Wenqi Li, Michael Aertsen, Jan Deprest, Sébastien Ourselin, and Tom Vercauteren. Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks. Neurocomputing, 338:34 2019.
[73] Guotai Wang, Wenqi Li, Michael Aertsen, Jan Deprest, Sebastien Ourselin, and Tom Vercauteren. Test-time augmentation with uncertainty estimation for deep learning-basse medical image segmentation. 2022.
[74] Min Wang, Hao Yang, Jincai Huang, and Qing Cheng. Moderate message passing improves cal- ibration: universal way to mitigate confidence bias in graph neural networks. Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 21681-21688 2024.
[75] Xiao Wang, Hongrui Liu, Chuan Shi, and Cheng Yang. Be confident! towards trustworthy graph neural networks via confidence calibration, 2022.
[76] Max Welling and Yee Whye Teh. Bayesian learning via stochastic gradient langevin dynamics. In Proceedings of the 28th International Conference on International Conference on Machine Learning, ICML'11, page 681-688, Madison, WI, USA, 2011, Omnipress.
[77] Yeming Wen, Dustin Tran, and Jimmy Ba. Batchensemble: an alternative approach to efficient ensemble and lifelong learning, arXiv preprint arXiv:2002.06715, 2020.
14

---

[78] Thomas Wolf. Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue. Anthony
Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. Huggingface's 's transform- ers: State-of-the-art natural language processing. arXiv preprini arXiv: 17Xiv:1910.03771,,
[79] Qitian Wu, Yiting Chen, Chenxiao Yang, and Junchi Yan. Energy-based out-of-distribution detection for graph neural networks. arXiv preprint arXiv:2302.02911 2023.
[80] Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural 2018.
networks? arXiv preprint
[81] Shuangfei Zhai, Yu Cheng, Weining Lu, and Zhongfei Zhang. Deep structured energy based models for anomaly detection. In Maria Florina Balcan and Kilian Q. Weinberger, editors, Proceedings of The 33rd International Conference on Machine Learning, volume 48 of Pro- ceedings of Machine Learning Research, pages 1100-110,, New York, New York, USA, 20-22 Jun 2016. PMLR.
[82] Xujiang Zhao, Feng Chen, Shu Hu, and Jin-Hee Cho. Uncertainty aware semi-supervised learning graph data. Advances in Neural Information Processing Systems, 13:12827-12836, 2020.
[83] Yin-Cong Zhi, Yin Cheng Ng. and Xiaowen Dong. Gaussian processes on graphs via spectral kernel learning. IEEE Transactions Signal and Information Processing over Networks, 2023.
[84] Kaixiong Zhou, Xiao Huang, Daochen Zha, Rui Chen, Li Li, Soo-Hyun Choi, and Xia Hu. Dirichlet energy constrained learning for deep graph neural networks. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances Neural Information Processing Systems, volume 34, pages 21834-21844 Curran Associates, Inc., 2021.
15

---

A Proofs
A.1 EBMs for i..d. Data
We first consider proofs for formal statements regarding logit-based EBMs in general (i.c. for i.id. data).
Proposition 4.1. Let fo R2 + R. be a piecewise affine function and lR U² Qi be the disjoint set of polytopes on which So is affine, ie. fo(=) w + b for x E Q1. Assuming the direction of the rows of each W to be uniformly distributed, the probability that Zo converges decreases exponentially in the number of non-closed linear regions L' and classes C.
Pr PriZa < o0] (1/2)C-1/
Proof. Consider an arbitrary direction a E IRd such that ||ж||| = 1. As shown in 28 (Lemma 3.1). there exists some & and og such that for all a Og holds that oz E Qt. We have fo fa())) W b(), and therefore:
Eo v) W az + b
We now distinguish between two cases as a -> oo:
If w (1) y 2 0, we have:
lim a W7 x = 00
a0000
lim a b(9) 0C
00000
lim Es( y) -C
And conversely if W.
(1) x - O, we have:
lim (a y) = 00
00000
Since we assumed the direction of W to be uniformly distributed, both cases occur with probability 0.5. Now consider the marginal energy assigned to oa::
Eo (o(x) = log Σ exp (- (oEa, v))
I min { Ex v1)
We now analyze when the marginal energy Fig (ax)) diverges toward xo and -o respectively. If for any {I,..., C} we have that w(()) x < 0. then:
lim Eg (ax) < lim min (E)
a-000
Since each wi x > 0 with probability 1/2, the limit diverges toward -o0, with probability at most (1/2)C.
Now consider open set Q with non-zero measure for over which the classifier fo is linear. With probability least 1/2, for e Q0) fo diverges toward cc.
16

---

We now can consider the integral over the unnormalized density implied by fo:
/ (xp(-EE((x))
Zo =
Ra
Σ
/ Qm)
exp(-Fa((a))
For any open region Q the energy will diverge toward -oo with probability (1/2) and therefore:
/ exp(-Es(z))dzz 00 Qm)
For Zo to be finite and well-defined, the integral over all I' open linear regions with non-zero measures need to converge simultaneously, each of which happens independently with probability (1/2)C. Therefore:
Pr [Zo <o0 (1/2)
Remark. Note that the assumption that the rows of each ( have uniformly distributed direction given when initializing the model from normal distribution. When training a classifier with the commonly used cross-entropy objective, the training incentivizes the model to assign high logits to exactly one of the classes for each datapoint. Therefore, the model is intuitively encouraged to have at least one row of (!) for which the æ that are in the corresponding region Q(1) have positive projection / > O. In practice, expect that for a trained model the probability of any a to have a negative projection onto all rows of the corresponding (!() to be lower than (1/2)C.
Theorem 4.2. For a piecewise affine classifier fe as in Proposition 4.1 po is well-defined.
Zo , Σ
/ (xp(-EEo(z
pyy " 00
Proof. We consider the corrected joint energy Eo (oax, y) for some sufficiently large a og and an arbitrary direction Rd
Eo y) = Eo(ox, y) 7 log Wo ((o(oaa)) My En)
7 fo((x) + 2 (fo(ax)y My) " (fo (yy + const
(( az (i)) + I (w ax + 10 m,y" " (W + b()) pby) + const
For large enough ag, the terms quadratic in aæ will dominate linear terms and we can write for some e >>
(g( 2 c* (W()) oz) T Σ (W ox)
a² <ee (1) x) Σ (Vew 2 "
Here, we pick Σ, such that the equation holds for all (l,...,C Note that this means that in the limit, each joint energy term bounded by the logarithm of the same Gaussian log E.). For the marginal energy it holds:
17

---

Eo(ax) -log] Σ exp (-e((ax; w)
- log Σ exp (-axx D 2 oa))
ax' T Σ ax log C
2 az Σ ax
Here, we absorb the constant log C into the quadratic term. Similar to the proof of Proposition 4.1 analyze the integral of the implied density for each linear region of the classifier Q
1 exp (-Eo(z)) dx =
/
exp (-Eo dx +
exp (-Eo Ex(()) dæ
Q10
Q ||0||2000
The first term integral of finite function finite region, which therefore will also be finite:
1 ag
= 1//o
exp (-Eo(z)) do 00
Q
2500
For the second term, notice that it is bounded by the (finite) Gaussian integral:
(
12000
exp
(-Eo(ze)) dx
Qai
||0| 2200
exp
(-x"'''-- 2)
dxc
Q
||
00
Lastly, since decompose Rd into finite of linear regions each of which the integral finite, the entire integral is finite.
f exp (-Eo(z)) dx = Σ
f exp (-Eo(x)) dx Qu:
1=1
(1)
+ (1)
asan
+Tazap)
1=1
< 00
Theorem A.1.1. For a piecewise affine classifier fa as in Proposition 4.1 and a piecewise affine feature extractor go(x), Po is integrable.
1 { exxp(=Eo(x, ()) My, "ay < 00
20
Proof. Note that both affine functions fo and go partition the input space Rd into finite set of linear regions. Intersecting their boundaries again gives rise finite set of linear regions. For bounded
18

---

regions, the integral over the induced density will be finite, so we discuss the unbounded regions here. For any such region Q(), we have fe( = w æ + by and ga(x) W + bg.
We now partition the Rd n Q into four regions using the kernel spaces of wy and W respectively, L.e. I& ((@") f+g w g 10 f where define 0f+g = ker( ker(W,) Q f--g = ker( ker( ( 9) n Q 0g-s ker ())) \ ker(wy Q and I f+g im(wr)) im (m(W)) Q())
We now decompose the integral over Q(1) into integrals over all four regions. Note that of these sets, only I f+g has a non-zero measure (if the affine classifier is not the null function). Therefore, we only have to focus on this domain. For some ag and I ||0||2
ag. we have:
po( & exp (-Eo(z.y))
My: Eg) a exp (w) x + b,) exp (-( + by) Mg (W + by)) exp (-x" -
For some Σ, that bounds the quadratic function in the exponential for ||0||| 2 ag. The integral over the region Q(1) now reduces to:
10
const * exp (-Eo(x;, (() Myy Ey)
#EQU
const
+
const * exp (-E Ex(x, (N)x,,)))( (g0(x) Myy da
Q!1
|| 200
const * exp (-x" (-x" Σn dx
const
+
Q
|| 1ell2200
const cxp E-x"'E11 dæc
const
Q
||0 1
00
Here, we pick E. to bound the quadratic terms of all Eg.. As previously mentioned, the kernel spaces of w and W, y have zero measure and therefore do not contribute to the integral. As previously, the integral over the entire domain IR decomposes into finite integrals over all Q().
L
Σ const exp (-Ee(a, ((go(() My Ey)dx = const +
Σ (1)
Corollary 4.3. For a piecewise affine classifier fo as in Proposition 4.1 and any E Rd almost surely:
lim (o(xx) = 0 ao
Proof. As per the proof of Theorem 4.2 for sufficiently large a we have for some Σ,²:
Ea(ax) 2 x: x''E''
19

---

From there, it follows directly that:
lim Po (a(ax)
lim exp( - Eg (ax))
a a0000
a-200
=exp
lim Eg((aa)) aa-20
S exp
lim -x" " 1
0
A.2 Graph-based Energies at Different Scales
Here, show that local energy (Equation (5)) and group energy (Equation (6) induce valid probability density when using linear diffusion operator PA and regularized energy according to Equation (8). Note that independent energy (Equation (4) is shown induce well-defined probability density as it is just graph-independent regularized energy.
Proposition A.2.1. For a linear diffusion operator PA(x) az + const, a > 0 and the regularized energy Eg(x, y). the local energy (x(x) induces well-defined density:
Exp(-EB,L 00
Proof.
Be.((x) - log Σ exp PA
(-Bo(x; v)
log Σ exp
(-aBBo(e, v) + const)
log Σ exp (-oxx
Mーー - +
ax Σ x + const log C
2 T Σ x
Again, we absorbed the constant terms as well as a into the quadratic term for large enough ||0||2 and used the quadratic bound from Theorem 4.2 From here, it is straightforward that:
J exp (-Ba,((a)) dx < const + / exp (-a" x) dx < oo
Propasition A.2.2. For a linear diffustion operator PA(x) ax + const, > and the regularized energy Eg( y), the group energy (x( induces a well-defined density:
f exp(- - (x xc
Proof.
Fa,a(x) PA (Fo(z))
-FFo (x) + const
> ax シー const
2 x E
20

---

Again, we have used the quadratic bound of Theorem 4.2 and absorbed a: > 0 for large enough | From there, it follows:
J exp (-Fo,o(xo) de const / exp (-x=='- 2) do x
Remark. We want to point out that most diffusion operations, especially the ones discussed in Oopeendix(.6jjnccludnee the propp-propngation smoothing used in our experiments, are of the form Pa(x) oz + const with a > 0. Since we only integrate over the features of a single node and keep all other features fixed, of these diffusion processes can expressed as weighted sum over nodes in the graph. Since we consider unweighted graphs, there are no negative edge weights which ensures that a > (as long as some sort of self-loop included). Therefore, linear diffusion processes can be seen positive affine transformations.
Theorem 4.4. For a linear diffusion operator PA(x) = aæ + const, a > 0 and the regularized energy Eo( v). GEBM induces a well-defined density:
J exp (-Bo,GERM (2:) doc < 00
Proof. We previously established quadratie bounds all three constituents of the aggregate energy for large enough ||/||2:
r 2 x T -
Ba,1(x) 2 x. Σ x
Ee,c(x) 2 x >x''a E
Hence, there must be one Σ E (27 Σ Eg' that bounds all three energy types. We now look at the GEBM model:
(0) log (exp (E),(()) + exp (Eo_)(a)) exp (Eo.o(z))
2 log (3 exp (x. Σ Σ.'x, x))
x: Σ x log
x Σー' x
As before, this bounds the density induced by GEBM:
/
{ exp (-Eo,GEBM (z)) dx I const + / exp (-pp-"''', x) dx <0
Remark. This directly imposes a restriction on which energies can be included GEBM beyond the three naturally arising graph-specific terms propose: As long as the energy by an energy that grows fast enough to ensure convergence, our framework accommodates it.
be bounded
We also explicitly write out the (unnormalized) GEBM density (2)) in terms of its constituents (x)) po. and (z)
(a.GEBM(x)) x Po.r (x: + (x) -1 (x)
21

---

B Experimental Setup
B.1 Datasets and Distribution Shifts
#Edges m
#Classes
Feature Demsity
Homoohily (%)
Edge Density my/n" (%)
Let-out-
#Nodes-Idd.
(Lex) fiu4
CorMM
2965
16316
2870
1.75
78.9
0.18
1650
Co LLM
2005
163116
364
7
n1a.
78.9
0.18
1650
Chessec
$230
10074
002
6
0.75
91.9
0.06
2977
PubMed
19717
85648
500
80.2
0.02
11812
Amazon Photo
233102
745
34.74
52.7
4555
Amazon Camputers
707
34.84
Y1.7
0.26
Counthor
163785
6805
0.88
80.8
0.05
Counthor Physics
36453
493824
8415
0.39
50.1
0.04
2X221
Table 5: Statistics about the datasets used in this work.
We use eight datasets in this work that we expose to similar kinds of distribution shifts. For CoraMI., we also create a version that uses continuous word embeddings instead of categorical bag-of-word features. We use the all all-MiniLM--L--22 sentence transformer provided by Hugging Face [78] to embed the abstracts that are provided for each paper (node) in the citation network. All datasets are taken from PyTorch Geometric [9]. They are licensed as C.C.O 1.0 (CoraML, CoraML LLM), C.C. -ww.ooooooiiaalaaaaaa..... Share Alike 3.0 (Citeseer), OdbL. 1.0 (PubMed).
We expose each dataset to the same distribution shifts which can be categorized into three families:
Leave-out-Classes. We pre-select subset of classes and designate them out-of-distribution. That is, we remove them from the training set and train the GNN on the remaining set of nodes. We focus on an inductive setting, where we also remove all edges linking o.o.d. left-out-class nodes to the training graph, thus preventing information leakage. In the transductive setting, o.o.d. nodes still contribute to the training signal as they may be connected to i.d. nodes. This distribution shift can be seen as occurring that are likely to cluster together. We distinguish between two kinds of selection processes for the classes to be left out: We either pick the last classes in the dataset (LoC (last)) or choose classes with the most heterophilic connection pattern, which distinguishes them even further from i.d. classes (LoC (hetero.)).
cluster level, as we introduce anomalous nodes
(ii) Feature Perturbations. We select 50% of nodes at random be o.o.d. and perturb their features by replacing them with random noise. We distinguish between three perturbation types that control the similarity of node features to training data: We generate features that are similar to i.d. data (near-o.o.d. in the following way: For each of the d features, we compute how frequent it occurs in the dataset. We then proceed sample each of these features independently with the corresponding success probability á from a Bernoulli Ber(ý). Instead, we can also set the success probability to p = 0.5 for each of the features and induce more severe distribution shift within the bag -fodddss domain of the dataset. Lastly, when sampling features according to normal distribution N (0, 1), we generate out-of-domain data that should, theory, easy to detect far-o.o.d. For the CoraML-LM and the PubMed datasets, features not bag-of-wort word. Therefore, the near-o.o.d shift is omitted and the far-o.o.d. shift needs to be considered within the domain of the data.
(iii) Structural. We induce structure-related shifts two ways: The first option we consider is to rank all nodes according to their local homophily. That is, we compute the ratio of neighbors with the same class: h4 = (w, E N₁ y/ = y, H/W We then designate 50% of nodes with the highest heterophily (i.e. lowest homophily) as o.o.d (homophily). The second structural shift ranks nodes according to their (approximate) Page Rank centrality and declares nodes with low values o.o.d. (Page Rank).
In the inductive setting, we remove the o.o.d. nodes from the training graph and re-introduce them during inference. We evaluate o.o.d. detection metrics on a validation/test set that includes both i.d. and o.o.d. nodes. While the training and validation set are randomized for each split, the test set shared across all splits to prevent data leakage. All results are reported over five different splits and five independent model weight initializations for each of them. Where appropriate, we also report standard deviations in Appendix C
22

---

Remarks regarding Page Rank Shifts. We study the centrality- -bsedd shift (which heavily correlates with a degree-based shift) for completeness reasons as a similar generation process is used in [79]. However, to which extent methods should be expected to assign high uncertainty to low-degree nodes is questionable: First, centrality and/or degree are feature-irrespective quantities. In contrast to homophily (nodes of similar classes have similar features), the confidence of a classifier may not (and should not) depend on the node degree/centrality. Furthermore, many baselines use symmetric normalization (Appendix C.6) that inherently favors high-degree nodes. Therefore, regarding node degree/centrality as a distribution shift will not test the quality of an uncertainty measure but instead favor all models that appropriate diffusion processes for most baselines. Regardless of our concerns regarding a centrality- bbaeed shift, we report results regarding that shift as well in Appendix [
evaluation
Note that while GEBM is not among the best-performing estimators in this setting, it still achieves the highest average o.o.d. detection rank overall, indicating that its merits in all other settings heavily outweigh its performance on the Page Rank shift.
B.2 Models and Training
At the backbone of all models, we use the same GCN 34 architecture not specified explicitly otherwise. We use one hidden layer of dimension 64, symmetric normalization Appendix [C.6 and add self-loops to the undirected (symmetric) adjacency matrix. We use ReLU nonlinearities, enable the bias term, and use dropout at p = 0.5.
All models are trained with the ADAM optimizer [33 with a learning rate of 10 10-3, weight decay of 10 and cross- cooosyy objective. We use early stopping on the validation loss with patience of 50, an absolute improvement threshold of 10 and select the model with the best validation loss. We implement
models in Py PyTorch (58) and Py PyTorch Geometric [19 and train two types of machines: (i) Xeon E5-2633 v4 CPU @ 2.20GHz with a NVIDA GTX 1080TI GPU and 128 GB of RAM. (ii) AMD EPYC 7543 CPU @ 2.80GHz with NVIDA A100 GPU and 128 GB of RAM.
As the GCN backbone used our experiments is lightweight, model training finishes within a few minutes and VRAM consumption is dominated by the datasets. Our post hoc method can be fitted and evaluated in negligible time (<1s).
For MC-Dropout [22), we use a dropout probability of = 0.5 which we also use for DropEdge [59]. At inference, we evaluate 50 samples to compute uncertainty. For ensembles, we train 10 backbones from different weight initializations independently. The Bayesian GNN 6 15 18 also is evaluated with 50 samples during inference and uses a KL-loss with weight 10-1 and trained with learning rate 10- and no weight-decay. We parametrize the weight distribution using log transform and initialize the mean and log scale to 1.0 and 33.0 respectively. Weight distributions are regularized to follow standard normal.
For GPN (64) follow the hyperparameters suggested by the authors and warmup training on the normalizing flow for 5 epochs with learning rate of le - 2. no weight decay during joint training and 1e 2 during warmup. The flow dimension is 16 and is composed of 10 radial layers. The cross-entropp regularization weight is 10-4, We use Page Rank propagation for 10 iterations at a teleport probability of 0.1.
For SGCN [2], we use the same GCN backbone and a GDK -piiorr at cutoff distance 10 and scale 1.0. For teacher training, we a learning rate of 10 weight decay of 5 10 and KL-loss weight of 10-
For HEAT (36], we use the hyperparameters suggested by the authors. In contrast to their study on the image domain, we do not have features with spatial extent and can not use standard deviation pooling on the volume. We set the temperature parameter of the combined HEAT model to -1, following the suggestion of the authors. Similar to their EBM backbone, we imitate the structure of the classifier and use 2-layer MLP with hidden dimension of 64 akin to the GCN backbone.
Our GEBM framework regularizes logit- joint energy at a strength 7. For downstream tasks. this parameter can, in general, be tuned. We find that an equal weighting of predictive energy and regularization performs well: We choose 7 such that the 95% quantiles of both the training logits and the representations on which the Gaussian density model is fit are in the same range. Furthermore, recent work on deterministie uncertainty argues that density-based -hased epistemic uncertainty should not be estimated directly from the logits [46). We follow this advice and do not fit and evaluate the
23

---

regularizer on the output layer (logits) of the GNN but instead on its penultimate layer representations. Note that for the ReLU networks we use, all formal proofs still hold in this setting as the penultimate
representation of node v is also described by piecewise affine function 28 (see A.I.1 As
smoothing operator PA. we employ prop-aprongttton smoothing with a = 0.5 for = 10 iterations. Lastly, we found that aggregating the energy terms at different scales (Equation operation instead of logsumexp to perform better in practice and use it for our experiments.
using sum
In our ablation regarding different model backbones, we use the default hyperparameters from the GCN backbone (e.g. number of hidden layers). For each model additional hyperparameters are set to the following values: For GATv2 [70 8 we use 8 heads and use summation to aggregate them. For SAGE [27, we use no normalization at the layers.
B.3
Uncertainty Evaluation
For sampling-base approaches, we compute aleatoric and epistemic uncertainty as entropy and mutual information respectively according to (24):
u
E D)
H pp x)]]
TM MI [8. xx D)
H p)
[p(y x)]] B)
H [p(y x]]]
For evidential approaches, we follow 664 and use the maximum softmax response max, Ply = c x) as aleatoric uncertainty and the total evidence Σ. Og as an epistemic estimate. Since we want to compare single measures of epistemic uncertainty, presence of network effects. For deterministic models (that are at the backbone of EBM-based approaches), we use the predictive entropy as measure of aleatoric uncertainty IH [p(v æ)] and the energy (x) as a measure of epistemic uncertainty. For all EBMs, we use a temperature of 1.0.
evaluate GPN's epistemic uncertainty in the
C
Additional Results
C.1 Out-of-Distribution Detection
We report AUC-ROC and AUC-PR metrics for the o.o.d. detection problem in an inductive and transductive setting with corresponding standard deviations over all five splits and five model ini- tializations each in Tables 6 10 and 12] Again,
report the average performance ranks are in Tables 9 11 and 13 As mentioned in Section 5.2 we consider a weighted average that does not favor any distribution shift family. That is, we assign weight of 1/6 structural and leave-out-class settings and factor in feature perturbations at weight of 1/9 each.
In inductive setting, GEBM outperforms all other basclines and achieves the best rank regarding both AUC-ROC and AUC-PR metrics. In the transductive setting, the two evidential methods GPN and SGCN outperform GEBM two datasets: We argue that this is due to feature leakage o.o.d. data is present during training and evidential models are explicitly encouraged to assign low evidence to anomalous nodes. We want to point out that it is unrealistic to assume that o.o.d. data is available in practice and therefore strongly argue in favor of the inductive scenario as more realistic benchmark. Nonetheless, GEBM is highly effective for transductive problems as well.
C.2 Improvement over Second Best Method
We evaluate the improvement in AUC-ROC scores of GEBM over the estimator assigned the overall second-best rank. To that end, average the model ranks not individually for each estimator and dataset, but instead compute an average over datasets and splits simultaneously. This way, we obtain global rank (over datasets and shifts) for each epistemic estimate. Again, to not favor certain classes of shifts, we assign weights such that each distribution shift family has an equal contribution. Based on AUC-ROC scores listed in Table we list the rank of each model in Table 14
While our approach, GEBM, also ranks the highest globally, a vanilla logit-based EBM is the next best approach. Therefore, compute the improvement in AUC-ROC scores over this model achieved by
24

---

MILD
ABCaOCT
ARC-OCT
ADC-O0C
AUC
(03600
Tub
AT
DICLSICD
DONAMCODD
CS
AT
D03600
3..
Tu 510m
52. 1i0m
Table 6: Out-of- Diito----itiiitiionnn detection AUC-ROC (1) using aleatoric or epistemic uncertainty (best and runner-up) in an inductive setting.
GEBM in Table [5] On all datasets, GEBM improves the AUC-ROC scores by 5.8 to 10.9 percentage points on average (16%-32% relative improvement) over the second-bes, ranked estimator.
C.3 Misclassification Detection
We report AUC-ROC and AUC-PR for misclassification detection in both an inductive and trans- ductive setting with the corresponding standard deviations in Tables 16 to 19 In alignment with previous work [64], we observe that often aleatoric uncertainty is more suitable for misclassification detection than epistemic uncertainty. Overall, GEBM performs similar to other epistemic measures. We again want to point out that the scope of this work does not encompass improvements on aleatoric uncertainty. The results on misclassification detection obtained in our work and previous studies indicate that this problem may be more suitable as benchmark for aleatoric uncertainty.
C.4 Calibration
Our framework, GEBM, is limited toward epistemic uncertainty estimation and leaves the aleatoric estimates and the classifier calibration unchanged. For completeness, we nonetheless report calibration
25

---

Model
CoraML
CoraML
Citeseer
PubMed
Amazon
Amazon
Coauthor
Couuthor
LLM
Computers
Photo
CS
Physies
GCN-MCD
10.7/20.1
10.1/19.9
9.8/19.3
9.2/18.8
10.2/19.6
10.2/19.7
10.0/19.3
10.5/20.4
GCN-DE
6.3/14.2
8.0/169
7.8/16.2
7.3/13.6
7.2/15.9
7.1/15.8
6.5/13.2
8.1/16.8
6.9/14.8
7.2/15.9
6.6/14.7
7.3/13.6
7.7/16.3
7.4/16.2
6.8/13,5
7.8/16.4
GCN-BNN
5.2/11.4
6.0/13.4
5.4/11.0
5.3/10.9
4.3/9.0
5.1/9.8
4.9/10.5
4.8/9.9
GCN-Ens
7.8/16.1
6.3/15.0
6.4/12.9
6.2/12.4
5.7/11.6
4.7/9.0
4.8/10.4
6,2/13.3
GPN
7.0/15.2
5.4/11.2
6.6/12.9
5.8/12.4
5.7/11.8
5.3/11.7
7.2/13.9
6.1/14.5
SGCN
5.2/9.5
6.1/14.6
6.0/12.6
5.0/11.7
9.1/18.5
7.6/15.1
9.3/17.9
4.6/11.2
GCN-EBM
4.5/7.9
4.0/8.2
5.2/10.3
5.2/12.0
4.2/10.7
4.9/10.9
4.1/7.1
4.478.4
GEN-HEAT
4.4/10.4
5.4/12.9
3,8/9.2
5.6/12.8
3.5/8.6
4.5/10.2
4.2/8.3
4.7/10.4
GCNSafe
5.3/9.1
4.4/8.1
5.2/9.5
5.4/8.9
5.9/13.2
6.4/12.9
5.5/10.7
5.8/11.7
GCN-GEBM
2.7
4.9449
3.3/5.7
38/77.4
2.5/4.2
2.7
2.7/4.3
3.0/5.0
detection rank (AUC-ROC) (L) of epistemic uncertainty versus other epistemic measures all uncertainty measures over all distribution shifts an inductive setting (best and runner-up).
Table 7: Average o.o.d.
for the GCN used at the backbone of our experiments. Note that since our model does not change the output of the classifier, the calibration could be further improved using post hoc methods like temperature scaling [26]. This, however, beyond the scope of this work.
Expected Calibration Error (ECE). The expected calibration error (J) (50) measures how well the predicted probabilities that normalized the interval [0, 1] match the true predictive accuracy. To that end, we bin each prediction into B = 20 bins according to their confidence (i.e. maximum softmax response max. pyy c z). For each bin, we then compute the average accuracy and compute the ECE as the mean over all bins weighted by their size.
Bx n
ECE- Σ
|accuracy (B)) confidence(Ri, (B:)]
Brier Score. A similar metric is the Brier score 7 (L) which computes the mean squared distance between the predicted probabilities and the one-hot encoded true labels.
Σ p xx)) vill - n
Brier
We report ECE and Brier scores for inductive and transductive settings Tables 20 to 23 In terms of calibration metrics, none of the considered approaches consistently shows strong merits. We remark that work on GEBM and GNN calibration is somewhat orthogonal and our framework can be applied to any well-calibrated GNN backbone.
C.5 Backbone Architecture
We report AUC-ROC and AUC-PR for o.o.d. detection using GEBM and different GNN backbones with corresponding standard deviations in Tables 24 and 25 GEBM is effective on all models and achieves the highest scores most distribution shifts.
C.6 Bias of Diffusion Operators
At the core of many GNNs and also our GEBM framework lies a diffusion operator PA R + R.. Here, discuss three typical realizations and the bias the introduce.
Symmetric Diffusion. Symmetric diffusion normalizes the adjacency matrix A D 1 AP-DD-11 Here, D diag(deg( -eg(,,)) is diagonal matrix of node degrees. The symmetric diffusion operator has a dominant eigenvector that correlates with the node degree Umax d!/2 and therefore also centrality 12]. Repeated application of this diffusion process to any arbitrary signal will concentrate confidence high degree nodes. Applying this diffusion to confidence measure like the logits of GNN will therefore always favor high degree nodes.
26

---

Sheld
AacT
A0C30001
ABC-3001
ADCROOT
AECROCY
O0360t
505 (78 fm.
44
925
163 2013
(200
Kilter
MUN-GREN
RON
Di No
AMRLE Than
750 1.
DCNERRM
GOM-NEDD
650
Ch
de H5.7
Table 8:
detection AUC-ROC (1) using alcatoric or epistemic uncertainty (best and runner-up) in transductive setting.
Random- Walk Diffussin. Gand diffusion normalizes the adjacency matrix as -= D which can be seen as a stochastic transition matrix of random walk on the graph. A different interpretation is that each node distributes its information uniformly to its neighbours which in turn sum incoming signals. It, too, has a dominant eigenvector that correlates with the node degree: Umax X d [12] and also favors high degree nodes when applied to a confidence measure.
Proel-Propagation Diffusion. Diffusion used by Label Propagation 300 normalizes the adjacency matrix as Á = AD-1 and can be seen as a smoothing procedure: Each node will update its features based on the average of its neighbours. Obviously, it has a constant dominant eigenvector and therefore does not favor nodes based on structural information. We use a repeated convex combination of the identity and this diffusion process the backbone of our GEBM framework: The smoothing behaviour enforces a uniform distribution of confidence
PA= (1I+ within clusters and therefore is suitable candidate for cluster-level anomaly detection.
C.7 Synthetic Experiments
We motivate the three energy terms that we use to compose the energy measure of GEBM through synthetic experiments. All of them arise naturally from interleaving graph diffusion and energy
27

---

Model
CoraML
CoraML
Citeseer
PubMed
Amazon
Amazon
Cocuthor
Cocuthor
LLM
Computers
Photo
CS
Physics
GCN-MCD
10.5/20.3
10.2/20.0
10.2/19.8
8.9/18.5
10.4/19.9
10.7/20.2 2
10.2/19.3
10.7/20.6
GCN-DE
6.7/14.3
8.0/16.9
7.5/15.6
7.2/13.5
6.6/15.4
7.3/15.8
6.6/14.3
7.8/16.6
GCN-MCD+DE
6.8/14.5
7.6/16.5
7.2/15.2
7.0/13.3
6.5/15.3
6.6/14.7
6.8/13.7
7.9/16.7
GCN-BNN
5.7/11.9
6.1/13.7
5.3/11.3
5.1/11.4
4.7/11.9
3.8/7.9
5.1/10.8
4.9/11.6
GCN-Ens
7.4/14.1
5.7/14.1
5.8/12.4
7.7/14.2
7.3/15.9
5.3/11.5
4.3/9.5
5.7/12.8
GPN
6.5/12.8
6.1/10.9
7.5/15.7
5.8/11.3
4.8/10.8
5.1/11.1
6.7/13.1
6.2/13.0
SGCN
3,9/6.3
4.417.5
4.3/7.9
3.9/8.2
8.1/17.2
6.7/14.0
9.2/17.7
4.4/9.9
GCN-EBM
5.2/9.7
5.1/10.7
5.1/11.4
5.9/3.4
4.6/12.5
5.6/13.3
4.3/8.3
4.6/9,4
GCN-HEAT
4.7/9.3
4.8/12.2
3.8/9.2
5.1/12.2
3,9/9.9
5.6/14.1
4.4/9.2
4.9/11.1
GCNSafe
5.8/9.6
4,419.4
5.4/9.9
5.6/9.1
6.5/13.8
6,5/13.4
5.7/11.0
5.9/11.6
GCN-GEBM
2.874.7
3.415.9
3.9
3.875.8
2.7/6.00
3.0/5.2
2.7/4.7
3.0/5.0
Table 9: Average o.o.d. detection rank (AUC-ROC) (L) of epistemic uncertainty versus other epistemic measures / all uncertainty measures over all distribution shifts in a transductive setting (best and runner-up).
marginalization, as discussed in Section 4.3 additional energy functions as well, e.g. if such information is available in designated downstream applications. As shown in this study, we find that the proposed energy functions already suffice to enable GEBM to be highly sensitive to various distribution shifts.
We want to highlight that GEBM allows to incorporate
A Median Energy
1.0
Group
0.75
Indep.
Local
0.5
2-
0.25
0.0
0.00
1k
2k
5
O.o.d. Cluster Size
Magnitude
Heterophily
(a) Cluster-Level
(b) Node-Level
(c) Structure
Figure 4: Energy of different types for anomalies of increasing severity on synthetic data. We vary the size of an o.o.d. cluster real data (left), insert per-node anomalies to the energies of an SBM (middle), and increase the heterophily in an SBM (right).
Node-Level Anomalies. We study node level anomalies by generating synthetic CSBM graphs [57] (intra-edge probability 0.05, inter-edge probability 0.001, = 1000, 7). We then generate c-dimensional logits (intuitively confidence scores) according to a standard normal distribution that is centered at (diup 4 for the dimension of the true class label of a node and 0 otherwise. This roughly resembles the logits of well-trained classifier which peak at the true unknown class. We then perturb fraction p 0.05 of the logits by replacing them with positive noise from a standard normal distribution that is rescaled by an increasing magnitude. For each magnitude, we compute all three energy terms for each node and compare the difference in energy to the corresponding unperturbed energy. Effectively, this monitors how the energy of a node changes under increasingly severe distribution shifts that affect the logits of a classifier. We visualize the median of the energy differences in Figure 4b) While all energies increase with higher magnitudes, the atructure-agnostic energy term of GEBM is the most sensitive to anomalies that are located at individual nodes. It does not suffer from the smoothing of anomalous scores induced by graph diffusion.
Cluster-Level Anomalies. We study anomalies that affect an entire cluster of nodes by iteratively introducing an anomalous cluster the graph. To that end, we leave out classes and train a GCN on the Amazon Photos dataset. At inference, we re- re-introduce nodes (and edges) of the left-out classes which can seen as anomalous from the perspective of the classifier. Similar to the aforementioned node-level anomaly, we monitor how each energy type changes while iteratively introducing the entire cluster(s) of o.o.d. nodes into the graph. Again, we see that the cluster-level energy term of GEBM is the most sensitive this distribution shift in Figure 4a
28

---

Past
AUGURT
di
Su 1933
925
66.3 2837
MAT
DCNEARCTHON
28
(200
750
1GEN G3M00
kmm
HEAT
MUN-GREM
RON
Di No
AMBLE Than
DCNERRM
GOM-NEDD
CS
P12
-71
920
Table 10: Out-of-DDstributin detection AUC-PR (1) using alcatoric or epistemic uncertainty (best and runner-up) in an inductive setting.
Structural Anomalies. The last anomaly type we consider is creating structural anomalies: To that end, we generate synthetic CSBM graphs similar to the node anomaly experiment. However, we adapt the intra- and inter-class edge probabilities to accommodate varying degrees of homophily: We fix the intra-class connection probability at p = 0.05 and vary the signal-to-noise ratio AsNr which implicitly defines the inter-class edge probability as p/RR We measure the degree of heterophily as 1--lop log Ssnr and again compare how individual node energies change for increasingly heterophilic graphs in Figure 4c. We find that the cluster-level energy of GEBM is the only energy term that is sensitive to structural anomalies.
Overall, these experiments motivate the use of the three naturally arising energy terms in GEBM: Each energy is sensitive to different family of distribution shifts that originate from anomalies at different structural scales on the graph: At the node level, with the local structure of a node and at the cluster-level.
C.8 Overconfidence of GNNs
We disambiguate between previous studies on the underconfidence of GNNs 74 75 and the over- confidence issue of piecewise affine networks tackled by our work. The former studies the calibration
29

---

Model
CoraML
CoraML
Citeseer
PubMed
Amazon Computers
Amazonn
Coauthor
Coauthor
LLM
Pholo
CS
Physics
GCN-MCD
9.7/18.9
9.2/19.0
9.4/18.77
9.1/18.7
9.1/18.3
9.9/19.1
10.0/19.2
10.5/20.4
GCN-DE
6.3/14.11
7.9/16.9
7.2/15.3
7.1/13.3
7.4/16.4
6.9/15.9
6.3/13.2
7.8/15.4
MCN-MCD+DE
6.8/14.7
7.2/15.9
6.0/13.9
6.6/12.8
7.9/16.9
7.6/16.6
6.7/13.5
7.4/15.1
GCN-BNN
5.9/12.3
6.6/14.1
5.6/11.8
5.8/11.4
4.6/10.4
5.3/9.7
5.2/11.6
5.3/11.9
GCN Ens-EEs
7.4/15.7
6.8/16.0
6.8/13.5
7.2/13.4
6.4/14.9
4.4/8.9
4.9/11.1
6.4/13.6
GPN
7.1/15.3
6.0/14.8
6,4/13.1
4.8/9.0
5.4/12.9
5.3/11.8
6.8/13.5
5.6/12.5
SGCN
4.7/9.1
5.9/14.8
5.8/13.1
4.2/7.4
9.4/18.7
7.4/14.8
9.4/18.6
4.6/9.9
GCN-EBM
4.2/8.6
3.8/7.6
4.8/11.9
6.5/12.5
3.4/8.7
4.6/10.7
3.8/6.6
4.2/8.7
GCN-HEAT
4.8/11.8
4.6/13.1
4.7/11.0
5.5/12.7
3.6/10.9
4.7/12.4
4.6/10.6
5.2/13.7
GCNSafe
6,4/1199
4.8/10.1
5.6/11.2
5.4/8.9
5.8/13.3
6.9/14.7
5.4/10.3
6.1/11.9
GCN-GEBM
2.7
32/5.7
3.677.7
3.6/7.0
6.87
3.0/5.0
2.8/4.7
3.0/
Table II: Average o.o.d. detection rank (AUC-PR) (L) of epistemic uncertainty versus other epistemic measures / all uncertainty measures over all distribution shifts in an inductive setting (best and runner-up).
Max. Softmax Prob.
GCN
0.8
GCN
Energy
GAT
100
APPNP
GAT
0.6
GIN
APPNP
0.4
GIN
0
10
0
Perturbation Strength
Perturbation Strength
Figure 5: Confidence (Maximum Softmax Probability and negative energy) for different GNNs at increasing distribution shift severity.
of GNNs on in-distribution data, and provides strong evidence for models to be underconfident,. This does not conflict with claim that with increasing distance from the training distribution, i.e. out-of-distribution data, piece affine GNNs become overconfident. We empirically verify this in Figure Both the energy and the maximum softmax probability confidence measures of the GNN architectures visualized increase (or converge to their maximum) under more severe normal feature perturbations.
C.9 Robust Evidential Inference
APENP
APPNP-EBM
77
GEBM (ours)
80
Accuracy (%)t
5.0
2.0
1.0
20
0.5
0
25
50
75
100
7
Budget (%)
Figure 6: Robust evidential inference using APPNP as backbone for GEBM at increasingly severy feature perturbations.
30

---

Past
712
825
8225
925
Kk.b 2910
(200
27
06.3 J0N
MUN-GREM
Di No
AMBLED Thah
DCNERTM
GCM-NED
Ch
10
d6 HAAT
5k1nm
103
Table 12: Dfttto-Diisriiuution detection AUC-PR (1) using alcatoric or epistemic uncertainty (best and runner-up) in transductive setting.
We apply GEBM to an APPNP model and expose it to increasingly severe feature perturbations.
Even though APPNP uses the same number of diffusion steps as GEBM (t = 10), it suffers from deteriorating accuracy even when a small portion of node features is perturbed. Since logits and energy are unbounded and likely to diverge under strong enough distribution shifts (see Proposition 4.1). fixed number of diffusion steps is insufficient to recover from perturbations. At the same time, the regularized energy of GEBM converges to zero and, therefore, a fixed number of diffusion steps can effectively maintain high robustness akin to evidential methods like GPN.
C.10 Partially Perturhing Node Features
The feature perturbation- based distribution shifts discussed in Section 5.] replace node features with noise. We additionally study a feature-level distribution shift that partially corrupts the features of an instance. To that end, we vary the fraction p with which an individual node feature is perturbed and again measure how well uncertainty estimators can detect this distribution shift. In particular, we replace d p uniformly and independently selected node feature a; with Gaussian noise and keep d - d p features as-is.
31

---

Model
CoraML
CoraML
Citeseer
PubMed
Amazon Computers
Amazon
Coauthor
Coauthor
LLM
Photo
CS
Physics
GCN-MCD
9.9/18.9
9.4/19.0
9.1/18.5
9.5/19.1
8.9/18.2
10.1/19.4
10.0/18.9
10.5/20.3
GCN-DE
6.3/13.9
7.4/16.1
7.2/15.6
6.8/12.9
7.6/16.7
7.3/16.4
6.7/13.3
7.1/14.4
GCN-MCD+DE
6.7/14.5
7.5/16.2
6.9/15.3
6.1/12.3
7.6/16.7
6.2/15.3
6.0/12.7
7.2/14.77
GCN-BNN
5.7/12.1
6.7/14.4
6.2/12.8
5.6/12.8
5.3/12.6
4.9/8.9
6.0/12.6
5.7/12.9
GCN-Ei -ns
7.2/14.2
6.8/16.1
6.3/12.9
7.6/13.9
7.2/16.1
5.9/12.3
4.5/11.0
6.4/13.8
GPN
7.0/15.1
6.2/12.9
7.0/15.4
5.1/8.4
5.1/11.2
5.4/11.6
6.6/12.0
5.9/9.8
SGCN
3.9/6.9
4.2/8.8
4.2/8.4
3.5/6.3
7.6/16.5
6.3/13.7
9.3/18.4
4.7/8.5
GCN-EBM
4.8/9.3
4.4/11.2
4.8/13.1
6.4/13.3
3.8/11.3
4.6/11.3
4.0/7.1
4.1/8.6
GCN- HENT
4.7/10.2
4.4/12.9
3.5/9.8
5.2/12.7
3,9/11.3
4.9/14.1
4.4/10.7
5.2/14.0
GCNSafe
6.7/12.1
5.3/10.8
6.7/12.5
5.8/9.5
6.4/14.7
6.5/15.6
5.7/10.9
6.1/1188
GCN-GEBM
3,2/5.2
3.5/
4,219.3
8.5
2.77
8.6
2.8/588
3.2/
Table 13: Average 0.O.d. detection rank (AUC-PR) (J) of epistemic uncertainty versus other epistemic
measures / all uncertainty measures over all distribution shifts in a transductive setting (best and runner-up).
GCN MCD
GCN
GCN MCD+DE
GCN
GCN
GCN EBM
GCN HEAT
GCN GEBM
Model
GPN
SGCN
GCNSafe
DE
BNN
Ens
Global Rank 1
19.6
15.3
15.2
10.7
12.6
13.0
13.9
9.4
10.3
10.6
5.0
Table 14:
Global o.o.d. detection rank of each model, averaged over all datasets and distribution shifts in an inductive setting based on AUC-ROC scores (best and runner-up).
We compare the AUC-ROC of different uncertainty estimators while increasing the fraction of perturbed features for each o.o.d. node in Figure 7 While both the softmax-level uncertainty of the vanilla GCN and the EBM again become overconhdent, GEBM can reliably identify this distribution shift. This coincides with observations made for feature perturbations that affect all node features simultaneously and justifies focusing on that distribution shift for the bulk of our study.
D Ablations
D.1
Energy Different Scales
We report AUC-ROC and AUC-PR for o.o.d. -dteectin ablating GEBM in an inductive and trans- ductive setting with standard deviations in Tables 26 29 Additionally, we rank all variants of GEBM (i.e. energies at different structural scales) similar to the o.o.d. detection experiments in Section 5.2 We observe our proposed GEBM consistently be the most effective over different shifts simultaneously. We can also confirm the effectiveness of the scale-specific energies at detecting distribution shifts that should be covered from their definition. Both the best and second-best ranking methods are GEBM and a variant that uses unregularized energy: This shows the efficacy of scale-aware EBM for graph problems.
D.2 Diffusion Process
We also ablate the diffusion operator PA and its hyperparameters, i.c. the type of diffusion process (see Appendix C.6). the number of diffusion steps as well the teleport probability a, in Figure g We evaluate the performance of GEBM on representative of each distribution shift family. As expected, low and high aid node-level anomaly detection while the opposite holds for cluster and local shifts. Label- Propagatoonn achieves satisfactory performance over the entire range of diffusion hyperparameters which justifes using in our experiments. In particular, we did not tune any hyperparameters for good -dete-dttt As stated in Appendix C.6 it does not bias the energy toward high-degree nodes which explains advantages over the other diffusion types. It performs well over broad range of hyperparameters making GEBM less sensitive those.
32

---

Dutoect
Abs./ / Rei
LaC clum
Abs./ / Rei.
Pertf poneri
Abs. / Rel.
Abs Rel.
Rerdi 5)
A/o. do
Abs. /Rel.
Pags Bank
Abs./ Ret
Homophily
Abs. Rei.
(Weighaeds Aug.
Abs. RRt.
CreaMi.
11.77+1.06
+1.0/-11.00
+ 10.4 15.5%
-35.07 -01 356
+00.// 1127--555500 756
-16.5
+5554+777 7%
+10./+ 0015/+531-00 9/
CouML LLM
11.37 11.0%
12/2 12.5%
1227 117.8%
15237 1358.29
20.57 25.04
15.7 67.0%
-5.57 18.8%
Clipseer
11.51 11.7%
10.8/ / 10.9%
11.27 1.9%
1925/ 113.9%
148./ 1207 116
18.11 27.2%
13.7 13.3%
9..9 33.00
PiiMaal
+5.17
-0.67
-85-77472
+182/-1700
-0.74430
Anuzan Comgutere
-7.58
42.47
-2057 +37
+36.0/-194.00
-7.97
Amazon
-26.7
+01.77
+109/
+22.47
-2.99 20.78
Table 15: Improvement of GEBM in o.o.d. detection AUC-ROC over a vanilla logit-based EBM (GCN-EBM) in an inductive setting.
AUCUBOT1
Asc?
8001
Ass1
AUC-BCC
ACC-ACC
AUCABCY
Asa
COS BNV
OOSTEAAT
COULE
16NGGEM
COSTBAT
16
66N-GEM
METS
CEN
DEN 18
66N-0RM
GBM
DOSTRAT
GENUS
60%, 30
1872
470, m94
16N-GEMM
Table 16: Misclassification detection AUC-ROC (1) using alcatoric or epistemic uncertainty (best and runner-up) in an inductive setting.
D.3 Feature Collapse and Spectral Normalization
A recent line of work identifies feature collapse as a limitation in uncertainty estimation for deter- ministic models on i.i.d. data [46. 69 Intuitively, feature collapse occurs when out-of-distribution data is mapped to the same regions of the model latent space as in-distribution data. Consequently, density-based uncertainty estimators such as the regularizer of GEBM can not separate both data distributions accurately and falsely attribute high confidence to out-of-distribution examples. To
33

---

Meai
AL MMC
Bay
50%
124
IGNNGEIN
FENU MET
DOCH
17A
GENSUS
GES
GEN-2E
115
Gd
Tun
ICEN 6300
20CH
WEAT
GES-METHDDE
515
115
CONCEM
550
GENARIN
DEN MED
CON-IN
GENSAS
GEN MEDHOE
GEN
128
GONGRIN
963u
Run
Table 17: Misclassification detection AUC-ROC (1) using alcatoric or epistemic uncertainty (best
and runner-up) in transductive setting.
mitigate this issue, spectral normalization has been proposed to enforce Lipschitz smoothness in classifier [46] 47]. We investigate if this regularization benefits GEBM as well in Appendix [D.3]
In contrast to work on i.i.d. data, we do not find spectral normalization consistently leading to improved -ut-of-distribution detection. We provide two possible explanations: First, feature collapse has not been observed to occur for graph problems. The diffusion process at the backbone of many GNNs may mitigate the issue. Second, spectral normalization enforces Lipschitz smoothness in terms of the L2 norm of the input features of node and its representation. This notion neglects the structural influence of other nodes and, therefore, may not be suitable to quantify feature collapse in non-i.i.d. domains such as graphs.
34

---

JEE PRT
GENERE
43
14.0
GENOMETHOE
GENGRM
GONAEIN
GEN-35N
GENEMETADDE
GAN
GENCRAT
GENOE
GENCO
GENSEED
GEDN-METHOE
22
CS
CENURRAT
UCT M013
315m
11h--
314--d
550
23..00..
31
115-
613 203.0
201
5L2
255
056.
Table 18: Misclassification detection AUC-PR (1) using aleatoric or epistemic uncertainty (best and runner-up) in an inductive setting.
GEBM
AUC-ROC -O-CRO
0.75
GPN
GCN
0.50
GCN-EBM
0.25-
0.0
0.5
1.0
Fraction of features replaced
Figure 7: Out-of-distribution detection AUC-ROC for different estimators for an increasing fraction of perturbed node feature dimensions on CoraMI.
35

---

AUCERI
AUCeRI
AUCERI
Jen
CDN-380 300-3MV
EST
Sa 31200
CON-RRAT
(0%
CONCS
65
CEN-1RM
GOS-MED
$1A -15
Choose
CEN JNN
GEN-
DEN MLD
COV-3N
GENSAb
GEN MES3-00
El
MAAT
DET 913
21
CON-MED
CON-UTD-DD
T12
555m
ST3 (121
3xl -022
621
223m
55
042
CEN
05
00. 282mn
GEN-RAT
212
118 0550-
E. 95cm
945m
19760
Table 19: Misclassification detection AUC-PR (1) using aleatoric or epistemic uncertainty (best and
runner-up) in a transductive setting.
36

---

Meki
Oue (e-ria
Charrd rouhh
Conrid Hicck
Oue/la/0ed
Ose Der/ta/0
Ohe rear
22 22 of
12 Bo of
Cealc
CEN bi
Al Micatthour Si
mocuet 214
50th
Ma Sag of hig
Plaaiiiige Nor
on
Da raa
Mian it
boot
MET
na
VIT3
Thwnt Sg of
Elpstalent Bun rBClec
WE of TThe
Whon Soou tur
Sheat fille
Fli Piotatorta Illle
Chesti Aur 18au
thpin ber 11 das
GOO MCD-EC
105
CPN
COULD
nak cku
Sdoot That d8 fro tholll Ton labu
Sharfi fil Bosd dhore
Shaelll buctthha
adoul Al aid
na cla
1kb bud hus
Siwer (SiiBi
Hund
66N ME3O
Must Elger ochor
sooilltee for
Macil (i Bal
Mixst Aan//
D0%U WITH
Mosi Rf
Whe 12 festllll la-
Elpal Blowraaaee
rkloo thi crikk
Most la hosee
Miast (7Siir
B3C4
Sitfe mhg of re Teos
The fiiiese fe
12 Blaantt kortttie
If Wilee
ialund aedencc che Tuo
Whork ta lan Lhot////
Ea
SCDR
Cha 1S0 1212..
Co (630a oe
2 Thulfhu Sorleholii Io
BS
Sdver 200 /8TT is
18 22 ou didee
1ike wf se
38 ont Mill for
GLN ME3--
Whaul tileetiiiin
Mieerihh Mieriihherizz rid
All durdidd hor
BGCX
33 ael (12b
OUN
Shantiion// Iclus
12 Ataneriiund asue
12tool Suhuel
120 11
dale
thast Ai
Rewer du tha
Tarott heatthan at Pid 1ad..
Be 18
Table 20: ECE (4) for different backbones clean data as well as i.d. and o.O.d. nodes after
distribution shift in an inductive setting (best and runner-up).
Label Prop.
Stochastic
Symmetric
0.90
Diff. Steps
LoC
0.85
0.80
Dift. Steps
(00 1)
0.9
Homophily
Diff. Steps
0.7
0.6
0.0
0.3
0.6
0.9
0.0
0.3
0.6
0.9
0.0
0.3
0.6
0.9
Figure 8: O.o.d. detection AUC of different diffusion operators on Leave-out-Classes (top), (0,
(middle) and homophily (bottom) shifts.
37

---

Vedi
kays
Hodk
Haregbiyy
Shud Buulitttt
11 2a Ted
1s di aod thie
SCCS
The 1ntier bhe
BOCk
30 ADres Te
Ea 18a-
S4 Mouridhoiitt.. Aluafth
64 S0
32 will
Thurt rekust talu
06%
S0th
the ckk.
Moofk Hid
Skast Basuii fur
M2 /163
Lieatide Jus Ii
Baunr tu 1108 tor
Klouth Ty 198 te
sheat Cor Seoi to-
Hi fuott Te- will
It Se rill besta ig Alorhhowe
shoet Mertiiir//be
Morit Beritt bur
ROCY
Baue on lin
23 2hnet23hr//ka
ta iaa (ks.
37 aut Saut diin
28rell4 and Li3
900%
2horl tu 2nr
las the.
Eartlthnn idhas
Most bo.
42o/4 bos dot bor /ate///
dhuest forehioe les
31 surii
the 1s lest te
id StStorath
S0OS
Elu (a0
Aly Alyartilautt Burtid ser
RIEN
Thus Mile fuit
Mant3 fur
25 Th bel
D0%
Spoft fo de bei
Theatt Jan he
Cheitt rit ho bei
Shu (8 beil het
the of tan
125 122 welithien
17 TTue ao Fildion
de sarilk iwiihhn
LE de ai tho
ast Suurr Jor of citiie
Eiae Sahee & t
Shoeth sochtau
Re Bdc
Styer/ttte 52 "Slyr/li Bilss
Eipr///ggut Not
"ye by by by
RIEX VET
Eignd Marr/ No:
2 (11 lo
M
Da Tast
45ord4 bo. lus
Man/ 21 cal
Masti (f Sur
4borf Bool 14
240 un 244 Bdine
205 Rab 216
220 221 19d
23 of o B.2....
Table 21: ECE (L) for different backbones on clean data as well as i.d. and o.o.d. nodes after
distribution shift in transductive setting (best and runner-up).
38

---

Besti:
Dan BO
I7 rOuh
102 /02
OCMMEEDDE
(U2) sad of 133bm
Rabeorti (65 Be
(od 133mm
rug no serte (82
(PR)
CUNLOE
GEN MCD-DE
(Eae Dos
Allue wew./iihaaeei ao
(9%)
(Allwetti (81
DON MET
CUTUMITNDE
CIN
03 E3 Bor.
A1Deettllber3 She
(3) (id ( E5
DEN MED
valanies
Iillee oh Blan
CENUMCTHDE
Thm
Mase
GP%
All Af L3les
Alliet Frae
826 82h--10dhh... 6.36
OUNAEDD
AlDo//BBeoff Eio.
(ioer0000n/111. the
DCN
Plar
atfww//iiloor Boos
GONMIEDDE
GPR
6d8 an Lifes
1277 indh werk
OONARED
Alleet Bibeer Mio
033 Laleee Sh.-
All Hith Hidbee
OCN
A
MCTHOR
(0%)
U/ PM of
a ina (84
DENOMET
Wiker jull fethes
OUTMER
Bd Mo.
aither Me Ban
Alllow/ Toaer 2oae
MEDDD
CIN
It' (Su.e
Hll Milee//tD..eeee 7034
MEN
A13 ADDa//tibeed 1hn
A1t aur the
608 100.w//0000a
100 cirt /edka.
CPN
IOCN 10E
Table 22: Brier score (4) for different backbones on clean data as well as i.d. and o.o.d. nodes after
distribution shift in an inductive setting (best and runner-up).
39

---

Meed
Deerl4
Rork-11
Sig Ficket
Auy Rook
Heuplay
112 /138m..
(ENUMIT)
BS5 17
Al Hill flyss
6.0 Aecheddab (1 2h..
827 A22ac//23ian 8.33
SOEN
(02 12has
LOCH
Mibeettbbork Bie
Coalt 13M
ROCD
612 /BBDer
calaclas
025
Albee/kitand WD (0--//22ion111 1it
MUEN
JOCH
BULD
AlBaasi (i Mase
Wus ael (1.2...
1LB SeeeriiBee Med.ee/iiiii (1.3
(DS 1623
SCEN
6a4 seftatte (aattoa 1hoe
ION
iTlael Blos
WC Man
on aod tiii (1 2013
833 (B
SEEN
Bid woit
LOCM/
WilhertHeer They
IlTree/iiiiow Phice
AlbasdBlllo Tros
1L5 wo5-wn/0d3l rBab...
au ae
Altaed B.3
AHee// Blus
nibae (1 Max
GON
OORMITEDDE
SDEN
M0 Of Lllts
pikker Barerrab Mige
(10 (ddoellikuud
LOON (OLD)
ROCK
(o /8 Bha
Uffuwd Thus Mee
Rollee thuue thas
GOM-E
Albae/klbarri
US E0 dio-
Ail Aibaeltttaed
All Aitosellll
SOON
UUS
LOCN
OUNEEE
Wil to Hul fellee
SOON
LOCN
5/6 Migartiiiien Bec lips
Eill poar and Bigge
BUAN
100
DEN
peer/eeaa.o 6 thes
iituee Bia- 19ss
SOON
000 (eee/ddbeer 2.3h
100 (o0ae//0000
CPN
Table 23: Brier Score (L) for different backbones on clean data as well as i.d. and o.o.d. nodes after a
distribution shift in transductive setting (best and runner-up).
Model
LoC (last)
LoC (hetero)
Ber(0.5)
Ber(z) (near)
N/ I) (far)
Homophily
Page Rank
GCN
EBM
89.920
88.4
58. 72n0
67.1220
20.42
71.221a
74.2
Safe
91.62077
20.5
53. a
56.9 7
36.1299
73.
50.6206
GEBM
91.620.
89.5
94,3
77.1ara
86.621
76.7
57.7
GAT
EBM
20.2
89.7
58. 18ar
55.621
14.1
11.1
51.421a
Safe
91.5-a.a
90.8 +0.8+0.9
54.6+2.
53.2+1.
45.4-1.6
70.3+1.9
49.9+1.1
GEBM
85.0+a.4
83.5+a.0
89.6
69.3
76,5-4s
72.6+2.5
53.3
EBM
26.5
71.924.9
Gaza
52.3az.
42.8=1.
53.2+1.9
69.2
GIN
Safe
79.0 =a.0
76.4
47.6
49.3+1.a
46.2=1s
51.2+1.
4
GEBM
80.7-4.3
76.4.o.1
51.6+1.9
51.4+2.
53.6-17
65.4
52.4+1.
SAGE
EBM
24.0
75.0
47.1
52.7
42.2
53. 23..2
68.921a
Safe
27.3
78.2
18.
49.2
46.621
51.1
48 1
GEBM
77.3
73.3
51.9
51.6217
54.7
62.7 13.0
53.0
Table 24: O.o.d. detection AUC-ROC(1) using different backbones on CoraML in an inductive
setting.
40

---

Model
LoC (last)
LoC (hetero)
Ber(0.5)
Ber(p) (near)
NN (far)
Homophily
Page Rank
GCN
EBM
85.5
18.s
53.9ga
63.5gaa
36.1
66.31a.
7162sa
Safe
76.5
41.0ga.
55.3gn
40.3_>7
63.7aa.
54.02a.
GEBM
88.0
75.
94.3
70.6
88.9
68.8
56.8
EBM
86.4
80.3
56.3
56.521.
45.2g
67.0.1.a
15.1
GAT
Safe
21.
83.1
53.5gaa
54.221
45.4g
62.1
53.3110
GRBM
79.5
64.4g19.
88.620
61.924.7
81.82
66.52a.
54.7a
EBM
70.2
49.1
46.0
52.8
44.0
51.41a.s
65.7
GIN
Safe
o2.3
51.4gm.1
47.9
50.3
46.6_2.
16.1
50.1
GEBM
75.1=a.s
52.7
51.2+z.
51.4+2.
52.9-2.
54.7+2.8
54.0+2.
SAGE
EBM
66.62a.2
35.321a.
46.4 46.4
53.5
43.4ga
52.524a
65.4
Safe
70.2
56.4
48, 48.5
49.9
46.9as
46.5aar
50.3
GEBM
70.6
49.3
51.5aza
51.4ez2
53.1
55.1asa
54.5a2a
Table 25: O.o.d. detection AUC-PR(") using different backbones on CoraML in an inductive setting.
LoC (lon))
LoC (fetems.)
Bertz)) (near)
Ber(0.5)
A, 1) (ar)
Page Rank
Homophily
Rank(L)
FBM
89.9
88.4g2
67.3
58.6
25.Tara
74.2
71.22
3.9
Indop.
81.8
78.5
87.1
99.51a.7
96,32aa
63.9
69.0.a.
3.7
CoraML
Local
87.5
86.91aa
57.4-14
57.9
63.1422
81,31a
4.3
Group
92.8tua
90.2
55.5
60.9
63.7ava
49.5gx.
71. 71.8
3.4
GEBM-N,
92.2
89.9
75.9
72.3g
1.2gm4
57.6
74.321
3.2
GEBM
91. 91.6.0.
89.52117
77.7
94.512.4
86,42a
67.7.t.
76.721.
2.5
EBM
87.1
87.5iaa
n.a.
70.0ss.e
33.0+2s
72.7.1.4
69.7.a.
3.3
CoraML- L.M
Indep.
78.9
82.0
92.
96.02zm
53.2
65.1gsa
3.6
Local
84.5
84.922
65
62.4g
50.9
80.621
3.7
Group
91.611a
91,411a
n.a.
58 6...4
62.4aa
49.6ua.7
70.7.1.
2.9
GEBM-E,
89.8
90.411.7
88.5
7.141.4
51.2-a.6
73.1
2.7
GEBM
88.4
89.7
n.a.
82.2
85.3gxa
51.92a.
75.0gs.
2.4
EBM
87.4
43.8
23.5a1a
66.4
52.0ea.
3.9
Citoseer
Indep.
78.2
76.521
69.3g
99.622
75.5gx
50.621
51.9gs
3.7
Local
86.02aa
90.811
60.021a
65.6
63.02ae
47.622
55.521
3.3
Group
90.611.
88.8111
55.5...a
70.11a
61.71a
46.2.a.
53.0.aa
3.6
GEBM-E,
90.0
87.8
63.9-1.0
45.4+10.
5.6+1.
48.2-1.
53.2-aa
4.0
GEBM
88.9
88.7
66.0gs.
96.3
72.1
48.3g
53.7
2.5
EBM
65.8 65.821.7
67.8
48.6244
27.5211
72.62a.
52.221.5
3.9
PubMed
Indop.
66.0ra.
59.611a
n.a.
88.617.
12.5
10.9
3.2
Local
68.3
66.3
56.1
61.4aa
47.7
62.6s1a
3.4
Group
72.8
71.4
n.a
50.5gaa
60.3ara
50.6ga
56.4gx
2.3
GEBM-E,
68.5
69.219
52.62m01
8.1anr
49.7
58.92ma
3.2
GEBM
70.9
67.215.
n.a.
84.0
75.7
19.4...c
19.0
2.6
EBM
75.2
78.1
57.8us.6
52.1
42.621a
90.3
60.0za.
4.1
Amazon Photoo
Indep.
75.61a3
76.3
91.41a.
6
95.8
59.7
61.1isa
3.4
Local
73.1
90.6
51.4
49.2
57.3s1a
54.0caa
68.4
3.9
Group
89.9
82.3aa
52.8
50.7
57.621
53.52
59.62
4.3
GEBM-E,
85.4
85.52a.
92.5a1.
71.7ass
0.5ana
62.0aa.e
65.9
2.6
GEBM
63.6
87.5
81.8
61.2
92.42a
58.0
18.3
2.7
EBM
75.9
85.5
56.6-s.7
54.9
40.1
85.3
61.5esa
3.6
Amazonn Compulers
Indep.
74.4gs
79.2
90.0g
82. Ian
96.1are
51.82
50.2ga*
4.1
Local
75.7
15.1
51.8
50.9as.
57.414
54.02a.4
69.2
3.9
Group
86.614a
89.2
53.3 s.o
52.311.
58.811a
51.3
56.1:24
4.1
GEBM-E,
86.7
89.7
89.9
84.3
0.1gma
65.3
2.2
GEBM
81.8
87.9
85.2gx
75.4gs
94.1gen
52.621
64.4g
3.1
EBM
89.9
94.2
60.6
71.6gsa
28 8an
81.9
60.5ga
3.8
Counhor-
Indep.
90.3
92.1a1
99.2
100.0gro
92.72a
64.42
58.2
3.2
C
Local
73.8
87.612.
57.0
59.1
62.7 22aaa
45.4saa
77.0
4.9
Group
96.011.
85.6
62.3
65.3
63.9
51.4-a.
58.5
3.8
GEBM-A E,
94.5
97.5
95.5
97.5
0.2
50.9
64.7
2,8
GEBM
92.8
96.
95.6g1
99.5
90.0gga
56.5gm *
69.5gr
2.5
EBM
93.1
97.01ma
59.4.s.a
62.4ss.4
39.5111
82.011.e
56.5im.
3.6
Coauther- Physis
Indep.
89.6
94.6
91.9.us
99,9
73.4
65.7
55.7
3.7
Local
91.7
96.3
55.9
58.02
52.3
66.
4.5
Group
95.5
98.3203
54.7
57.0g
56.72
54.7
56.4gm
4.1
GEBM-E,
96.610s
98.410.
81.2.s.
90.0.1.7
3.6ane
62.2.1.a
58.3
2.8
GEBM
91.9
98.4103
81.81aa
95.3
72.74sa
60.1
61.310.
2.4
Table 26: O.o.d-detection AUC-ROC (f) using different EBMs in an inductive setting.
41

---

Model
LoC (avi))
LoC (hetemu.)
Bertp) (near)
Ber(0.5)
[V 1) (far)
Page Rank
Homophily
Rank(
EBM
85.5
75.1
63.8
53.9
35.941
71.6aa
66.3.2.4
3.2
ConML
Indep.
74.7
58.
99.5
97.92x
62.9g
61.4ga
3.6
Local
78.2
68.8
56.0=at
57.1
62.52xa
54.92a.4
77.72ta
4.1
Group
89.711.
73.1
51.7 Le.0
00.91a.
51.1.a.9
61.11aa
4.3
GEBM-E,
88.8
74.3
69.2
63.4
30.641.
55.9-1.
64.5ias
3.5
GEBM
88.0
75.1
71.3gz
94.6gaa
88.8gx
56.8e1.
68.8gaa
2.2
EBM
84.0saa
75.9
66.52sa
3.0
Indep.
69.6
63.0as
82.1
97.5gx
54.7
58.6g
3.5
CoaML LLM
Local
25.2
61.924a
n.a.
62.lasa
60.52aa
54.4
77.621.
3.5
Group
88.1
77.9
56.311.
59.51aa
51.6.a.
59.7
3.3
GEBM-E,
87.7
78.8
n.a.
83.2
30.7
52.4-s.a
64.5
2.8
GEBM
84.7
76.8
72.8ea
87.3gx
53.2
67.5gaa
2.6
EBM
71.0aa
76.314a
59.0.2.4
42.62a.
34.8a114
59.1
51 8asa
3.6
Indop.
51.8ia
59.111.
63.5
19.6
85.4
49.8
51.3
3.7
Citeseer
Local
63.7
82.0aaa
57.5=
67.2es.
62.54sa
48.5
55.5
2.9
Group
71.3gs
75.7
52.7
70.9
60.5gx
45.7
52.1
4.0
GEBM-Ep
74.3
74.1
56.3
42.9as.
30.7 30.7a14
46.821.4
52.1
4.2
GEBM
69.11as
78.5
$9.31..0
96.612.7
75.31a
47.2
52.411
2.7
EBM
53.51aa
33.01s.
n.a.
16.1
39 19.1
66.0
53.7
3.4
PubMed
Indep.
51.7us
248122
89.1
89.5
50.5
53.3
3.2
Local
50.4sia
29.0eaa
n.a.
55.8
61.1asa
47.5ga.
61.4
3.8
Group
60.62s
40.6277
59.1
50.4gm
52.32
2.6
GEBM-E,
$7.2
39.01a.7
n.a.
47.7a5
31.8204
49.0
56.021.
3.1
GEBM
56.7
33.11o.a
83.8
49.0
16.1
2.5
EBM
63.3
51.4aaa
59.0=s.
52.0gs.
45.1
89.0gs.4
62.1asa
3.8
Amazonn
Indep.
65.6
48.92a
88.7
58.922
97.621
57.2
60.3
3.4
Pholo
Local
65.6
73.9
50.8
19.1
56.911
55.0.a.
65.8
4.0
Group
80.611.4
49.1
52.5-1.a
19.8
56. Ta
51.2-a.6
55.0.
4.6
GEBM-E,
76.2
30.5
63.8
2.7
GERM
74.9
62. l
77.0
55.7
94.8g1
55.5g
66.4g
2.6
EBM
53.7ara
79.1
56.6
54.2ga.
43.5g1
81.8
60.0gsa
3.3
Amazon Compnilens
Indep.
49.42se
63.92s.
85.9
73.
97.8241
49.521
57.8
3.9
Local
18.314a
75.317.2
50.7
50.1
57.01aa
52.2
67.31
4.1
Group
70.1
77.4is.
53.2-1.6
51.8
58.812a
51.8-sa
4.3
GEBM-A E,
68.4
81.3
84.9
75.9
30.62ua
49.9
60.3gsa
2.7
65.0g
77.62m
79.5
67.4gm
96.1124n
50.1gr
61.3g1
EBM
89.611.
69.4.1.
71
37.2.0.
77.8
60.9
3.6
Coauther-
Indep.
90.2
63.7
98.7
100.0.c.
95.9
61.5as.
55.0
3.0
CS
Local
57.4
61.72*4
47.5ga
77.1gia
4.9
Group
96.320
80.8241
60.8
64.221
62.6217
52.8
51.621a
3.9
GEBM-E,
95.1
88.612.
92.6.e.
96. 06.011.4
30.6 61.
58.1
59.7
3,0
GEBM
93.4
81.6sa
93.9
99.61o.
92.8+aa
55.2
65.7
2.5
EBM
79.8
92.011.4
58.8 1.2
62.1
43.6
77.3
56.9
3.1
Coauthor- Physies
Indep.
82.5
90.6...
83.9
56.3esa
3.5
Local
64.5ga
84. 84.2g1
53.0
54.5
57.9g
50.9
68. 68.92
4.5
Group
88.2
95.21o.7
54.0=s.
56.7aa.7
56.241a
53.5
62ne
4.2
GEBM-E.
89.212.4
95,310a
71.8....
85.11a.
31.010.5
58.1
56 56.81a.
3,0
GEBM
84.7aa
95,410a
77.1
94.1
80.1s44
55.9-1.
61.3
Table 27: O.o.d-detection AUC-PR (f) using different EBMs in inductive setting.
42

---

Model
LoC (avi))
LoC (hetems.)
Bertp) (near)
Ber(0.5)
[V 1) (frr)
Page Rank
Homophily
Rank(
EBM
87.5
85.312a
68.7
61.1esa
24.741a
73.0
67.9.2.
4.1
CorML
Indep.
7
7
98.52
55.9gm
63.9
3.7
Local
86.9
S5.421a
56.52n
54.52aa
62.1a1a
49.9
79.821.
4.3
Group
91.711.
89.611.
07.9...0
61.311.
62.8
18.1
68.911.
3.3
GEBM-E,
90.1
88.0.1.
79.6+1.
76.9ta.
1.4sna
52.3-1a
71.14aa
3.1
GEBM
89.7
87.9
78.9
92.4ga
89.8gx
52.7
73.
2.6
EBM
28.2
69.4
69.0asa
3.5
Indep.
89.7
79.7297
91.92
95.3gm
51.2
62.7gaa
3.4
CoaML LLM
Local
89.7213
85.912.
n.a.
54.2am.
63.021a
51.229.5
79.721
3.2
Group
89.71u.
91.011
57.512.9
62.91a.
51.2
67.712.
3.2
GEBM-E,
91.2
90.6
n.a.
88.9
7.6auo
51.0sa.
71.9aaa
2.9
GEBM
89.7
89.1
81.1
86,0aa
51.2gna
72.9
2.6
EBM
85.7
85.5
65.8.1.o
55.0as.
23.3a1a
66.321.3
50.621.
4.0
Indop.
76.3
14.3
69.2
100.0.c.
i3.5
50.6
19.2
3.7
Citeseer
Local
85.7
89.1
61.0=s.
72.7as.a
66.34sa
47.4.s.
54.7a0a
3.2
Group
88.7
87.2
5
73.0at
66.2
45.8
51.6214
3.4
GEBM-Ep
88.4
85.429.
64.7
89.8
5.52er
47.8
51.32sa
4.2
GEBM
87.1
86.7
66.611.2
18.5
81.1iaa
18.0
$1.91a
2.5
EBM
66.1
62.8
n.a.
58, 5....
71.8
53. 13.4
3.9
PubMed
Indep.
61.rss
59.612.
89.9
75.2
50.8
53.5
3.4
Local
66.7
63.2
n.a
57.5exa
60.9
47.8
61.62sa
3.5
Group
69.3gs
70.42s
58.62
58.92s
51.02
53.4gh4
2.5
GEBM-E,
68.42..
65.4as.a
n.a.
86. 7
8.0an4
49.421.
59.921.
3.1
GEBM
68.413.
66.3
84.0
73.0
49.6
57.1
2.4
EBM
75.2
75.9
58.5
51.5es
41.3as
90.5
56.8
5.1
Amazonn
Indep.
75.5
76.2
86.8ga
59.72a
93.828
58.221
62.2g
3.1
Pholo
Local
76.9
91.41aa
71.6110
56.3
77.5amn
54.52a.
62.211.
3.6
Group
89.7
81.515a
72.7
57.0
77.5
53.4-1.1
62.2
3.5
GEBM-E,
66.3
0.5gm
60.7
2.9
GERM
84.2ga
87.4as
83.3
58.4g
92.0k2
57.8g1
62.2g
2,8
EBM
75.3
84.5gxa
57.8
54.62a
40.8a1a
87.2
60.6gax
4.1
Amazon Compnilens
Indep.
77.321
79.521
89.5
80.7248
94.2262
50.921.
55.921a
4.1
Local
77.1
89.5
52.2
50.51a.7
57.911
53.7
68.2
3.6
Group
89.7
91.+ia
58.841a
51.3-1.
52.4ras
3.6
GEBM-A E,
85.
89.1
91.7gxa
81.8
0.2am
52.7ema
63.74ns
2.7
84.8
84.3gm
72. 72.9gs
92.01ar
52.1gm
61.0g
EBM
G1..
91.8
68.7
70.9
29. 29/1ooa
81.5
59.5
3.8
Coauther-
Indep.
89.5
88, Sa.9
99.1
100.0=co
97.6.1a
63.2
55.6
3.0
CS
Local
73.1
57.62s.
58.1
63.92
45.6ga
77.8
4.9
Group
96.521
93.8197
61.721
64.0a
64.5214
51.82a
54.521
3.8
GEBM-E,
94.211a
96,411
95.3...
97.51a.5
0.4aa
59.1
63.7
2.8
GEBM
92.7
93.612a
96.0is.e
99,2
95.14pm
56.0-a.7
68.4saa
2.7
EBM
91.51a
95.311.4
59.7
62 62.0
39.610.
S3.0
56.5
3.8
Coauthor- Physics
Indep.
28.2
92.9
49.5
81.5
66.4
54.9saa
3.7
Local
89.7g1.
93.82aa
53.9
55.7g
58 58.8g0
52.4
67.9g
4.5
Group
94.812.
97.8107
56.1
67.8
58.341o
55.2
56.621.
3.6
GEBM-E.
95.711.a
97.6111
82.5
90.01a.
1.0.e6
63.0.a.4
58.7
2.8
GEBM
93.7
97.2
83.8
93,44a7
78.6
60.9
61.9.1.
2.7
Table 28: O.o.d-detection AUC-ROC (f) using different EBMs in transductive setting.
43

---

Model
LoC (lam)
LoC (hetero.)
Berlf) (nnar)
Ber(D.5)
[V[0, 1) (far)
Page Rank
Homophity
Rank(()
83.0zaa
60.62aa
66.1
56.3asa
35.0214
70.2
62.52aa
3.6
ConaML
Indep.
72.2ut
53.512a
82.1
98.1
99,8.0a
16.3
56.6
3.7
Local
78.0
64.7aas
53.8
52.4
61.9.aa
54.1-s.4
76.0
4.0
Group
88,0
72.04a7
55.5=aa
60.0es*
61.Taar
50.5
57.6asa
3.8
GERM-A-
86.5
71.6aa
71.7a**
67.5gma
30.121a
52.8
61.4ga
3.6
GEBM
86.1
73.2
71.2:44
91.615a
91.7ata
53.3.a.7
64.311a
2.4
EBM
84.0247
73.0aa.
66.0a4.4
41.02a
69.62a.5
64.72s.
3.3
CoraML: L.M
Indop.
Local
85.9
85,9
64.0ssa
58.815.
n.a.
82.6144
52.5
6194aa
11.2.114
52.5=a.4
.2.5
77.2
a5.3
3.7
3.3
Group
85.9
75.0
n.a.
55.9gss
61.7
52.5ga
57.7
3.5
GEBM-Ep
88,32a
78.3249
84.8
31.Tara
52.629
62.621.
2.1
GEBM
85.912a
61.6
n.a.
72. 2.sa
88.412
52.5...4
65.211.
2.7
EBM
66.3
72.1
60.8
50.0as.
35.3
60.2
51.3ea.
3.4
Indep.
48.6
57.7
62.7
100.0grm
96.228
50.0g
49.6
3.7
Citoseer
Local
60.424.
78.7
58.6...4
73. 73.42as
65 65.8
48.3
54.721a
2,9
Group
67.015.4
71.91a
52.8
7141as
61.5124
45.7
51.110
3.9
GEBM-E,
68.8
69.8
56.4-x.
51.7as.
31.2s2a
47.2
51.3s4.4
4.4
GEBM
6
59.1
98.6
83.3g
47.6
51.4ga
2.9
EBM
53.9
28.6
53.1
38.0so
64.4gaa
55.1
3.2
PuhMed
Indep.
50.5g1
24.321
n.a.
90.3ga
84.927
50. gan
52.5ga
3.4
Local
50.6
27.7
56.413.
60.2aan
47.2
59.611.4
3.7
Group
58.81a.4
18.2
n.a.
58 12.
57 57.1ia
49.7 49.7.a.s
50.1
2.6
GEBM-E,
56.5par
34.3
57.7ana
31.30a
48.4
S7.5
2.9
GEBM
55.9gs
30.8
83,3gn
77.3ga
48.4
54.3gra
2.9
EBM
64.3
19.1
39.3...6
62.0ua.a
44.311a
89.5
58.9
3.8
Indep.
66.4
48.3
81.1isa
f2.8
95.8
55.8
58.612.
3.4
Photo
Local
62.5
75.5
67.1
52.1
79.4
55.3
58.62ua
3.9
Group
80.0ge
50.0
2m.
52.1
79.4
51 51.3ga
58.621
3.7
GEBM-Ep
78.317.
52.3
90.8s1.e
89.92s
30. Tan.
57.62a.s
58.612.4
2.7
GEBM
77.1
63.2 u0..
75.7 4.9
51.91s.
94.4
55.3 a.9
58.61a.
3.5
EBM
50.9
17.3
57.4...1
53.31a.
7
84.4
59. 59.1is.
3.8
Amazon Computers
Indep.
49.6.2.
63.9 63.9
84.1.2.
70.5
96,7-4a
49.2
65.3
4.1
Local
49.2
79.8
49.0g
57.32x
51.8
67.7
3.9
Group
72.8ga
79.92>
58.021
51.72
58.5 58.529
48.9
49.3gna
4.0
GEBM-E,
61s.1
19.9
87.9
73.615.
30.31o.
0..0
59.610.7
2.6
GEBM
66.81a.5
80.4
77.1-1.9
61..i.
94.744a
49.7
59.5
2.6
EBM
87.8
72.3
69.5
72.2g
38.02oa
77.5
60.5ga.
3.6
Countor-
Indep.
88.721a
68.9274
98.8
99.92m.
98.7211
60.72n
53.02a.
3.0
CS
Local
61.0
16.914.a
55.2
56.31
63.31La
47.6.a.9
77.8
4.9
Group
96.5
74.8aaa
60.8-1.e
63.4ss.
63.6641
52.9-a.r
48.8
3.9
GEBM E,
94.7
84.7
92.8
96.3es
31.2
57.5
59.3esx
3.0
GERM
92.
74.927
94.9ga
99.22n
96.4g1
54.0ga
64.7g1
2.5
FBM
75.3gs
88.02aa
59.4g1
61.1g
43.322
78.4
57.52a
3.6
Counthr- sics
Phy shes
Indep.
60.5ga.
79.422.
91.521.
99.1
88.929
61.1
55.52a.4
3.2
Local
57.31as
74.5177
53.0.e.
53.9
58.
58.5111
51.1.a.1
70.5
4.8
Group
86.41aa
93.742a
54.7=s.4
56.8
57.34oa
54.2-s.s
53.5.s.
3.9
GEBM-A E
86.621
93.1
76.3
84.4g
30.7
58.8
57.6 57.6gs
2.8
GEBM
80.62a.
91.524a
78.62xx
90.924
84.4am
56.7
61.7an
2.7
Table 29: O.o.d-detection AUC-PR (1) using different EBMs in transductive setting.
Model
LoC
Ber(()
A 1)
Homo.
APPNP
EBM
90.5
71.9
17.0
73.3
Safe
91.6
55.9
33.1
73.6
GEBM
88.6
67.9
67.9
78.8
0.5
EBM
60.3
52.2
17.9
52.2
GCN
Safe
58.7
50.0
31.9
50.8
GEBM
14.1
35.5
99.6
32.8
0 1.0
1.0
EBM
75.4
60.6
3.6
61.6
GCN
Safe
74.4
53.9
20.0
61.6
GEBM
29.9
37.5
99.6
40.4
2.0
EBM
84.1
75.4
2.6
67.2
GCN
Safe
89.8
60.1
17.4
69.1
0-
GEBM
89.4
61.1
99.6
75.9
Table 30: O.o.d. detection AUC for APPNP and GCN with spectral normalization at different weight
scales a.
44

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We identify three main contributions in the introduction Section T Our novel model is described in Section 4.3 the formal proofs regarding integrability of EBMs and given in Sections 4.2 and 4.3 4.3 novel evidential interpretation of our EBM framework can found in Section 4.4 We support the claimed efficacy of our model at different structural scales by its strong -i-f----ittitttionn detection performance on a comprehensive suite of distribution shifts (Appendix
Gaussian regularization
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We discuss limitations in Section 6 We only aim at epistemic uncertainty estimation and can not improve on aleatoric metrics. Our method is tailored toward node classification homophilic setting, which is also acknowledged. Our method is less effective centrality-basse splits, which is discussed in Sections 5.2 and 6 and in-depth in Appendix C.6
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best
45

---

judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
We explicitly state the assumptions that we deal with piecewise affine classi- fiers and provide formal proofs for all statements in Appendix A For the core theoretical contribution, we provide intuition in Section 4.2
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proof' provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide code to reproduce our experiments online. Furthermore, we additionally also list all hyperparameters for all baselines and model explicitly in
Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (c.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution primarily new model architecture, the paper should describe the architecture clearly and fully.
46

---

(c) If the contribution is a new model (e.g.. . large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
We provide code reproduce our experiments.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
Souiides/CoeeSumiise
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ //nips ptipss.cc/uulicc/uuudde http://pbblcc/uiiiico//ooeeuuaaa Swwm//ooooesssssieeoeeeea.. iimmiisssionn aswssssiicciiiii for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Experimental details, entailing dataset splits, architecture choices, optimization, etc., are provided in Appendix Additionally, they are listed as default confi configurations in the public code.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
47

---

Answer: [Yes]
Justification: We omit error- error-bars and standard deviations in the main text for clarity, but supply them in Appendix C They are explicitly referred as standard deviations.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Details about compute resources and a discussion about memory and training time are provided in Appendix B
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't 't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: Our research aims improve reliability of GNNs. All datasets are commonly used benchmarks that are publicly available. Our paper fully conforms with the code of ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
48

---

The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We aim to contribute to making Al more reliable by improving on uncertainty quantification. We discuss the impact of our work beyond commonly known risks and concerns research on Al Section
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation. generating fake profiles, surveillance), faireess considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: We do not believe that model has an increased risk for misuse. We encourage users for active evaluation of the uncertainty provided in Section
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
49

---

Answer: [Yes]
Justification: We cite papers producing the commonly used node classification benchmark. We downloaded all datasets from Py PyTorh Geometrie, described in Appendix [B] We report licenses where we could find them. All datasets have been used in open research for several years and form well-establish benchmark.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g.. website), the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided. this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: Our paper does not provide new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
50

---

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
51


Reviews:
[{"review_id": "9EN0AGki09", "reviewer": "Reviewer_nGhJ", "summary": "This paper introduces a method for post-hoc epistemic uncertainty estimation in logit-based Graph Neural Networks (GNNs) by aggregating energy scores at different levels, including node, local, and group levels. Extensive experiments show the effectiveness of the proposed framework.", "questions": "1.\tHow does the paper perform inductive training on the GCN backbone when OOD nodes and edges are excluded during the training phase? Does it use graph sampling or data augmentation techniques?\n2.\tIn corollary 4.3, what is meant by ' any $x\\in \\mathbb{R}^d$ ’? Please provide a precise range for $x$ or probability. \n3.\tIn the Equation (9), the regularized energy from three structural scales equally contributes to the final energy score. Table 3 shows varying impacts of energy at these scales. Why was the decision made to use equal weighting?\n4.\tThere are inconsistencies between some model names mentioned in Section 5.1 and those in the tables.\n5.\tWhat are the differences in distribution shifts used in this paper compared to those in GPN or GNNSafe, and why did the authors make these changes?", "limitations": "YES", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1.\tThe paper rigorously evaluates the proposed method under various experimental conditions, such as out-of-distribution (OOD) selection, different GNN backbones, and both inductive and transductive evaluation settings.\n2.\tIt comprehensively aggregates uncertainties at multiple levels in the graph, including node-level uncertainties, class-specific neighbor information, and propagated energy through diffusion.\n3.\tThe manuscript is well-structured, with a clear presentation of concepts, logical flow, and detailed preliminary knowledge.", "weaknesses": "1.\tThe paper lacks a detailed discussion on the selection of hyperparameters, especially for the diffusion module $P_A$. Specifics about the parameters $\\alpha$ and $t$ mentioned in Appendix C are not sufficiently discussed. Including ablation studies on different graph diffusion architectures, such as label propagation referenced in the Appendix or APPNP used in the GPN paper, would enhance the paper.\n2.\tThe paper states that common GNNs suffer from overconfidence due to their similarity to findings on ReLU neural networks[1]. However, literature [2] [3] suggests that predictions from shallow GNNs are typically under-confident. The paper will benefit from evidence on the over-confidence issue of GNNs. \n3.\tSection 4.4 discusses the relationship between energy scores from logit-based classifiers and total evidence in evidential models. The paper lacks an explanation for why the proposed model outperforms evidential models in epistemic uncertainty prediction, particularly how it addresses the feature collapsing issue in density-based models [4].\n\n[1] Matthias Hein, Maksym Andriushchenko, and Julian Bitterwolf. Why relu networks yield high-confidence predictions far away from the training data and how to mitigate the problem. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 41–50, 2019.\n\n[2] Wang, Xiao, Hongrui Liu, Chuan Shi, and Cheng Yang. \"Be confident! towards trustworthy graph neural networks via confidence calibration.\" Advances in Neural Information Processing Systems 34 (2021): 23768-23779.\n\n[3] Wang, Min, Hao Yang, Jincai Huang, and Qing Cheng. 2024. “Moderate Message Passing Improves Calibration: A Universal Way to Mitigate Confidence Bias in Graph Neural Networks”. Proceedings of the AAAI Conference on Artificial Intelligence 38 (19):21681-89. https://doi.org/10.1609/aaai.v38i19.30167.\n\n[4] Mukhoti, Jishnu, Andreas Kirsch, Joost van Amersfoort, Philip HS Torr, and Yarin Gal. \"Deep deterministic uncertainty: A simple baseline.\" arXiv preprint arXiv:2102.11582 (2021).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their very thorough review and specific pointers for improvements. We are happy that the reviewer likes our rigorous evaluation and structure.\n\n**Weakness 1, Ablations on Diffusion**: We ablate $t$ and $\\alpha$ and the diffusion operator in Figure 1 (global response) and report performance on leave-out-classes, feature perturbations, and the homophily shift. Label Propagation is used per default in GEBM and GNNSafe while APPNP and GPN use symmetric diffusion. As expected, low $t$ and high $\\alpha$ aid node-level anomaly detection while the opposite holds for cluster and local shifts. Label-Propagation achieves satisfactory performance over the entire range of diffusion hyperparameters which justifes using it in our experiments. In particular, we did not tune any hyperparameters for good o.o.d.-detection. As stated in Appendix C.6, it does not bias the energy toward high-degree nodes which explains its advantages over the other diffusion types. It performs well over a broad range of hyperparameters making GEBM less sensitive to those.\n\n**Weakness 2, Evidence on GNN Overconfidence**: Previous work indeed finds that GNNs are under-confident on in-distribution test nodes. Our claim regards confidence under distribution shifts for which we are not aware of similar studies. We supply evidence for the overconfidence of GNNs in Figure 2 (global response): We find that both logit-based and unregularized energy-based confidence increase far from the training data. This confirms our theoretical analysis and justifies the use of distance-based regularizers. We are grateful for this pointer, and adapt our paper to explicitly disambiguate (L.157): “We remark that previous studies found GNNs to be underconfident on in-distribution data […] while the aforementioned issue of overconfidence arises from high distance to training data induced by a distribution shift (see Appendix …).”\n\n**Weakness 3, Advantage over Evidential Methods and Feature Collapse**: This is an interesting point: We do not claim energy to be the strictly superior choice to evidential methods. The ablation in Table 3 shows aggregating uncertainty at different structural scales is the driving factor in GEBM’s effectiveness: This enables it to outperform the evidential GPN. Transferring this paradigm to evidential methods is an interesting direction for future work which we added to L.332: “While GEBM enables robust evidential inference, future work may build upon its paradigm of aggregating different structural scales in the graph for evidential methods.”. One merit of EBMs is that they offer a theoretically sound and well-motivated way to combine uncertainty at different structural scales.\n\nRegarding feature collapse, we are not aware of any evidence for this phenomenon in GNNs. We expect that strong spectral normalization (small $\\sigma$) helps detect severe feature perturbations. We confirm this by ablating spectrally normalized GCNs in Table 1 (global response), but do not find improvements consistent over all shifts. It also decreases performance on other distribution shifts. We conjecture that either feature collapse is not as prevalent in GNNs (potentially due to smoothing) or that L2-based measures as in [1] are not suitable for the graph domain. While beyond the scope of our work, we believe this to be a daunting direction for future work and add to L.332: “Further development regarding the density-based regularizer, for example by studying the effects of feature collapse […], may improve the energy that GEBM is based on.”\n\n**Question 1, Data Augmentation:** We do not perform any data augmentation in our work.\n\n**Question 2, Precise Definition for x**: Thank you for the pointer, this was phrased imprecisely: The statement holds for any x almost surely, i.e. the set of x for which it does not hold (the kernel space of the affine function) has measure zero. We adapt our phrasing to “and any $x \\in R^d$ almost surely:”.\n\n**Question 3, Energy Weights**: We choose equal weights as we do not assume prior task-dependent knowledge about which structural scale is more important than others. We see tuning these weights on o.o.d. data as unrealistic since such data is unavailable in real scenarios. We believe that the effectiveness of even the most uninformed (uniform) choice underlines the merits of the paradigm behind GEBM. Practitioners can also adapt these weights to incorporate prior knowledge. If the reviewer thinks changing Equation (9) to account for weights is beneficial, we are happy to adapt the paper accordingly.\n\n**Question 4:** Thank you for pointing it out, we fix this by changing “GNNSafe → GCNSafe” and “Heat → GCN-HEAT”.\n\n**Question 5**: Our suite of distribution shifts is a superset of the GPN benchmark, with the most notable addition being a structural shift. The single-graph distribution shifts of GNNSafe fall into the same categories as our benchmark: First, they, too, use left-out-classes. Second, they interpolate between node features similar to our feature perturbations. While there is no guarantee this provides semantically meaningful features (especially for bag-of-word features), we use different noise distributions to control similarity to in-distribution data by matching its modality. Lastly, GNNSafe uses a fixed SBM to perturb the structure: Our homophily-based shift induces a more realistic structural shift as the o.o.d. nodes are drawn from real data. Our benchmark studies the same types of shifts as both related works pointed out by the reviewer. Should they believe that we miss a relevant setting, we are happy to include it in the updated paper as well.\n\nWe again want to thank the reviewer for the interesting and helpful discussion points that helped us to improve the manuscript.\n\n**References:**\n\n[1]: Mukhoti, Jishnu, Andreas Kirsch, Joost van Amersfoort, Philip HS Torr, and Yarin Gal. \"Deep deterministic uncertainty: A simple baseline.\" arXiv preprint arXiv:2102.11582 (2021).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3Q1diEQqx0", "reviewer": "Reviewer_Q6TT", "summary": "The paper defines an integrable (regularized) energy function to capture epistemic uncertainty via energy a pretrained model. The energy function is a function of the logits so the method is a post-hoc model agnostic. The authors define a diffusion-based hierarchical energy propagation (structure agnostic + local diffusion + group diffusion) which both leads to quantification of uncertainty in graphs, and an evidential model prediction.", "questions": "1. Building on weaknesses no. 1. (*W1*), do you assume some homophily property like $\\mathbb{P}_{v_i \\sim v_j}[y_i = y_j] \\ge p$? Is heterophily graph a theoretical limitation of your approach? If yes can you elaborate on the theoretical insight behind it? Note that I see the heterophily is mentioned as a limitation but mostly I can not find a sound explanation of why it is other than just leaving it as an assumption. \n\n2. In evaluation with feature perturbations, why didn't the authors use a random XOR shift instead of a total replacement of features? In that case, you have control over the magnitude shift; intuitively I expect the uncertainty to grow with a correlation to the perturbation probability, but here I can just see the endpoints of the experiment I just mentioned -- fully perturbed node and original node. In general, you can also define the feature shift by randomly selecting from the noise or the original features and controlling the randomness as the magnitude of the shift.\n\n3. In Fig. 2. (robust evidential inference), I can not understand why the result is non-trivial. If the graph is homophily, a simple diffusion over predicted probabilities with a strong coefficient can have a significant denoising effect in the prediction. This is especially in case the perturbation is sparse. What is the model evaluated in Fig. 2? Does it have a similar enhancement in robustness for models that already have a diffusion step like APPNP?", "limitations": "I think the limitations are mentioned clearly.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "I count the regularization of the energy function and the theory behind it as a strong point of the paper. \n\nAlso defining an energy-based model for graphs to address uncertainty is a strong starting point for uncertainty quantification on graphs which is really under-explored by the current time.\n\nI also see that the authors have provided a complete experimental setup (with some minor exceptions which I addressed in the weaknesses).\n\nIn total, I find this paper a strong paper, however I believe that it can be stronger with a better flow of the text, and more contribution in theory for the quality of the UQ.", "weaknesses": "1. The authors evaluate their method structurally (line 227) via leaving least homophilic nodes or low-page rank centrality as o.o.d. This seems like there is an implicit assumption of homophily in the graph which is not stated anywhere. In other words, I assumed that the propositions, or setup I should see an assumption of \"homophily $\\ge$ some constant on expectation...\". I see that they referred to this assumption in the limitations, but it is better to be mentioned somewhere in other sections as well.\n\n2. *Clearity:* However successful the method is, I do not see a clear intuition on why these three levels of propagation should be combined and why all are aggregated with weight = 1. For this, I expected an intuitive introductory experiment to clearly show what happens to a node before and after each certain diffusion. More importantly, I see theory to show that the regularization $\\pm$ diffusion is integrable, or is not infinity anywhere, but I did not find any theory behind why the approach leads to a good uncertainty quantification in the end; like comparing it with an oracle or finding bounds on the distance from unseen ground truth probability. I also did not see any synthetic experiment in this direction which might help a lot.\n\n3. *Experiments:* (1) I see that in scores like ECE and Brier score the method is not the best. Is there any intuitive explanation for why this is the case? I also strongly recommend the authors to mention that in the limitations of the paper. (2) I see the absence of study on models that have diffusion at the probability space; e.g. APPNP. If the GEBM can improve upon these methods then clearly there is some additional information passed in the energy domain.\n\n4. *Minor Typos:* (1) Line 237 the term \"fitted\" is used twice. More important (2), in Line 163 there should be \"for $\\boldsymbol{x} \\in {Q}_l$\" added somewhere to show that the definition of $f(\\boldsymbol{x})$ is limited to that polytope.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their in-depth review and interesting questions and are happy to see that they find our paper strong.\n\n**Weakness 1 & Question 1, Homophily Assumption**: We assume homophily throughout our work. We agree that is benificial to explicitly state this assumption early on and add to our background section, L.66: “Our work is concerned with homophilic graphs, i.e. edges are predominantly present between nodes of the same class.”\n\nAll diffusion processes we study (Appendix C.6) rely on homophily. Formally quantifying the effectiveness on these operators for different problems with respect to an explicit homophily value is difficult. There are, however, empirical studies on the performance of GNNs that rely on these diffusion processes [2]. We have addressed this in L.197: “Intuitively, this is a smoothing process that relies on the in-distribution data to be homophilic. In case of non-smooth (heterophilic) data, the energy will be high for in-distribution data which is undesired behavior.”. We believe that formally studying the homophily assumption for graph diffusion is future work.\n\n**Weakness 2, Energy Types and Weights:** Uniform weights do not assume prior knowledge about the problem. We believe that tuning these weights on out-of-distribution data may be unrealistic, as such data is unavailable in real scenarios. The effectiveness of even the most uninformed (uniform) choice shows GEBM’s merits on different distribution shifts without any tuning.\n\nWe agree that the paper benefits from a motivating experiment and intuition for each energy type.\n\n**1. Independent Energy** uses no diffusion and therefore targets anomalies at individual nodes. Formally, this is motivated as diffusing energy from non-anomalous low-energy neighbors would decrease the energy of an anomalous node.\n\n**2. Group Energy** uses diffusion to increase the energy of a node if anomalous high-energy neighbors are present and is fit to detect cluster-level anomalies. A similar formal argument can be made here.\n\n**3. Local Energy** - in contrast to Group Energy - does not aggregate class-wise energy before the diffusion. Therefore, it aggregates local average per-class evidence for a node and can identify evidence conflicts arising from structural anomalies.\n\nWe devise synthetic experiments that support the intuition behind all three energy types in Figure 4 (global response): First, we induce artificial structural anomalies into real data by iteratively introducing a left-out-class (anomalous) cluster node-by-node. Group Energy shows the highest increase. Second, we induce per-node anomalies into a synthetic SBM graph and increase the magnitude, and observe Independent Energy to be the most sensitive. Lastly, we sample SBMs of varying heterophily (measured as the log ratio of inter-class to intra-class edge probabilities) and confirm that only Local Energy detects the structural anomaly. We propose to add this synthetic experiment along with the intuitive explanation to Section 4 of the updated paper. We would be happy to hear the reviewer’s opinion on that.\n\n**Weakness 3, Calibration and APPNP**: GEBM is a post-hoc estimator and does not affect the calibration of the backbone classifier and we report its ECE and Brier score for completeness only. We will explicitly highlight this more clearly in the limitations section, L.330: “The GCN backbone used in this work does not consistently achieve the strongest performance in both tasks.”. We also want to point out the calibration could easily be improved e.g. with temperature scaling [1]. We view this as outside the scope of our work as it is unrelated to GEBM’s epistemic uncertainty.\n\nWe also add APPNP to the ablation of model backbones (Table 1, global response). GEBM is the only method that consistently detects all families of distribution shifts. This confirms that multi-scale uncertainty also improves on models based on just diffusion. We are thankful for that pointer as it makes a strong point in favor of GEBM.\n\n**Weakness 4, Typos:** Thanks, we fixed that typo.\n\n**Question 2, XOR Feature Shift**: Thanks for the nice proposal: We experiment with this XOR shift (Figure 5, global response) and observe that GEBM is the only method that reliably provides good performance with an increasing fraction of perturbed features.\n\n**Question 3, Advantage of Evidential Inference**: The model evaluated in Figure 3 is a GCN. We agree that diffusion in general can denoise anomalous predictions. The key advantage of evidential methods is that with increasing anomaly severity the evidence approaches zero while, for example, logits are provably likely to approach infinity. Therefore, a fixed amount of denoising steps is sufficient to recover from arbitrarily severe corruption while non-evidential methods require more diffusion iterations the stronger the anomaly is. As requested, we can provide evidence for this through an experiment with APPNP (Figure 4, global response). The evidential interpretation of GEBM on an APPNP is significantly more robust than APPNP alone. We add to the paper, L.289: “The advantage of this evidential perspective is that the evidence approaches zero for increasingly severe distribution shifts. Therefore, a fixed number of diffusion steps effectively counteracts the influence of anomalous neighbors when making predictions for a node.”\n\nWe are very grateful for the pointers. We believe that the changes to our manuscript and additional experiments help to clarify and consolidate the core assumptions and results of our work.\n\n**References:**\n\n[1]: Chuan Guo, Geoff Pleiss, Yu Sun, Kilian Q. Weinberger. “On Calibration of Modern Neural Networks”. International Conference on Machine Learning (ICML) 2017.\n\n[2]: Palowitch, John, et al. \"Graphworld: Fake graphs bring real insights for gnns.\" *Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining*. 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "y7oRmMZSUu", "reviewer": "Reviewer_VZ1v", "summary": "This paper explores the challenges associated with quantifying uncertainty in Graph Neural Networks (GNNs), particularly in domains involving interconnected data such as graphs. The authors propose a novel method called GEBM, which aggregates energy at various structural levels. This approach enhances predictive robustness and improves performance in distinguishing between in-distribution and out-of-distribution data across different anomaly types and datasets.", "questions": "1. From the equations presented, it appears that this method is not limited to graphs. Are there potential challenges in applying this method to tasks beyond graph-related domains?", "limitations": "The limitations are discussed in this paper. GEBM is a post hoc epistemic estimator; it does not improve aleatoric uncertainty or its calibration.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper introduces a novel method based on energy considerations. The authors provide sufficient background and related work on the topic, making the main idea accessible even to readers unfamiliar with the subject.\n2. Theoretical proofs and theorems are adequately presented to support the proposed methodology. The paper includes comprehensive experimental settings and achieves state-of-the-art performance across various tasks.\n3. The paper is well-organized and well-written, enhancing clarity and readability.", "weaknesses": "1. The experiments predominantly focus on node-level tasks. It would be beneficial to explore how the proposed method performs on graph-level tasks as well.\n2. The paper lacks an ablation study specifically on different types of energy used in the GEBM model. Understanding which energy components are critical remains unclear.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their thorough review and are happy that they like our paper overall.\n\n**Weakness 1, Graph-Level Tasks**: Indeed, transferring GEBM to graph-level tasks is an interesting avenue: The energies proposed by our framework are designed with a node-level objective in mind. If suitable knowledge of how graph-level properties relate to individual nodes, GEBM would certainly offer a powerful framework to combine them effectively. Devising an aggregation scheme would be a daunting problem that is not straightforward to solve, in general, and beyond the scope of our work. We will mention that direction in the future work, L.332: “Future work may build upon the framework of aggregating energies at different scales for graph-level tasks by defining and adequately aggregating node-level energy at different scales into a per-graph measure.”\n\n**Weakness 2, Ablation of Energy Types**: There seems to be a misunderstanding regarding this ablation study. We provide ablations regarding the different energy types (local, group, and independent) for all distribution shifts and datasets in Tables 3 and 26 of our paper. We will highlight that this is an ablation of the individual energies more clearly by adding the following to our paper, L297: “Each of the former corresponds to only using one of the energies that the GEBM framework is composed of.” We hope that this sufficiently avoids this potential misunderstanding for future readers.\n\n**Question 1, Non-Graph Domains**: It is true that energy-based models are not limited to the graph domain, in general, and have also successfully been applied to i.i.d. domains [1]. As we showcase in Section 5.3 (in particular Table 3) of our paper, the effectiveness of GEBM primarily comes from aggregating energies at different structural scales in the graph. This is inherently bound to graph problems. However, we agree with the idea of the reviewer that also other domains may benefit from the paradigm of composite energy at different resolutions: One could, for example in the domain of time series, define energies at different frequencies and aggregate them into a single measure similar to GEBM. Also, density-based regularisation is likely to improve EBM-based uncertainty in other domains. The key challenge would be to develop energies that arecapable of capturing anomalies of different types. How these terms would look most likely depends on the problem. We will acknowledge this nice idea in the future work section of our paper, L.332: “The effectiveness of GEBM also motivates the development of aggregate energy-based epistemic uncertainty for other domains. We leave transferring our approach to non-graph problems for future work.”\n\nOverall, we want to thank the reviewer for their review. We hope that we could clarify the misunderstanding regarding the ablation asked in Weakness 2 and appropriately address how GEBM could be extended to non-node-level or even non-graph-level tasks in the future work section of our revised manuscript.\n\nReferences:\n\n[1]: Liu, Weitang, et al. \"Energy-based out-of-distribution detection.\" *Advances in neural information processing systems*33 (2020): 21464-21475.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all the reviewers for the time spent providing valuable feedback. We added all experiments and ablations provided in the pdf to the final version of our manuscript. We briefly want to summarise the most relevant additions to the paper. In the individual rebuttals, we propose word-by-word changes for these points.

- We add APPNP as another GNN backbone to our ablation study to show that the success of GEBM is rooted in describing uncertainty at different structural scales rather than simply diffusing the corresponding scores. On APPNP, GEBM outperforms existing methods and enables more robust evidential inference.
- We ablate different diffusion modules and hyperparameters to justify the choice we made for GEBM and find that GEBM is highly effective over a broad range of settings.
- We justify each energy type and provide an introductory synthetic experiment to ease an intuitive interpretation.
- We supplement additional evidence on the over-confidence of GNNs, distinguish it clearly from under-confidence on-distribution data, and elaborate on the role and empirical relevance of feature collapse for these models.
- We provide clear directions for future work that extends to graph-level and non-graph-level domains.

Overall, we are very happy with the constructive input and believe that based on the reviews we were able to further solidify our approach from an intuitive, theoretical, and empirical perspective.

Meta-review: The paper proposes a simple and effective post hoc method applicable to any pre-trained GNN that is sensitive to various distribution shifts. The novelty of the proposed method has been recognized by all the reviewers. The paper also provides strong results on datasets with various distribution shifts. The results show the proposed method could improve the generalization performance of existing backbones. 

All the concerns from reviewers have been well-solved from the rebuttal. All reviewers support to accept the paper.
Decision: Accept (spotlight)