Self-Consistency Improves the Trustworthiness of Self-Interpretable GNNs
Authors: Wenxin Tai
Ting Zhong
Goce Trajcevski
Fan Zhou
Venue: ICLR 2026 Poster (2026)
Keywords: Self-interpretble GNNs; Trustworthy; Consistency; Faithfulness
Primary area: learning on graphs and other geometries & topologies
TL;DR: We identify the mismatch between SI-GNN training and faithfulness evaluation, show its connection to self-consistency, and propose a simple SC loss that consistently improves explanation quality without architectural changes.

Abstract: Graph Neural Networks (GNNs) achieve strong predictive performance but offer limited transparency in their decision-making. Self-Interpretable GNNs (SI-GNNs) address this by generating built-in explanations, yet their training objectives are misaligned with evaluation criteria such as faithfulness. This raises two key questions: (i) can faithfulness be explicitly optimized during training, and (ii) does such optimization truly improve explanation quality? We show that faithfulness is intrinsically tied to explanation self-consistency and can therefore be optimized directly. Empirical analysis further reveals that self-inconsistency predominantly occurs on unimportant features, linking it to redundancy-driven explanation inconsistency observed in recent work and suggesting untapped potential for improving explanation quality. Building on these insights, we introduce a simple, model-agnostic self-consistency (SC) fine-tuning strategy. Without changing model architectures, SC consistently improves explanation quality across multiple dimensions and benchmarks, offering an effective and scalable pathway to more trustworthy GNN explanations. Our code is publicly available at \url{https://github.com/ICDM-UESTC/SelfConsistencyXGNN}.

Published as a conference paper at ICLR 2026
SELF-CONSISTENCY IMPROVES THE TRUSTWORTHINESS OF SELF-INTERPRETABLE GNNs
Wenxin Tail, Ting Zhong', Goce Trajcevski Fan Zhou it f University of Electronic Science and Technology of China
$ Key Laboratory of Digital Media Technology in Sichuan Province
$ lowa State University
wxtai look. com, zhongting@uestc.. edu gocet25eiastate edu, fan. www.@uestc.edu.coo
ABSTRACT
Graph Neural Networks (GNNs) achieve strong predictive performance but of- fer limited transparency their decision-making. Self-Interpretable GNNs (SI- GNNs) address this by generating built-in explanations, yet their training ob- jectives are misaligned with evaluation criteria such as faithfulness. This raises key questions:
(i) can faithfulness be explicitly optimized during training, and (ii) does such optimization truly improve explanation quality? We show that faithfulness is intrinsically tied explanation self-consistency and can therefore be optimized directly. Empirical analysis further reveals that self-inconsistency predominantly occurs on unimportant features, linking it to redundancy-drie drivven explanation inconsistency observed in recent work and suggesting untapped po- tential for improving explanation quality. Building on these insights, we intro- duce simple, model-agnostic self-consistency (SC) fine-tuming strategy. With- out changing model architectures, SC consistently improves explanation quality across multiple dimensions and benchmarks. offering effective and scalable pathway to more trustworthy GNN explanations. Our code is publicly available at https::/ thub. https:/ggg....c.cooccoo//oSSSS// fCons stencyXGNN
INTRODUCTION
Graph Neural Networks (GNNs) have achieved remarkable success across a wide range of tasks, from social network analysis at.. 2022a) molecular property prediction (Wang al. 2022).
Despite their effectiveness, however, GNNs are often criticized as black boxes, which hinders their adoption in high-stakes and scientific domains (Pfeifer al.) 2022 Rajput Singh 2022
Warmsley et al. 2022) To address this issue, Self-Interpretable GNNs (SI-GNNs) (Velickovic et al. 2018 Lin
al. 2020 Sui et al. 2022 Miao al. 2022) have been proposed, which jointly learn predictions
and explanations in an end-to-end manner, making interpretability an intrinsic part of the model. This design not only makes cerision-making more transparent but also yields structural insights that can facilitate causal discovery and scientific understanding.
Evaluating the quality of GNN explanations typically follows two paradigms (Agarwal al. 202.3
The first is explanation accuracy (Ying et al., 2019). which compares model-generatee subgraphs against annotated ground-truth explanations. The second faithfulness (Yuan et al.) 2021) which evaluates whether the explanation correctly highlights the input elements that are truly relevant to the model's prediction (Azzolin et al. 2025a). Because faithfulness does not depend on ground-truth annotations, it can be applied more broadly across datasets and tasks, and has therefore become widely adopted evaluation metric. However, there remains a mismatch between what we optimize and what we evaluate: SI-GNNs are trained with cross-entropy plus conciseness regularization (Tai et al. 2025), yet they are evaluated by faithfulness.
"Corresponding author.

---

Published as a conference paper at ICLR 2026
x10-1
4.5
3.0
DIFF
1.5
0.0
UNIMP IMP SMGNN(A)
UNIMP IMP GSAT(A)
UNIMP IMP SMGNN(B)
UNIMP IMP GSAT(B)
UNIMP IMP SMGNN(C)
UNIMP IMP GSATIC)
UNIMP IMP SMGNN(D)
UNIMP IMP GSAT(D)
Figure 1: Self-inconsistency measured the L1 difference (DIFF) between two explanations. Re- sults are shown for SMGNN (Azzolin et al. 2025b) and GSAT (Miao et al.] 2022) across four benchmark datasets: A BA-2MOTIFS, B MR, C= BENZENE, D MUTAGENICITY Within each panel, UNIMP and IMP denote unimportant and important features, respectively.
This mismatch naturally raises two questions:
(i) Can faithfillness, conceptual property, be optimized during training? (ii) Even if feasible, necessary- does truly improve explanation quality?
To answer first question, a common evaluation of faithfulness (Amara et al. 2022 is to feed identified explanation graph subset back into the model: if the prediction remains unchanged, the explanation is deemed faithful. This procedure implicitly relies on the SI-GNN being self-consistent. If the explanation is truly decisive, the SI-GNN should consistently highlight the same graph sub- set upon re-use. Consistent explanations in turn lead to consistent predictions, thereby satisfying the faithfulness criterion. This perspective suggests that faithfulness can indeed be optimized by introducing an alignment loss that enforces agreement between successive explanations.
Answering the second question is more subtle. While faithfulness can be optimized, it is not obvious whether doing so truly improves explanation quality. To investigate, we conduct an empirical study and find that, without faithfulness-orienteed training. the first-pass and second-pass explanations of an SI-GNN can differ significantly. Using benchmark datasets with ground -oound-truth explanations, we further observe that self-inconsistency primarily arises from instability on features labeled unim- portant, while important features remain stable (Figure [T. This observation resonates with recent findings (Tai et al., 2025) on explanation inconsistency across repeated training runs of the same SI-GNN, where inconsistency was traced to cetennddnny-xpplliiin
irresponsibly assigning high importance to unimportant features when budget allows. That work further showed that addressing redundancy improves explanation quality. Since study shows that self-inconsistency also con- centrates on unimportant features, it suggests potential connection: addressing self-inee ouffinccossitency may address redundancy well, thereby improving explanation quality in similar manner.
In this work, we propose to incorporate self-consistency into the training objective of SI-GNNs. On top of the standard objectives, add ceff-consistency (SC) loss that minimizes discrepancies between successive GNN explanations (Section B.T). This simple yet general addition requires no modifications the model architecture, and can be seamlessly integrated into existing SI-GNNs. We show that SC training drives feature importance scores toward a small set of near-fixed lev- els (Section 3.22 while conciseness regularization governs which levels selected (Section 3.3) This interaction reshapes the explainer's 's behavior, resulting in more responsible and stable impor- tance
assignments, assignments. Extensive experiments across representative SI-GNNs and benchmark datasets
demonstrate that SC consistently improves the trustworthiness of GNN explanations in terms of consistency, accuracy, faithfulness, and informativeness.
2
PRELIMINARIES
2.1
SELF-INTERPRETABLE GNNS (SI-GNNs)
We first formalize the task and model setup. A graph G = (V, E) consists of a set of nodes and a set of edges &. In graph classification, GNN can be decomposed into GNN encoder h2 G - Bd that maps G to a representation, and a classifier hy : Rd - R that produces the prediction, giving he lz. A SI-GNN adds explainer hG,, yielding f he hzz hG,, where hG, selects
2

---

Published as a conference paper at ICLR 2026
graph subset G3 S G the explanation. Following prior work (Tai et al. 2025 focus structural features (edges) in the instance-level setting.
Definition 1 (Learning Objective). Given (G, Y). the objective is to select subset G2 of G that maximizes the mutual information (MI) with the label while promoting explanation conciseness:
max TGss; Y) 3. R(G.), G.SG
where ((:-)) denotes MI, R(G,) 2)) is an conciseness regularizer, and controls its strength. The form of R(G,) differs across SI-GNNs, and we review the most common ones below.
Existing SI-GNNs can be broadly categorized into four groups attups--ateentoo-aaseed causal-based size- constrained, and Ml-coostraineed-deeeenddit on their subset selection strategy (Tai al. 2025).
(1) Attention-based methods leverage attention mechanisms (Vaswani et al., 2017) to assign im- portance scores directly, without explicit regularization to enforce explanation conciseness. A rep- resentative example is GAT (Velickovic et al. 2018), which directly assigns importance scores to edges via attention mechanisms. The model is trained solely with classification loss:
LGE = CC,
where LCe is cross-entropy loss.
(2) Causal-based methods employ causal inference techniques (Pearl 2014) to identify causal patterns beyond spuriosity-prone statistical correlations. A representative example is CAL (Sui
et al. 2022), which employs disentanglement and intervention to identify causal edges:
LGE CCE(Y, Yl.)) + B (KI. (IIPo(Y|),)]]00 (G))) + LCE (cE(X,Y"(G,UU"")
where and y are pre-defined hyperparameters, G, G G, the complement of G., and G², represents the result of intervening on G, (e.g.. replacing its latent representations with those from other batch samples). QQY) is often set to a uniform distribution.
(3) Size-constrained methods enforce explanation conciseness by penalizing the size or sparsity of the selected subset. representative example SMGNN (Azzolin al. (2025)), which adds sparsity loss term to encourage concise and human-understandable explanations:
G.
CGE = LcE((,
(4)
G
where |G,| and GG| denote the number of edges in the selected subset and the original graph.
(4) MI-contraineed methods minimize the MI between the selected subset and the original graph, providing an information theoretic route to enforce explanation conciseness. A representative ex- ample is GSAT (Miao al. 2022), which minimizes the MI between G, and G via KL divergence:
LGE = (Y, Y G..) + 3 DRL (Po (G. (QQ())),
(5)
where Q(G.) is Bernoulli distribution. Other SI-GNNs are discussed in Section
2.2 EXPLANATION REDUNDANCY
Recent work Tai al. 2025 has revealed that explanations produced by SI-GNNs are often incon- sistent: when the same model is retrained with different random seeds, the explanations for a given instance may vary substantially. Through analysis, this inconsistency was attributed to explanation redundancy, as current SI-GNNs impose insufficient conciseness constraints, leaving excess budget that allows high importance scores to be irresponsibly assigned to unimportant edges. For instance, in GAT no explicit regularization is enforced, making redundancy unavoidable in principle; CAL avoids false negatives (missing important edges) but does not penalize false positives (including unimportant edges), thereby failing to discourage redundancy; SMGNN controls subset size via sparsity coefficient B3 but this typically set small to balance accuracy and conciseness, weaken- ing the constraint and allowing redundancy persist; and GSAT faces the same issue- issue-although it theoretically minimizes MI, in practice its authors report that retaining 50-80% edges yields the best accuracy, which is far too many eliminate redundancy.
3

---

Published as a conference paper at ICLR 2026
SC-Driven SI-GNNs
Shared Parameters
Tuning
Freezing
GNN
Explainer
GNN
Classifier
LCE
Lse
GNN
Explainer
Co YIG
(
Figure 2: Overview of the SC training framework. A SI-GNN is first trained with the standard loss (Section 2.1]. The encoder is then frozen, and the explainer and classifier are fine-tuned with
an additional alignment loss that enforces the first-pass explanation G! (from G) to match the second-pas. explanation G! (from )!!).
This phenomenon can be formalized under a unified budget-constrained formulation:
max I(G,;
G,EG. JG.ISM
where M denotes the maximum allowed subset size (implicitly determined by R(Gx)). Define G as the ground-truth explanation subset. When M 2 |G there exist Σ (lG-G; distinct subsets that achieve the same maximum MI. This multiplicity of valid subsets makes possible for unimportant edges to be included without affecting prediction accuracy, thereby formalizing why insufficient conciseness regularization naturally gives rise redundancy in existing SI-GNNs.
While Tai et al. (2025) has focused on cross-model explanation inconsistency, our study reveals self-inconsistency within single SI-GNN-which also occurs on unimportant edges. This motivates us to explicitly enforce self-consistency during training reduce explanation redundancy and thereby improve explanation quality.
related form of instabiiity-explaaatiee
METHOD
3.1
SELF-CONSISTENCC TRAINING FRAMEWORK
Our framework consists of two steps. We begin by revisiting the workflow of a standard SI-GNN, which forms the basis for the first step. GNN encoder first updates node representations:
V= = GNN(G).
An explainer, typically multilayer perceptron (MLP), then predicts edge importance scores:
Wij vM)P((v::vs)),
Ogj = awwy),
where [ denotes vector concatenation and denotes the sigmoid function. Each edge eij is included in the explanation subset G₂ with probability Orj. and we adopt the Gumbel-Sigmoid trick (Jang et al.) (201)) enable differentiable sampling. The selected subset G, then processed by the same GNN encoder and pooled into a graph-level embedding, which is finally passed to an MLP (classifier) for prediction. Vanilla SI-GNNs optimized using a combination of cross- entropy and conciseness regularization (Section 2.1 see Section B for implementation details.
Step 1:
Pretraining. We first train the SI-GNN with the standard objective until convergence, and then freeze the GNN encoder. This ensures that representation learning is not disrupted by the additional SC loss introduced later, so that SC loss only influences the explainer's behavior.
Step 2: Seff-Consistency Fine-Tuning Given graph G, the explainer produces an explanation
G!!)) We then feed ( back into the model, obtaining a second explanation G((2) If (()) truly
captures the decisive structure, the explainer should reproduce it consistently (i.e., G(1) = (( To
enforce this, we introduce SC loss minimize the discrepancy between successive explanations:
Lsc
IG(!) G(
4

---

Published as a conference paper at ICLR 2026
The final training objective is the combination of the standard SI-GNN loss and Lsc
CFT = LGE + 7) Lsc.
(10)
where yj a pre-defined hyperparameter. The proposed two-step SC training framework (illustrated in Figure 2) is simple yet general: it requires modification to the model architecture, and can be seamlessly applied to wide range of existing representative SI-GNNs.
Define the second-pase mapping each edge score as:
T(a) = (g(a)), a a [0,
(11)
where g(-) denotes the explainer's pre-activation on the second pass (obtained through the GNN encoder and explainer), and a the sigmoid function. Intuitively, enforcing explanation self- consistency is equivalent to requiring T(a) & i.e., edge scores remain stable across passes. We next formalize this idea through the notion near fixed levels.
3.2 NEAR FIXED LEVELS INDUCED BY
We formalize the concentration to few levels without assuming the specific form of g(-).
Definition 2 (Near fixed level). Given tolerance (0, 1), [0, 1] is e-near fixed level of T (T(a) - al S E. We denote the set by S. {a: [T(aa - al S E}.
Window condition. For any candidate level (0, 1), the near fixed condition is equivalent to:
- = g(a") logit(a" - =), logit(a" + +e)].
(12)
P_ denotes the logit function. Thus, achieving self-consistency requires the where (p) In 1-p explainer's pre-activation g(a") to fall inside logit window around a"
Window size. The width of this admissible window is:
2s (a e) = logii + - logit =) a²(1 at)'
(s small).
(13)
For interior points a* E (0, 1), the window is finite, and hitting such levels depends on model archi- tecture and optimization trajectory. When + 0 or into a one-sided threshold:
+ 1, the two-sided window degenerates
near 1: 2(a")2((s). near 0: g(a") < -T(E),
(14)
where 7(E) In 1=t These half-space conditions correspond the flat saturation regions of the sigmoid, making 0/1 relatively easier to achieve than interior levels.
Below, we analyze from gradient perspective how, with adequately strong conciseness regulariza- tion (CR), SC can improve explanation quality.
3.3 INTERACTION WITH CONCISENESS REGULARIZZATION
Consider the joint training loss:
LeT LCB + R(G.) +1 Lsc.
(15)
Let oij denote the importance score of edge eij. The gradient w.r.t. ai is computed as:
Oc dayj
ÔLCE
+ 3. aR(G.)
DLsc
+
(16)
Dau
Dais
dai
classification
stability
The three gradient components play different roles: the first term pushes important edges upward (toward 1) because they needed for prediction; the second term depends the specific form of CR: SMGNN encourages sparsity (asj - 0), whereas GSAT encourages independence (a;j + 0.5); the third term stabilizes scores by pulling them toward near-fixed levels.
5

---

Published as a conference paper at ICLR 2026
(a) SMGNN (B=5e-2) (b) SMGNN (3=5c-1)
(c) GSAT (3=5e-2)
(d) GSAT (3=5e-1)
(e) SMGNN+SC (B=5c-2) (f) SMGNN+SC (8=5c-1) (g) GSAT+SC (B=5c-2) (h) GSAT+SC (8=5c-
Figure 3: Distributions of edge scores assigned to unimportant edges (defined by the dataset ground truth) BA-2MOTIFS dataset using SMGNN and GSAT. Each subplot shows the distribution of edge scores for unimportant edges over training epochs (vertical axis).
Table 1: Experimental results SMGNN and GSAT. Bold numbers indicate improvements over the raw baselines. Best, and second- results are highlighted in different colors.
BA-2MOTIES
3MR
BENZENE
MUTAGENICTTY
1. SHD (%)
1 AUC
SHD (%)
AUC (%)
SHD (%)
A (9)
SHD (%)
ATC
SMGNN
10.44:24.41
99.32:10.36
9.44:2.24
97.00:00.77
16.06:16.57
84.38=2.71
12.65:1..18
98. 18.3290..9
SMGNN-EE
4.99=1.88
99.59=t0116
5.20=0.72
98.25=0.21
8.55=2.79
91.38=0.%6
6.21=1.20
96.96=0.16
SMGNN
3.48-2.200
99.87--0.05
5,47--3.39
08.39-0.88
7.19-2.62
1.56
2.51-1.02
98.30 0.554
SMGNN+SC+EE
1.52
99.90.10.00
3.05
99.25 0.02
4.14.0.82
92.20
0.34
98.96 0.11
GSAT
4,58+1181
98. 08.64
12.35:4.555
98, 08.38=00.11
6.93=2.75
00.66=0.899
10.08-24.558 58
90
GSAT-FF
2,10:-0.67
98.60+0.16
5.79-1100
99.16-00.03
3,25 1.09
92.18-0055
4.70-1.60
99.36 0.05
GSAT+SC
2.73.1002
99.30110.12
6.67.13.73
9..29
2.32..0.62
92.80
2.38 0.84
99.38
OSAT+SC+EE
1.9::0.52
99.35:10.03
4.00:10.38
99.39:0.011
1.14:0026
93,53220.11
1.06:0.42
0.00
(S)
1 ACC (%)
(%)
1 ACC (%)
J FID (5)
1 ACC (9)
J FID
1 ACC (%)
SMGNN
95.50 12.52
1.90.11.01
97.30
2.07.10.62
11.12.10.62
1.72.11.09
89.53
SMGNN-EE
100.00:00000
98.25:00.21
92.16:10.24
00.36:10.0
SMGNN-SC
0.00.10.00
99.65
0.97
0.61
SMGNN+SC-EIE
100.00:00000
99.65:0000
93.24::0.19
92,11
0.00+00.00
100.00-00.00
0,90++0228
98.55+0050
1.80-0.84
91.48++0.87
1,11-0.61
92.43-+
GSAT+EE GSAT+SC GSAT+SC+EE
100.00 0.00
99.15 00.24
92.17
93.00 0..4
0.00:0.00
100.00=0.000
0.31:10.29
09.55:20.16
0.74=0.14
92.31:0.32
0.17=0.23
03.48
100.00=00000
99.65=0.00
92.61=0.18
93.76:20.77
As result, when 3 is weak, CR barely influences edge scores. Important edges are driven by the classification loss, while unimportant edges can freely occupy any feasible near-fixed level under SC; When 3 moderately strong. CR actively suppresses unimportant edges: under SMGNN they are pushed toward 0, while under GSAT they converge toward 0.5. Important edges remain near 1, as the classification term dominates their gradients. This regime achieves the best explanation quality, since CR and SC jointly drive unimportant edges toward low and stable importance scores (Figure 3) When 3 is strong, CR overwhelms classification: in SMGNN both important and unimportant edges collapse near 0. while in GSAT they collapse near 0.5. Consequently, explanation quality degrades, as important edges can no longer be distinguished from unimportant ones (Figure 14 in Section D.I.
4 EXPERIMENTS
Datasets. Following prior work (Tai et al.) 2025), we evaluate on four benchmark datasets: the syn- thetic dataset BA-2MOTIFS (Luo et al. (2000) and three real-world molecular datasets 3MR (Rao) et al. 2022), BENZENE Morris et al. (2020), and MUTAGENICITY (Morris et al., 2020).
Baselines. We implement SC four representative SI-GNN backbones mentioned in Section In addition, we compare against Explanation Ensemble (EE) (Tai et al. 2025), a post-hoc strategy designed to mitigate redundancy and improve explanation quality.

---

Published as a conference paper at ICLR 2026
1.0
1.0
1.0
Edge Scare
Edge Score
0.5
0.5
8.5
Edge
Edge
0.0
0.0
50
100
0
100
200
300
600
1200
0
100
200
300
Instance Indcx
Instance index
Instance index
Instance Inocx
BA-2MOTIFF
(b) MR
(c) BENZENE
(d) MUTAG
1.0
1.0
1.0
Edge Score
Eage: Scare
0.5
0.5
0.5
0.5
Edge
Edge
0.0
50
100
0.0
0
100
200
300
0.0
600
1200
0.0
0
100
200
300
Instance index
Instance index
Instance index
Instance index
(e) BA-2MOTIFS (+SC)
(I) MR (+SC)
BENZENE (+SC)
(h) MUTAG (+SC)
Figure 4: Distributions of edge scores assigned to important and unimportant edges (defined by the dataset ground truth) across datasets using SMGNN. The first row shows vanilla S1-GNNs, while the second row shows models trained with SC. Results for GSAT are provided in Figure 15
(a) BA BA-2MISS
(b) MR
(c) BENZENE
(d) MUTAG
02
6/4
22
92
62
64
O.
65
34
06
(c) BA- 2MOTISS (+SC)
(f) MR (+SC)
(g) BENZENE (4SC)
(h) MUTAG (+SC)
Figure 5: Unimportant edge weights across 10 SMGNN runs (GSAT results in Figure 16..
Input
Ground Truth
+SC 1: AUC 0.97
+SC 2: AUC 0.97
+SC 3: AUC 0.97
+5C 4: AUC 1.00
+SC S: AUC 0.96
Figure 6: Explanations generated by five independently trained models (SMGNN vs. SMGNN+SC).
Metrics. We adopt four widely used metrics. Soft Structural Hamming Distance (SHD) measures explanation consistency, ROC-AUC (AUC) quantifies explanation accuracy, classification Accuracy (ACC) reflects informativeness for downstream prediction, and Fidelity- (FID) evaluates faithful- ness. Further details about experimental settings are provided Section C
4.1 EFFECTIVENESS ON CR-BASED SI-GNNS
As discussed in Section 3.3 the effectiveness of SC depends appropriate strength of CR. We therefore first evaluate SC on two representative SI-GNNs with CR, SMGNN and GSAT. The results summarized in Table [ We draw three key observations: (i) SC improves explanation quality

---

Published as a conference paper at ICLR 2026
BA 2MOTIFS (SMGNN)
MR (SMGNN)
BENZENE (SMGNN)
MUTAG (SMGNN)
Raw: G14)
Raw: 6
+SC: G
+SC: 6!a)
Figure 7: PCA visualization of explanation embeddings. Each line connects two successive repre- sentations of the same instance; shorter lines indicate better stability (GSAT results in Figure l7
Table 2: Pairwise similarity between embeddings of G and G.a²) across 10 runs. Higher cosine (COS) similarity and lower LI distance indicate better ceff-consistency.
BA BMOTIFS
MR
BENZENE
MUTAGENICITY
Mothod
T COS (%)
J (S) (5))
(%)
L (%)
COS (%)
L (8)
COS (1)
L (%)
SMGNN SMGNN-SC
99.68=0.38
16.87-8.30 30
95.81=6900
51.01+32.81
99.11=1.63
33.23-+-00766
98.79-2.56
48. 18:445.077
99.98
5.69
99.92 0.52
18.82
99.96 0.14
5.93
GSAT
98.87=1.56
23.82=12.05
95.57=10.52
12.35:12..99
19.15=1.63
29.99: 17.60
99 00.66-00665
16.84=16.43
GSAT
98.71+2.18
14.31
98.65
6.30-19.94
99.66-11.3 1.23
9,59-11.66
99.94-00.19
5.71+66.99
Table Experimental results on GAT and CAL. Bold numbers indicate improvements over the raw baselines. Best and second-best results are highlighted in different colors.
BA-2MOTIFS
MR
BENZENE
MUTAGENICTTY
Method
1 SHD (%)
1 AUC (%)
SHD (%)
AUC (%)
SHD (%)
1 (UC
J. SHD (%)
AUC (%)
GAT
10.41:14.37
99.31-0..35
14.04:2.005
97.25-0067
14.28-1.776
83.5142.24
11.85
91.71:16.00
GAT+SC
16.63-115,21
99.14:22.15
8.30.13.31
97.66-11000 1.00
19.43:19.4
85.1315.00
0.04
81.79112.31
GAT+CR+SC
3,48:22.20
99.87=0005
5,47:33.99
98.39=0.88
7.19=2.62
90,07:1..6
2.51
98,30:00.54
CAL
11.40:14.04
98.64
14.51:14.33
96.25 1.59
21.25.18.91
77.64.1..05
21.82111155
96.39.1..39
CAL+SC
17.54:15.31
98.80-2.41
9.08.13.54
97.23=0.49
11.32:17.00
88.74.13.06
0.0110001
96.52:12.07
CAL+CR+SC
1.90:11544
98.90=2.44
7.92::2.40
97.64=0.76
6.25=2.07
89.87:40.88
2.78:11552
97.93:20.46
(9)
ACC
FID
ACC (96)
1 FID (56)
1 (C)
(I))
ACC (98)
GAT
2.50:17.50
97.10
2.63.1.933
96.5411.46
2.06..065
91.00.0.66
0.3710..3
92.97.10.78
GAT+SC
0.00::0.00
100.00:20.00
0.10:::.22
0955
0.82:0061
92.46:10.32
0.00:0.00
93.65:0.77
GAT+CR+SC
0.00=:0.00
100.00=:0.00
0.00::0.00
99.65=0.00
0.97=0.59
92.57=t0.44
0.61+0.54
91.18+1.07
CAL
17.10:117.21
01.901111.6
8.17112.52
94,22-2.11
4.97
84.3115.66
2.0611114
11.22:11.37
CAL+SC
0.00:20.00
100.00:10.00
1.56:20.76
97 029
2,45=0092
91.772045
0.03::0110
93.11:10.00
CALACRASC
0.00::0.00
100.00=10.00
5.02:t5.86
98.27=0.33
3.11=1111
91,49++0.54
1.22+11.20
90.88+0.71
in all cases. Compared to the vanilla SI-GNN, SC consistently enhances explanation consistency (SHD), accuracy (AUC), faithfulness (FID), and informativeness (ACC). (ii) SC outperforms EE in most cases. Despite EE's ability to mitigate redundancy, SC achieves better performance while being more efficient (~5xx faster) and more general (compatible with all standard evaluation metries). This highlights SC as an appealing alternative pathway for addressing redundancy. (iii) SC and EE are complementary. When combined, SC+EE yields further gains over either strategy alone, suggesting that SC and EE tackle redundancy from different, reinforcing perspectives.
To better understand how SC improves explanation quality, we complement the main results with visualizations and case studies. Figure shows the distributions of edge scores for important and unimportant edges across four datasets with 8 0.05. Consistent with our analysis in Section 3.2 SC drives important edges toward 1 while pushing unimportant edges toward other low, near-fixed levels, resulting in more stable assignments. Figure 5 shows that the collective allocation induced by SC is stable: under the same conciseness regularization, score assignments remain similar across runs, yiclding consistent explanations. illustrates that, after SC training, the explainer assigns importance scores more responsibly. Com- plementary results for GSAT are provided Section D.I
case study on the BENZENE dataset (Figure (6) further
Since SC explicitly reduces discrepancies between successive explanations, it naturally lowers FID: consistent explanations yield consistent representations and predictions. Figure Figure/sshon shows PCA plots where SC produces visibly shorter connections between successive runs, and Table confirms with
8

---

Published as a conference paper at ICLR 2026
1.00
1.00
1.00
Edge Score
0.75
Edge Score
0.75
Edge Spore
Edge Score
0..5
Unimportant
0.50
0.50
Important
0.25
0.25
0.25
0.00
0.00
5e-2 1e-1 Se-1
5e-2
1e-1
5e-1
5e-2
1e-1
5e-1
5e-2 1e-1 5e-1
8
B
(a) BA-2MOTIFS
(b) MR
(c) BENZENE
(d) MUTAG
Figure 8: Edge scores under different values of 3 using SMGNN (GSAT results Figure
1.00
1.00
1.00
1.00
Eage Score
Edge Score
Scor
Edge ScoR
0.75
0.50
Edge
0.25
Unimportant
0.25
0.25
0.25
Important
0.00
0.00
0.00
10
100
0
10
100
1
10
100
10
100
(a) BA-2MOTIFS
(b) MR
(c) BENZENE
(d) MUTAG
Figure Edge scores under different values of 17 using SMGNN (GSAT results in Figure
higher cosine similarity and lower I.1 distance. To further validate effectiveness and rule out con- founds such as extended training time (pretraining the GNN encoder and fine-tuning the explainer and classifier), we include additional experiments in Section D.2
4.2 EFFECTIVENESS ON NON-CR CAN-CR-ASSED SI-GNNS
As discussed in Section 2.2 GAT and CAL lack explicit conciseness constraints. this setting, ap- plying SC alone may cause unimportant edges to drift toward high scores (e.g.. Figures 12e 12h and Figure 13h in Section D.I. which instead degrades explanation quality. Table confirms this: for instance, GAT+SC's AUC drops BA-2MOTIFS and MUTAGENICITY. When CR is added, SC improves explanation quality across all datasets in 28 out of 32 cases. Importantly, without CR the degradation can be severe (e.g., SHD increases by 6% and AUC drops by 10%), whereas with CR even the few unfavorable cases show only marginal gaps (1-2%). This contrast highlights that CR provides a stable foundation on which SC can consistently enhance explanation quality.
4.3 PARAMETER SENSITIVITY
Figures and anaayze the effect of the two hyperparameters and 77. Consistent with our analysis in Section 3.3 3 can influence the selection of hear-fixed levels. For example, in Figure Sal when
3==0 O, even unimportant edges may be pushed toward as 3 increases, their scores shift toward O, leading to a clearer separation between important and unimportant edges. In practice, / is selected by maximizing validation accuracy and adjusted through heuristic checks e.g., whether importance scores are sufficiently separated to yield informative explanations. As result, 3 typically lies in suitable range, and our strategy can be directly applied without additional tuning.
Compared to 3, our method is less sensitive to As increases, unimportant edges that would otherwise receive high scores are gradually suppressed toward lower values, while important ones remain unaffected: even very large 1) yield relatively stable results. This may be due to freezing the encoder, which prevents SC from perturbing the representation space. Moreover, since SC primarily targets redundant edges (Figure [T). its optimization rarely conflicts with the main training objec- tive. We defer more detailed discussion to Section E Notably, is the only additional parameter introduced by our strategy, yet its low sensitivity makes the approach readily applicable in practice.

---

Published as a conference paper at ICLR 2026
5 CONCLUSION
In this work, we discussed the fundamental mismatch between the training objectives of SI-GNNs and the faithfulness criteria by which they are evaluated. We showed that faithfulness is inherently tied to explanation self-consistency, and that enforcing self-consistency during training can serve as direct and principled way optimize faithfulness. Our analysis revealed that self-inconsistency arises mainly on unimportant edges, linking it to redundancy and motivating simple yet effective solution SC. Extensive experiments across diverse benchmarks demonstrated that SC consistently improves explanation consistency, accuracy, faithfulness, and informativeness, while remaining easy to apply in practice. future work, we plan to extend SC from graph benchmarks to A14Science domains, where trustworthy explanations are crucial for advancing scientific discovery.
ACKNOWLEDGEMENT
We thank Zeng Wang for her constructive discussions and valuable feedback. We also thank Yaqian Liu for her help with proofreading the manuscript and double-checking the experimental results. This work was supported in part by the National Natural Science Foundation of China (Grant No. 62176043, supporting Ting Zhong; and Grant No. 62572097, supporting Fan Zhou).
ETHICS STATEMENT
This work uses only publicly available benchmark datasets for graph explanation tasks. No hu- man subjects, personally identifiable information, sensitive data involved. Our work aims to improve explanation quality, which we believe has positive implications for transparency and trustworthiness in Al systems.
REPRODUCIBILITY STATEMENT
We have made every effort to reproducibility of results. All datasets used in this work are publicly available benchmark datasets for graph explanation tasks. The details of training pro- cedures, model architectures, and hyperparameter settings are provided in Sections and C Each experimental result is averaged over 10 independent runs to ensure reliability. An implementation of our method, along with training scripts and pre-trained model checkpoints, provided on Github.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Chirag Agarwal, Owen Queen, Himabindu Lakkaraju, and Marinka Zitnik. Evaluating explainability for graph neural networks. Scientific Data, 10(1):144, 2023.
Kenza Amara, Zhitao Ying. Zitao Zhang, Zhichao Han, Yang Zhao, Yinan Shan, Ulrik Brandes, Sehastian Schemm, and Ce Zhang. Graphframex: Towards systematic evaluation of explainabil- ity methods for graph neural networks. Learning on Graphs Conference (LoG), pp. 44-1. Proceedings of Machine Learning Research (PMLR), 2022.
Steve Azzolin, Antonio Longa, Stefano Teso, Andrea Passerini, al. Reconsidering faithfulness in regular, self-explainable and domain invariant gnns. In Proceedings of the International Confer- ence on Learning Representations (ICLR). Curran Associates, Inc., 2025a.
Steve Azzolin, SAGAR MALHOTRA, Andrea Passerini, and Stefano Teso Beyond topological self-explainable gnns: A formal explainability perspective. In Proceedings of the International Conference on Machine Learning (ICML), 2025b.
Xavier Bresson and Thomas Laurent.
Residual gated graph convnets.
arXiv preprint
arXiv:1711.07553, 2017.
Xuexin Chen, Ruichu Cai, Yuxuan Zhu, Julien Horwood, Zhifeng Hao, Zijian Li, José Miguel Hernández-Lobatoo et al. Feature attribution with necessity and sufficiency via dual-stage pertur- bation test for causal explanation. In Proceedings of the International Conference on Machine Learning (ICML), 2024.
Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in neural information processing systems (NeurIPS), 2017.
Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterizatioo with gumbel-softmax. In Proceedings of the International Conference Learning Representations (ICLR), 2017.
Yang Ji, Ying Sun, Yuting Zhang, Zhigaoyuan Wang, Yuanxin Zhuang, Zheng Gong, Dazhong Shen, Chuan Qin, Hengshu Zhu, and Hui Xiong. A comprehensive survey on self-interpretablle neural networks. arXiv preprint 2xXi:2500111634 2025.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Proceedings of the International Conference on Learning Representations (ICI.R, 2015.
Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional net- works. In International Conference on Learning Representations (ICLR), 2017.
Boris Knyazev, Graham W Taylor, and Mohamed Amer. Understanding attention and generalization in graph neural networks. In Advances in Neural Information Processing Systems (NeurIPS), volume 32, 2019.
Chris Lin, Gerald Sun, Krishna C Bulusu, Jonathan R Dry, and Marylens Hernandez. Graph neural networks including sparse interpretability. arXiv preprint arXiv:2007 00000001., 2020.
Yi Ju Lu and Cheng Te Li. Gcan: Graph-aware co-attention networks for explainable fake news detection social media. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 505-514, 2020.
Dongsheng Luo, Wei Cheng, Dongkuan Xu, Wenchao Yu, Bo Zong, Haifeng Chen, and Xiang Zhang. Parameterized explainer for graph neural network. Advances Neural Information Pro- cessing Systems (NeurIPS), 33:19620-1961, 2020.
Dongsheng Luo, Tianxiang Zhao, Wei Cheng. Dongkuan Xu, Feng Han, Wenchao Yu, Xiao Liu, Haifeng Chen, and Xiang Zhang. Towards inductive and efficient explanations for graph neural networks. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMII), 2024.
Siqi Miao, Mia Liu, and Pan Li. Interpretable and generalizable graph learning via stochastic atten- tion mechanism. In Proceedings of the International Conference on Machine Learning (ICML)), pp. 15543.1 Proceedings of Machine Learning Research (PMLR), 2022.

---

Published as a conference paper at ICLR 2026
Christopher Morris, Nils M. Kriege, Franka Bause, Kristian Kersting, Petra Mutzel, and Marion Neumann. Tudataset: A collection of benchmark datasets for learning with graphs. Proceed- ings of the International Conference Machine Learning (ICML) 2020 Workshop Graph Representation Learning and Beyond, 2020.
Judea Pearl. Interpretation and identification of causal mediation. Psychological Methods, 19(4): 459, 2014.
Bastian Pfeifer, Anna Saranti, and Andreas Holzinger. Gnn-subnet:
Disease subnetwork detection with explainable graph neural networks. Bioinformatics, 20-ii126- 2022.
Phillip E Pope, Soheil Kolouri, Mohammad Rostami, Charles E Martin, and Heiko Hoffmann. Ex- plainability methods for graph convolutional neural networks. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp- 10772-10781, 2019.
Nitendra Rajput and Karamjit Singh. Temporal graph learning for financial world: Algorithms, scal- ability, explainability & fairness. In Proceedings of the ACM SIGKDD Conference Knowledge Discovery and Data Mining (KDD), pp. 481 1818-4819, 2022.
Jiahua Rao, Shuangjia Zheng, Yutong Lu, and Yuedong Yang. Quantitative evaluation of explainable graph neural networks for molecular property prediction. Patterns, 3(12), 2022.
Sangwoo Seo, Sungwon Kim, and Chanyoung Park. Interpretable prototype-based graph informa- tion bottleneck. In Advances in Neural Information Processing Systems (NeurIPS), 2023.
Yongduo Sui, Xiang Wang. Jiancan Wu, Min Lin, Xiangnan He, and Tat-Seng Chua. Causal atten- tion for interpretable and generalizable graph classification. In Proceedings of the ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), pp. 1696-1705, 2022.
Wenxin Tai, Ting Zhong, Goee Trajcevski, and Fan Zhou Redundancy undermines the trustwor- thiness of self-interpretable gnns. In Proceedings of the International Conference on Machine Learning (ICML), 2025.
Ioannis Tsamardinos, Laura Brown, and Constantin F Aliferis. The max-min hill-climbing bayesian network structure learning algorithm. Machine Learning, 65:31-78, 2006.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in Neural Informa- fion Processing Systems (NeurIPS), 2017.
Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. Proceedings of the International Conference on Learning Representations (ICLR), 2018
Yuyang Wang, Jianren Wang, Zhonglin Cao, and Amir Barati Farimani. Molecular contrastive learning of representations via graph neural networks. Nature Machine Intelligence, 4(3):279- 287, 2022.
Dana Warmsley, Alex Waagen, Jiejun Xu, Zhining Liu, and Hanghang Tong. A survey of explainable graph neural networks for cyber malware analysis. In IEEE International Conference on Big Data (Big Data), pp. 2932-2933 Institute of Electrical and Electronics Engineers (IEEE), 2022.
Shiwen Wu, Fei Sun, Wentao Zhang, Xu Xie, and Bin Cui. Graph neural networks in recommender systems: A survey, ACM Computing Surveys (CSUR), 55(5):1-37, 2022a.
Yingxin Wu, Xiang Wang, An Zhang, Xiangnan He, and Tat-Seng Chua. Discovering invariant rationales for graph neural networks. In Proceedings of the International Conference on Learning Representations (ICLR), 2022b.
Keyulu Xu, Weihua Hu. Jure Leskovee, and Stefanie Jegelka. How powerful graph neural networks? In Proceedings of the International Conference on Learning Representations (ICLR), 2019.
12

---

Published as a conference paper at ICLR 2026
Zhitao Ying. Dylan Bourgeois, Jiaxuan You, Marinka Zitnik, and Jure Leskovec. Gnnexplainer: Generating explanations for graph neural networks. Advances in Neural Information Processing Systems (NeurIPS), 32, 2019.
Junchi Yu, Tingyang Xu, Yu Rong, Yatao Bian, Junzhou Huang, and Ran He. Graph information bottleneck for subgraph recognition. In Proceedings of the International Conference on Learning Representations (ICLR), 2021.
Junchi Yu, Jie Cao, and Ran He. Improving subgraph recognition with variational graph informa- tion bottleneck. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), 2022.
Hao Yuan, Haiyang Yu, Jie Wang, Kang Li, and Shuiwang Ji. On explainability of graph neural networks via subgraph explorations. In Proceedings of the International Conference Machine Learning (ICML), pp. 12241-12252. Proceedings of Machine Learning Research (PMLR), 2021.
Hao Yuan, Haiyang Yu, Shurui Gui, and Shuiwang Ji. Explainability in graph neural networks: A taxonomic survey. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 45 (5)95782-5799, 2022.
Shichang Zhang, Yozen Liu, Neil Shah, and Yizhou Sun. Gstarx: Explaining graph neural networks with structure-aware cooperative games. Advances in Neural Information Processing Systems (NeurIPS), volume 35, pp. 19810-19822) 2022.
13

---

Published as a conference paper at ICLR 2026
APPENDIX
A RELATED WORK
Self-Interpretable GNNs (SI-GNNs) are designed to generate explanations as an intrinsic part of their prediction process (Ji et al. 2025). By coupling interpretation with prediction, they aim to produce explanations that are more faithful to the model's 's decision-making than post-hoc ap- proaches Ving et al. 2019 Luo et al. 2020 Pope et al. 2019; Yuan et al. 2021 Zhang et al. 2022] Chen et 2024 Luo et al. (024). A typical design pattern involves two stages: (i) selecting crit- ical graph subset, often through soft or hard masks over edges or nodes, to serve as the explanation; and (ii) making the final prediction based solely this explanatory subset.
Existing approaches differ mainly in how they extract the explanatory subgraph (Tai et al. 2025 aitention-based approaches (Velickovic et al. 2018 Knyazev et al. 2019 Lu & Li (2200) directly assign importance scores via attention mechanisms without explicit constraints on conciseness. Causal-lcarning methods (Wu et al. 2022b; Sui et al. 2022) employ disentanglement and interven-
tions (Pearl, 2014) to isolate causal factors. Size-constrained methods (Lin et al., 2020 Luo et al.]
2024 Azzolin et al., 20256) encourage concise explanations by penalizing subset size sparsity.
Mutual information (MI)-conntrained methods (Yu et al., [021]] 2022] Miao et al., 2022 Seo et al., 2023
minimize the MI between the explanatory subset and the full graph to enforce conciseness.
Our work is complementary to these directions: instead of introducing new subset selection strate- gies, we focus on enforcing coff-consisteney during training. This provides a simple, model-agnoste way to optimize faithfulness and enhance explanation quality across different SI-GNNs directly.
Beyond the SI-GNNs discussed above, two recent studies provide relevant perspectives for our work. Azzolin et al. (2025)) questioned the reliability of faithfulness itself, showing that good faithfulness scores do not necessarily imply high-quality explanations. Their observation directly inspired our second research question-whethee whether explicitly optimizing faithfulness can genuinely improve expla- nation quality-and motivated us to adopt a broader evaluation that considers explanation consis- tency, accuracy, faithfulness, and informativeness together. Tai et al. (2025) investigated explana- tion inconsistency across repeated training runs of the same SI-GNN and traced it to explanation redundancy, i.e., the tendency of explainers to irresponsibly assign high importance unimportant edges when budget allows. Our study corroborates their findings and further shows that redundancy also underlies related form of instability self-inconsistency within single SI-GNN. Inspired by these insights, we hypothesized that mitigating self-inconsistency should also mitigate redundancy, and thereby improve explanation quality-a link already established in Tai al. (2025).
Our experiments confirm this hypothesis: under certain conditions, self-consistency training indeed improves explanation quality. More broadly, our work enriches the characterization of redundancy by revealing its manifestation as celf-inconsisteccy within a single SI-GNN, and further provides a new perspective for mitigation through a simple, efficient, and model-agnostic training strategy.
B SI-GNN PIPELINES
The generic pipeline of SI-GNNs can be described as follows. A GNN encoder first computes node embeddings:
V= = GNN(G).
(17)
An explainer module (typically an MLP) then estimates edge importance scores from the embed- dings of incident nodes:
Wi (TP(([::v))).
0gj = (wsy)
(18)
where (v:: denotes concatenation and a is the sigmoid. Each edge egy is included in the explana- tory subgraph G, with probability aij. yielding the factorized distribution:
P(G.) II Bern(asg).
(19)
14

---

Published as a conference paper at ICLR 2026
To enable differentiable sampling during training, follow Miao al. (2022) Luo al. (2024)
and employ the Gumbel -iggmoid trick (Jang et al. 2017):
P(ey) + σ ((Inc - In(1 c) + wsy)/r),
(20)
with 6 ~ Uniform(O, (0, 1) and temperature т. At inference, the score Oaj is used directly.
The sampled subgraph GN is processed by the same encoder to obtain graph-level embedding:
2, POOL(GNN((..),
(21)
where POOL is permutationn-invarian readout (e.g., sum, mean, or max). A classifier MLP finally maps 2a to the prediction:
Y MLP(zz).
(22)
The training objectives follow Section 2.1 Note that CAL differs from the other three SI-GNNs in how it handles the explanatory subset. While Equation (21) describes the standard formulation where G, is reused as input to the encoder, CAL does not reuse G, this way. More specifically, CAL has three classifiers corresponding to the three terms in Equation (3: (1) the first classifier takes G, as input; (2) the second classifier takes G. as input; (3) the third classifier takes G, UG, as input. During training, CAL applies intervention to Gg, but intervention is applied at inference time. Therefore, the third classifier actually receives the full graph G at inference, and its output is used CAL's final prediction.
Remark on SC Training. While the generic SI-GNN pipeline employs the Gumbel-Sigmoid trick during training enable differentiable sampling, make slight modification for SC training. Specifically, when constructing the paired subgraphs G( and G! used in the SC loss, we bypass the stochastic sampling step and instead use the deterministic edge scores to form G!") and G!?). This design avoids the additional randomness introduced by Gumbel noise, which would otherwise confound the SC alignment. At inference, we use Oij as the final importance scores, the same as prior work (Miao et al. 2022).
C EXPERIMENTAL SETTINGS
Baselines. We evaluate our method on four representative SI-GNNs, each reflecting different design principle: GAT (Velickovic et al. 2018), SMGNN (Lin et al. 2020). CAL (Sui et al. 2022). and GSAT (Miao al.) 2022). All baselines are re-implemented under unified training framework to ensure comparability (Tai et al. 2025). Specifically, GAT is trained with the standard classification loss. SMGNN combines classification with a sparsity regularizer, omitting the entropy term since Gumbel sampling already produces near- binary masks (Tai et al. 2025). CAL follows its original 3- classifier formulation, but for stability replace its 1-layer classifiers with 3-layer MLPs, applying this change consistently across all baselines.
In addition, compare against Explanation Ensemble (EE), post-hoc strategy designed to mit- igate redundancy and improve explanation quality (Tai et al., 2025). For each SI-GNN type, we train 10 models with different random seeds. EE is then constructed within each type by ensembling five models. This choice is dictated by the SHD metric: computing SHD requires two independent ensembles, and using more than five models would force reuse across ensembles, undermining the validity of the comparison.
Metrics. We adopt four metrics, each probing distinct aspect of explanation quality.
Consistency (edge-level). To measure explanation stability, we follow Tai et al. (2025) and com- pute the Structural Hamming Distance (SHD) (Tsamardioos et al. (006) across runs with different random seeds:
2 SHD(ee)) N(N
Σ
(p) a
(g) a.
(23)
ISPSQNN
where N is the number of runs. Unlike prior work that thresholds edge weights (e.g.. at 0.5), we compute SHD directly continuous scores, avoiding arbitrary cutoffs that can strongly bias results.
15

---

Published as a conference paper at ICLR 2026
Accuracy (instance-level). Following Wu et al. (2022b); Miao et al. (2022) ; Rao et al. 20022) , use ROC-AUC (AUC) to evaluate whether annotated ground truth edges receive higher importance than irrelevant ones. AUC is particularly useful for detecting redundancy, but its reliability depends on the reliability of the ground-truth annotations. As verified in Tai et al. (2025). the datasets we use align well with the model's ss decision rationale, ensuring valid evaluations.
Faithfulness (instance-level). Faithfulnesss evaluates whether the selected subgraph G, preserves the model's 's decision (Yuan et al. 2022). We adopt Fidelity (FID"), which we simply denote as FID in the main text, defined as:
FID (G,G,) 1- ((11((() = c((G)));
(24)
where c(-) is the predicted class. An alternative is Fidelity* (FID+), which checks whether removing G, changes the prediction:
FID = [c[ G.)) c(f(G)))
(25)
However, for SI-GNNs, FID+ confounds explanation quality with distribution-shift generalization, since G \ G2 is never used to train the GNN encoder. Therefore, we report FID in the main text, while including FID results in Section D.2 for completeness.
Informativeness (instance-level), Finally. we report classification Accuracy (ACC) (Wu et al,, 20226 Miao al., 2022 Amara al. 2022). While not direct metric of explanation quality, ACC serves as sanity check: if the explanations are informative, predictive performance should be preserved when restricted explanatory subsets.
Implementations. We follow the setup described in prior work (Tai et al. (0025) and adopt 2-layer
GIN (Xu et al., 2019) encoder (hidden sizes 64, 64) with dropout rate of 0.3. The explainer is a 3-layer MLP (hidden sizes 256, 64, 1) with dropout rate of 0.5 for predicting edge weights, and the classifier is a 3-layer MLP (hidden sizes 64, 64, 1) for final predictions. Hyperparameters are tuned on validation sets: learning rate from {0.01, 0.005, 0.001, 0.0005, 0.0001 and regularization coefficients (e.g.. )) from {0.01, 0.05, 0.1, 0.5, 1, 5, 10, 50, 100), guided primarily by classifi- explanation conciseness. All experiments repeated
cation accuracy with additional heuristics 10 times with different random seeds to ensure reliability. Implementations are based on Py PyTorch and training is performed with the Adam optimizer (Kingma & Ba 2015).
relpiineedrreeeedddd Because our method follows a two-stage training scheme, we directly use the checkpoint released by Tai et al.) (2025) the outcome of the first stage. On top of these, we fine-tune the explainer and classifier for 100 epochs while keeping the GNN encoder frozen. All hyperparameters are kept identical those in Tai al. (2025), with the only additional hyperparameter being the SC coefficient 17. which we set to 1 for all experiments. For BA-2MOTIPS, we found that the 33 setting reported Tai et al. (2025) (5e-2) yields relatively weak conciseness constraints; we therefore also consider a stronger B (5e-1). For GAT, the +CR variant corresponds to SMGNN, while CAL+CR follows the same implementation as SMGNN.
D MORE EXPERIMENTS
For completeness, provide additional experimental results in this section. We divide the presen- tation into two parts:
Complementary results. These are experiments already discussed in the main text, where we reported only a subset of results due to space limitations.
New experiments. Beyond the main text, we conduct further evaluations to give com- prehensive assessment of our method. These additional experiments serve to validate robustness, generality, and provide deeper insights into the behavior of SC.
D.1 COMPLEMENTARY RESULTS
Figure 10 Figure 13 present complementary results on all four classes of SI-GNNs, where we ex- plicitly track the evolution of edge scores assigned to unimportant edges during training with SC. For
com http://ssss..s...ooo.o../////sts wtwps///aaaaaaaaaaaaaa//aaa///a
archive
16

---

Published as a conference paper at ICLR 2026
(a) BA BA-2MOS
(b) MR
(c) BENZENE
(d) MUTAG
(c) BA-2MOTIFS (+SC)
(f) MR (+SC)
BENZENE (+SC)
(h) MUTAG (+SC)
Figure 10: Distributions of edge scores assigned to unimportant edges (defined by the dataset ground truth) across datasets using SMGNN. The first shows vanilla SI-GNNs, while the second row shows models trained with SC. Each subplot shows the distribution of edge scores for unimportant edges over training epochs (vertical axis).
9z NY. 635
M5 656 KM
k5
XE XI. EAS 030 BRS
(a) BA-2MOTIFS
(b) MR
(c) BENZENE
(d) MUTAG
XM 046 C55
JM
A00 AM 66 65
655
3M
3M
035 AGC
(e) BA- BA-2MOTIFS (+SC)
(f) MR (+SC)
BENZENE (+SC)
(h) MUTAG (+SC)
Figure 11: Distributions of edge scores assigned to unimportant edges across datasets using GSAT.
Plotting convention follows Figure 10
Li5 1a Klu 5.o 5.6
0U6 600
AS
000
(a) BA-2MOTIFS
(b) MR
(c) BENZENE
(d) MUTAG
QUB
XM
1088 0.000 0/00 C/00 61W
(e) BA- MA-TIFS (+SC)
(1) MR (+SC)
(g) BENZENE (+SC)
(h) MUTAG (+SC)
Figure 12:
Distributions of edge scores assigned to unimportant edges across datasets using GAT.
Plotting convention follows Figure 10
17

---

Published as a conference paper at ICLR 2026
(a) BA-2MOTIFS
(b) MR
(c) BENZENE
(d) MUTAG
(e) BA AMOIIMS (+SC)
(f) MR (+SC)
(g) BENZENE (+SC)
(h) MUTAG (+SC)
Figure 13:
Distributions of edge scores assigned to unimportant edges across datasets using CAL
Plotting convention follows Figure 10
SMGNN
(b) SMGNN+SC
(c) GSAT
(d) GSAT+SC
6TS O28
cooo 8G2
614
(c) SMGNN
(1) SMGNN+SC
@g) GSAT
(h) GSAT+SC
Figure 14: Impact of excessively strong CR regularization BA-2MOTIFS with SMGNN and GSAT. The top row shows the distributions of edge scores for unimportant edges (UNIMP), and the bottom row shows those for important edges (IMP). When 3 is overly large, CR collapses both important and unimportant edges (toward 0 in SMGNN and 0.5 in GSAT).
SMGNN and GSAT, we observe that unimportant edges converge toward stable low-score near fixed points. In contrast, for GAT and CAL, which lack CR regularization, applying SC alone may instead push uninformative edges toward arbitrarily high scores (e.g.. Figures 12e and 12h and Figure 13h
Figure 14 illustrates the impact of excessively strong CR regularization. When 3 is too large (5e0 for SMGNN and 5el for GSAT), CR dominates the training dynamics: in SMGNN both important and unimportant edges collapse near 0, while in GSAT they collapse around 0.5. Without SC, the distributional shape of edge scores is already distorted; consequently, applying SC merely forces these distorted scores to concentrate further. This highlights that SC is effective only when the CR strength is set within reasonable range, where the underlying separation between important and unimportant edges is still preserved.
Figure 15 extends the scatter plot analysis of Figure f to GSAT. We observe a similar effect as in SMGNN: with SC, important edges consistently driven toward high scores (near 1), while unimportant edges converge to lower, near-fixed levels. This separation reduces variance across runs and yields more stable and interpretable assignments, demonstrating that the effect of SC is not limited to cizsstoontd regularization but generalizes to MI-constrained models like GSAT.
Figure 16 presents the ridge plots for GSAT, complementing the SMGNN results in the main text (Figure 5). We again find that, without SC, the importance scores of uninformative edges fluctu- ate substantially across runs, reflecting the instability (irresponsibility) of the explainer. In con-
18

---

Published as a conference paper at ICLR 2026
1.0
1.0
1.0
1.0
Edge Scare
Edge Sport
0.5
0.0
50
100
0.0
0
100
200
300
0.0
0
600
1200
0.0
0
100
200
300
Instance index
Instance index
Instance index
Instance Inocx
BA-2MOTIFFS
(b) MR
(c) BENZENE
MUTAG
1.0
1.0
erymage map
1.0
Eoge Score
Edge Scare
Edge Score
Edge Score
0.5
0.0
100
200 Instance Index
100
300
0.0
600
1200
0.0
100
200
300
Instance Index
Instance index
Instance index
(e) BA BA-2MOSIFS (+SC)
(f) (+SC)
g BENZENE (+SC)
(h) MUTAG (+SC)
Figure 15: Distributions of edge scores assigned to important and unimportant edges across datasets
using GSAT. Plotting convention follows Figure
02
64
16
12
14
88
64
16
65
34
66
(a) BA AMOTMSS
(b) MR
(c) BENZENE
(d) MUTAG
92
64
26
10
24
26
22
64
6.8
69
24
(e) BM-TIFS (4SC)
MR (+SC)
gg) BENZENE (4SC)
(h) MUTAG (+SC)
Figure 16: Unimportant edge weights across 10 GSAT runs.
trast, SC encourages these scores concentrate around stable, low values, making their allocation consistent across different seeds. This confirms that the stabilizing effect of SC generalizes from size-constrained MI-constrained SI-GNNs.
Figure provides a complementary visualization for GSAT. As with the results reported in the main text, observe that SC training substantially reduces the distance between successive explanation embeddings, leading to shorter connections in the PCA space. This result confirms that the stability benefit of SC generalizes beyond size-constrained SI-GNNs to MI-constrained SI-GNNs. Together with the quantitative similarity results in Table 2 these visualizations highlight that SC consistently enhances explanation faithfulness across different SI-GNNs.
Figures 18 and 19 complement the SMGNN results in the main text by showing the sensitivity of GSAT the two hyperparameters 3 and 17. Consistent with our earlier analysis, governs the separation between important and unimportant edges by determining their selected near-fixed levels. Compared to 3, 1 exhibits greater stability, as generally pushes unimportant edges toward low values while preserving high scores for important ones. These results confirm that conclusions about parameter sensitivity generalize across both size-constrained and MI-constrained SI-GNNs.
Note that instead of evaluating SC on both GIN and GCN (Kipf & Welling 2017), we report results using GIN only. Empirically, we observe that SC does not behave properly on GCN: when SC is applied, nearly all edges tend to receive high importance scores. In fact, even without SC, the origi- nal GCN explanations exhibit very weak discriminability- the absolute gap between important and unimportant edges is very small. We conjecture that this phenomenon is related to the aggregation
19

---

Published as a conference paper at ICLR 2026
BA 2MOTIFS (GSAT)
MR (GSAT)
BENZENE (GSAT)
MUTAG (GSAT)
Raw: 614)
Raw: 6
+SC G
+SC: 6!a)
Figure 17: PCA visualization of explanation embeddings for GSAT. Each line connects two succes- sive representations of the same instance; shorter lines indicate higher stability.
1.00
1.00
Edge Score
Edge Score
0.75
D.75
Edge Score
0.75
0.50
0.50
Edge
0.50
0.50
0.25
0.25
Unimportant
0.25
Important
0.00
0,00
0.00
Se-1
le0
5e0
5e-1
1c0
5e0
5e-1
1c0
5e0
0.00
5e-1
1e0
5e0
8
8
(a) -A-2MSS
(b) MR
(c) BENZENE
(d) MUTAG
Figure 18: Edge scores under different values of /3 using GSAT.
1.00
1.00
1.00
Edge Score
0.75
Edge SCOre
0.75
Edge Score
0.75
Edge Score
0.75
0.50
0.50
0.50
0.50
0.25
Unimportant
0.25
0.25
Important
0.00
0.00
0.00
0.00
100
10
100
10
(a) BA MMTTISSS
(b) MR
(c)
BENZENE
(d) MUTAG
Figure 19: Edge scores under different values of r using GSAT.
mechanism of GCN. After multiple rounds of message passing, node representations tend to become smoothed, reducing inter-node distinguishability and making it difficult for the explainer to extract effective structural signals. We further evaluated SC GatedGCN (Bresson & Laurent (017) and GraphSAGE (Hamilton et al. 2017). whose architectural designs better preserve representational diversity across layers. In these settings, SC remains effective.
D.2 NEW EXPERIMENTS
To further validate the robustness of findings and address potential concerns about evaluation and training design, we conduct three complementary sets of additional experiments. First, we report results under Fidelity+ (FID+) to provide more complete view of faithfulness evaluation. Second, we examine whether the observed improvements of SC could simply stem from extended training in our two-stage framework. Third, we analyze the necessity of freezing the encoder during SC fine-tuning by evaluating several alternative training configurations.
D.2.1 FIDELITY EVALUATION
For completeness, we also report results under Fidelity (FID+), the other widely-used measure of faithfulness that evaluates whether removing the explanatory subgraph G, changes the model's prediction. Unlike FID which directly measures whether G, alone suffices for the prediction, FID inherently mixes explanation quality with the encoder's ability to generalize under distribution shift. Since SI-GNN encoders never trained on G G.,, their representations such inputs
20

---

Published as a conference paper at ICLR 2026
Table Fidelity evaluation on four SI-GNNs. Bold indicates improvements over the raw baselines.
2MOTIES
BENZENE
MUTAGENICITY
Method
FID (%)
FID (%)
FID (%)
FID (%)
FII (%)
FID (%)
FIID" (%)
"FID" (%)
SMGNN
0.50
12.60113.82
1,90=1.1
40.83:1.33
2.07=0662
52.99:12.03
1.72:11.09
8.68:12.98
SMGNN+SC
0.00=0.00
52.60-3.29
0.00
39.20=0062
0.97=0.59
05.35==0.88
0.61=0.54
10.24=1994
GSAT
0.00
46.40.66.65.
0.20-0.28
36.12.1.92
1.80-0.84
42.90.10.118
1.11.10.61
2.30.11.90
GSAT+5C
0.00:10.00
48.20=11.133
0.31-0.29
39.41:0.55
0.74=0.14
28.58:12.25
0.17:0.233
0.74:10.00
GAT
2.50=7.50
15.90
2.63=1.93
41.04#1.19
2.06=0.65
48.8347.39
0.37=0.53
0.98
GATASC
0.00:10.00
36.30:24.11
0.21
38.62:1.04
0.80-0.37
54.96:11.20
0.00:10.00
0.00:10.00
GAT+CR+SC
0.00=00.00
52.60=3.29
0.00=0.00
39.20=:062
0.97=0.59
05,35==0.88
0.61=0.55
40.24=1.94
CAL
17.10217.21
36.00=8.82
8.17-2.52
33.6743.86 86
4,97-3.44
41.1548.54
2.0621.114
5.0048.18
CAL-SC
0.00:10.00
38.30:24.09
1.56
39.38-10.80
2.45-0.22
55.21110.96
0.03:00110
0.10:2030
CAL-CR+SC
0.00::0.00
32.90-3.96
5.02-5.86
39.90=00.83
3,11=1111
1..08=11200
1,22+11.00
40.44::4055
be unreliable. This confound often obscures the actual contribution of the explainer. As shown in Table 4 our method with SC consistently improves over baselines on FID but the advantage on FID is less clear. This outcome is expected: SC explicitly designed to enforce self-consistency
on G.. which directly aligns with FID- but not with FID+. However, poor encoder generalization
on G G2 can significantly influence FID+. Therefore, while we provide FID+ for completeness, our main text focuses on FID together with other complementary metrics (consistency, accuracy, informativeness) for a more reliable and comprehensive assessment explanation quality.
D.2.2 DOES SC SIMPLY BENEFIT FROM LONGER TRAINING?
Since our framework adopts a two-stage design-firs first training the GNN encoder and then fine- tuning the explainer and classifier with the encoder frozen- one might suspect that the observed improvements come simply from longer training rather than SC itself. To rule this out, we conduct an ablation where we fix the encoder and fine-tune only the explainer and classifier under two settings: r (two-stage training without SC) and r) = (two-stage training with SC).
As shown in Table 5 fine-tuming without SC does not consistently improve explanation quality; in some cases even degrades performance (e.g., GSAT on BA-2MOTIFS, where AUC drops by 13%). This indicates that extended training alone is insufficient for achieving better explanations. While fixing the encoder and fine-tuning the explainer and classifier can sometimes provide modest gains, this is not due to additional training epochs (our experiments show models already converge well before 100 epochs), but more likely because fixing the encoder avoids feature drift from joint optimization and thereby stabilizes training. By contrast, once SC is introduced, explanation qual- ity consistently improves across all datasets and metrics, outperforming not only the corresponding vanilla SI-GNNs but also, in most cases, their counterparts trained without SC loss. This demon- strates the unique value of SC training: it makes the explainer more responsible in assigning impor- tance scores, leading to more stable and higher-quality explanations.
Such a two-stage training scheme is not incidental but grounded in our motivation: explanation redundancy mainly arises because the explainer, when given sufficient budget, tends to behave irre- sponsibly. Therefore, SC loss only influences the explainer. The classifier is fine-tuned alongside the explainer because shifts in the explainer's 's behavior may alter the feature distribution of the graph representation, requiring the classifier to adapt correspondingly.
Remark on Efficiency. Compared to vanilla SI-GNNs, our two-stage framework introduces moder- ate additional training overhead (roughly doubling the training time) but yields substantial gains in explanation quality. Relative to EE- first community propposed solution for improving explana- tion quality method is significantly more efficient: since EE ensembles K independently trained models, our approach requires only 1/K (i.c., ~20% of the memory or runtime cost at inference, while also avoiding the need to train multiple models.
Taken together, these results show that SC goes beyond faithfulness optimization: it provides a more efficient and effective mechanism than EE for mitigating redundancy, while also complementing EE- togethr they achieve better performance than either alone- and ultimately offers a scalable pathway more trustworthy GNN explanations.
21

---

Published as a conference paper at ICLR 2026
Table 5:
Experimental results on SMGNN and GSAT (with different 77).
Notation (bold, color)
follows the same convention as Table
BA-2MOTIFS
3MR
BENZENE
MUTAGENICITY
Method
SHD (5)
AUCC (5)
SHD (%)
1 AUC (%)
J. SHD (%)
(U)
S (S)
1 AUC (%)
SMGNN
10,44.1441
99.32.1.36
9.44.12.24
97.00-0.77
16.06.16.57
84.38.1.711
12.6513.18
98.12.100.9
5MG
4,99-1.45
99.59+0.16
5.20+0.72
98.25 0.22
8.55+22.99
91.38+0.96
6.21+1.20
98.96-0..
SMGNN-SC (v=0))
8.46:3.48
09.26:20499
13.01=3.02
98.72=0.35
8.30:22.39
88.72:1.255
9.68:2.34
97.04=0.74
+SCEE (y=0)
4.02::1.21
99.01.10.27
7,34
99.18 0.00
4.71:20.88
91.75.1:0.50
5.28
08.
SMGNN-SC (
3.48:22.20
99.87:10.005
5.479:3.39
98.39-0.88
7.19=22.22
90.07:1.56
1.51::1002
0,.30=00,54
+MGNN+SC-EE (p=1)
1.20:1.03
99.90.10.01
3.05
0.14
92.20.10.43
0.44
0..96
GSAT
4.58=1.81
08.44=0061
12.38=+3.55
98. 0.:3031
6.93=22.75
90.66+t00.99
10.08:2:4.58
99.01=00331
GSAT-ER
2.10-0.67
98.80 0.16
5.79-11.07
99.16 0.03
3.25-1.09
92.18+00599
4.70+1.66
99.56+0.05
GSAT+SC (y=0)
5.03=1.78
85,22:4-4.26 4626
9.52:42.82
99.17-0.23
5.30=0090
92.79::0.44
6.71=:0.65
09.04=0.33
GSAT+SC-EE (1=1))
2.25110.88
86.33-13.54
1.48
99.42-0.01
2.49:00.31
93.60:10.13
3.03
0.
GSAT+SC ()=
2.73=1.02
99.30::0.12
6.673
98,91-0.29
2.32:00.62
92.80::0.36
2.38
99.38 0.05
GSAT+SC-EE (p=1)
1.1910.52
99.35
4.00.10.38
99.39 0.04
0.14::0.2
93.53.10.11
1.06:0.42
99.4110.01
(S))
1 ACC (98)
J FID (%)
T ACC (96)
(9)
(C)
J FID (8)
ACC (9))
SMGNN
0.50:0092
15.50:112.52
1.90
97.30=1.31
2.07:00.2
91.12:10.22
1.7241.09
89.53:10.99
SMGNN-EID
100.00::0000
98.25 0.21
92.16:10.24
90.36:10,70
SMGNN-SC (y=0)
0.10:0.30
99.90.10.30
99.65-0.00
182:20443
92.33.10.45
1.45:0.488
SMGNN (2=0)
100.00=0000
99.65 0.00
92.81++0.18
93.39-0.47
SMGNN-SC (yel)
0.00=0.00
99.80:20.60
0.00
99.65-0.00
0.97=00.99
92.57=00444
0.61
91.18=1.077
SMGNN+SC+EE (p=1)
100.0010.00
99.65=0.00
93.24:10.19
92.11::0.45
GSAT
0.00
100.0010.00
0.901:.228
98 55.55.0000
L80020.84
91.48:20..87
1.1120661
92.43=11.00
GSAT-EE
100.00::0.00
99.15 0.24
92.1710..28
93.00:10.44
GSAT4SC ())
0.80
100.00:0.00
0.55+-023
99.07
1.38=0.26
92.88:+0.23
0.98+0.57
93.14=0.79
(SAT+SC-EE
100.00 0.00
99.65 0..00
93.19:10.15
93.88
GSAT+SC (y=l)
0.00=0.00
100.00=0000
0.31
0.74:0.14
92.31:00.22
0.17#0.23
93.48=0.48
GSAT+SC-EE (1=1)
100.00:0000
99.65-0.00
92.61:100.18
93,76:00.77
Table 6: Experimental results on SMGNN and GSAT (with different training strategies). Notation (bold, color) follows the same convention as Table
BA-2MOTIFS
3MR
BENZENE
MUTAGENICITY
Method
J.HDD (5)
AUC (8)
J (%)
A (5)
J SUD (%)
1 AUC (%)
SHD (%)
AUC
SMONN
10.44.14.41
99.32.0..36
9.44:12.2
97.0010.77
16.06.6.57
12.6513.18 84.38.1.771
98.12.10.39
SMGNN-SC (S1)
7.7126.14
99.3120.45
15.55:2.33
95.38:1.51
19.3325.43
73.59:1.69
8.58:14.01
97.3020.67
SMGNN-SC (82)
3.68
99.64 0.23
9.76.13.49
96.5910.78
14.18
80.78.13.05
9.12-12.86
98.4410.47
SMGNN-SC (S3)
8,28=10.61
99.61=0.89
12.11=33.13
97.6010.48
10.91=2.66
8188::2.554
7.69:13..9%
97.91 20.42
SMGNN-SC (84)
3.10=2.07
99. 89 0.00
5.71=1.41
98.76220300
7.01=1.82
$8.37=11.99
2.79::0,97
98.69
SMGNN+SC
3,48-2220
99.87 0.05
3.37
98.39--0.88
7.19-22.2
9.50
2.51+11.02
98.30 0.54
GSAT
4.58+1181
98.44-0.00
12.35+33.55
98. 98.38-+0.31
6.934-2.75
90.66-+0.89
10.08++4.58
9-031
GSAT+SC (8I)
3.73=1.511
98.96=0.88
3.94=33.3
97.81+0.544
5.21
90. 14.34tt1.0 1.02
4.20-+2.01
98, 00
GSAT (82)
2.32--0.84
99.13 0.33
7.60
97.53-+0.60
4.31 +1.78
90.97 1.03
4.77-4-2.78
99.29 0.
GSAT+SC (83)
2,53
99.08 0.19
6.91
98.6910022
3.19 3.19
10.78.10.73
3.88.12.48
99.1610.J
GSAT+SC (S4)
184=0077
0.22
6.21:13.46
98.95:10.27
2.65=0058
92.65:10.33
4.39:14.32
99. 09.20=0.08
GSAT+SC
2.73=1.022
99.30=0.11
6.67=33.33
98.91
2.32=0062
92.80==0.36
2.38++0.84
99.38=0.05
(I)
1 ACC (9)
(96
ACC
FTD (%)
1 ACC
T FID (9)
T (9)
SMONN
0.50:20.22
95. 15.50
1.902
1.33
2.07:10.62
91.12:10.62
1.7241.009
89.5310.99
SMGNN-SC (81)
0.30=00.46
59.50=0.50
0.38
97.85=0.94
1.61=0.53
90.32=20.93
0.81=20.38
9..54=1.56
SC (82)
0.20=0044
99.90-0.30
1.28-00.1
98.24-00.5
1.67=0059
90.6 00.66-t: 1.37
1.15+0.66
19 16
SMGNN-SC (83)
0.00
100.00 0.00
0.36
98.44
1.13 0.49
92.21
0.77
90.27 0.84
SMGNN+SC (S4)
0,00
100.0000.00
0.45:10.49
99.2710.42
1.66
92.45.10.36
0.68100.4
1.37
SMGNN-SC
0.00=0000
0000
0.00::0.00
99.65=10000
0.97=0.59
92,57:20.44
0.61=20.54
91.18=1007
GSAT
0.00
100.00
0.90.10.28
98.85
1. L..84
91.48.10.87
1.11110.61
92.43 1.00
GSAT+SC (SL)
0.50:0.50
99 99.60900.9
0.76:10.40
98.34110.46
1.19:00.47
92.03:10.53
0.17:10.23
1.15
GSAT+SC (S2)
0.40=0049
99. 00.60
1.00=0055
18-1022.52
0.17
92.36-20.37
0.24=:0.30
92.87=0.88
GS (83)
0.10=00.00
99 99,90=00330 30
0.03=:0.10
99.62:40.10
1.03=0035
92.83-00544
0.17+00.33
93.11-00.88
GSAT+SC (54)
100.00 - 0.00
0.10- 0.16
99.62 0.10
L,5 0.19
92.45 0..36
0.20--..22
93.24 0.83
GSAT+SC
0.00
100.0011..00
0.31:10.29
99.5510..16
0.74
92.31.10.32
0.17.10.23
13.48
D.2.3 ABLATION: SHOULD THE ENCODER BE FROZEN DURING SC FINE-TUNING
Our SC training framework freezes the GNN encoder during the fine-tuning stage (Step 2), so that the SC loss affects only the explainer (and the classifier that depends on its outputs). To examine whether this design choice is necessary,
evaluate four alternative configurations. These variants differ in (i) whether the encoder is fully or partially trainable during Step and (ii) whether gradients from the SC loss are allowed to update the encoder. For clarity, denote these settings as S1-S4:
22

---

Published as a conference paper at ICLR 2026
4.5 x10-1
3.0
1.5
0.0
UNIMP IMP
UNIMP IMP
UNIMP IMP
UNIMP IMP
UNIMP IMP
UNIMP IMP
UNIMP IMP
UNIMP IMP
SMGNN+SCIA) +SAT+)C(A) SMGNN+SCIB) GSAT+5C(BS SMGNN+SCIC) GSAT+SCIC) SMGNN+ SC(D) GSAT+SC(D)
Figure 20: Self-inconsistency measured the Ll difference (DIFF) between explanations after
SC training. Plotting convention follows Figure]
S1: Joint training, SC updates the encoder. The encoder, explainer, and classifier are all trainable, and the SC loss backpropagates through the entire encoder.
S2: Joint training, SC does not update the encoder. The encoder is trainable under the standard SI-GNN objective, but gradients from the SC loss are blocked before reaching the encoder.
S3: Last encoder layer trainable, SC updates the encoder. Only the last encoder layer is unfrozen, and the SC loss is allowed to update this layer.
S4: Last encoder layer trainable, SC does not update the encoder. The last encoder layer participates in standard SI-GNN optimization, while the SC loss remains restricted to the explainer.
Results in Table 6 provide the following practical guidance: (1) Do not allow the SC loss to update the encoder. (2) Do not fully unfreeze the encoder during SC fine-tuning. (3) We suggest freezing the encoder, which yields the most stable performance in practice.
E PRACTICAL CONSIDERATIONS OF THE SC COEFFICIENT
This section provides deeper discussion of the coefficient introduced by the SC loss. We address two aspects. First, in Section E.I we show that incorporating SC does not alter the optimal explana- tory subset, but instead reshapes the optimization landscape by discouraging redundant solutions. Second, in Section E.2 we analyze how to select in practice: by deriving upper bounds on the expected SC loss, we can restrict its search space during training, and we further report empirical findings that a simple choice 1) 1 already works well across all datasets.
E. PRESERVATION OF OPTIMAL EXPLANATIONS
Recall the original SI-GNN objective Tai et al. 2025
max G.SG, G..IMM
T(G,; Y),
(26)
where M is the budget and G denotes the ground-truie truth explanatory subset. With SC, the augmented training objective becomes:
I(G()
(27)
aCc a ca. IG M.
c Se
where ( is the explanation induced by feeding G back into the model.
Since Figure I demonstrates that only unimportant edges exhibit self-inconsistency while impor- tant edges are inherently stable, it follows that the ground-truth explanation G; must also be self-consistent. Therefore, G remains a feasible solution under the SC-augmented objective and achieves the same maximum MI as in the original one. In other words, SC does not alter the optimal explanation but reshapes the optimization landscape by discouraging redundant subsets that exploit unimportant edges. For comparison, we present the results after SC training in Figure 20
23

---

Published as a conference paper at ICLR 2026
E.2 PRACTICAL GUIDELINES
The SC coefficient vj is the only additional hyperparameter introduced by our framework. During training, its search range can be guided by analyzing the expected value of the SC loss. Recall that the augmented objective (for SMGNN and GSAT) is:
LET = LcB + 3- R(G.) + Lsc.
(28)
The following result provides a theoretical upper bound on the expected self-consistency error.
Proposition Let @, and denote the parameters of the GNN encoder (fixed), explainer, and classifter, respectively. Define EDD as the empirical expectation over set of samples 'D. Let o" arg ming ED (G; 0, d, t)}, and & denotes any perfectly self-consistent solution that satisfy & arg ming ED [ G.:; 0, o)
Then, we have:
log C + Ω
ED (G)) 00
(29)
where the number of label types, and Q2 is the maximum value of 3 R(G.).
Proof. Since of is the minimizer of ED (G; o, v)), we have:
Ep [an(G: .b) + n +b)+ L G,,)
< fGE (G;0, + +n Lsc(G ".?)), 0,
(30)
Since (so a
0, 5) O. we have:
Ep (G; 0, x., v) + (sc(G!), 0, s &ED (GE(GG,,,, w)
(31)
Due to the non-negative property of LGE, the above inequality can be further simplified:
ED (G: 0" w) n Lsc (G (( x²) s ED [or(G; w))
(32)
Ep 1 ( の*)
Ep [oo(G;;, v)) Loe O";
.))
(33)
ED n (sc(G 0 x*)
s SED (o
(34)
ED CG v)
En (s
(35)
Suppose the number of samples different classes is the same. The worst-case for the GNN ex- plainer happens when it discards all edges relevant to the label, but preserves all irrelevant edges. Under such circumstances, we have a perfectly self-consistent explainer but the downstream classi- fier cannot obtain any information from it. Thus, we have:
LGE = LCs + 3 R(G)) < log C + B BR(.))..
(36)
Substituting Equation (36 into Equation
(35 we can get:
log C Ω
EP [sc(GG)
G( 0, }5} 5
(37)
where 1 is the maximum value of 3. R(G.).
For SMGNN that uses the sparsity loss, the worst case is:
G.a
max B R(G.) 3 G.
B.
(38)
GG|
24

---

Published as a conference paper at ICLR 2026
Thus, for SMGNN, we have:
Ed [so(GG)) 00 のつ) v
log C
(39)
For GSAT that uses KL divergence constrain G,, the worst case is:
max B- G.
BR(G,, = Gl
Σ
aij log
ay + (1 (1-ais) ans) log
1 0ij
1-r
ageE
au
1 1-0kj
I 3 max oi
Oij log
+ (1 - asj) log
(40)
--
where r E [0, is hyperparameter controlling the conciseness of the GNN explanation. To cal- culate the maximum value of Equation (40), need to calculate the first and second derivatives of p(-). Setting the derivative equal zero, we solve o(l-r) 1, which yields = r. The second derivative a[ is always positive, indicating that the function attains local minimum at and maximum at end points, So the maximum value is: 3-max(log())) log( +-) Thus, for GSAT, we have:
r(i-a)
log C' + B max(log(#)) logg) (og((==r))
ED [ : <
(41)
In practice, will be too large too small-too small r means that edges will be discarded, which makes the training process unstable, and r that is too large cannot provide valuable explanations. In our experiments, we set r = 0.5, so we have: I log C+
5
Given the tolerance factor 5, first formalize candidate set S(v). (0, l Then select from S(n) based on validation performance. In our experiments, we find that = already yields good enough results on all datasets.
F THE USE OF LARGE LANGUAGE MODELS
ChatGPT were used an assistive tool during the preparation this paper. Their role was lim- ited to improving the clarity and readability of the text (c.g., language polishing) and checking the presentation of mathematical derivations. All research ideas, methodological designs, experiments, analyses, and conclusions were solely developed and validated by the authors, who take full respon- sibility for the content of this work.
25


Reviews:
[{"review_id": "0rQs2QoynS", "reviewer": "Reviewer_25Yx", "summary": "The authors propose a simple but sensible model-agnostic strategy for improving the faithfulness of self-explainable GNNs. The idea is to penalize the model for \"changing its mind\" whenever fed with a local explanation of a certain prediction during training. The technique is validated using four approaches and compared\nagainst an alternative (explanation ensembling) on four datasets, with encouraging\nresults.", "questions": "None.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- **Originality**: The idea behind the penalty is aligned with existing results,\n  but otherwise original -- to the best of my knowledge.\n\n- **Quality**: The proposed idea is sensible. The empirical setup is also good\n  -- the choice of datasets, metrics and competitors all look good. I appreciate\n  how the authors clearly distinguish between faithfulness (measured with Fid-)\n  and plausibility/explanation accuracy. This is surprisingly rare in the\n  literature.\n\n- **Clarity**: The text is very clear and well structured. The visualizations\n  are helpful.\n\n- **Significance**: The contribution is welcome and I think it bridges a\n  serious gap in the literature, by making theoretical insights practical.\n  The fact that the approach is model agnostic also helps.\n\nTL;DR: good paper, I like the idea and the execution.", "weaknesses": "- **Originality**: As I mentioned, I believe the proposed technique follows\n  naturally from existing results (eg Azzolin et al, who the authors mention).\n  This is however not a major issue for me - the penalty, as I mentioned,\n  is novel.\n\n- **Clarity**: My only real complaint is that Table 1 has too many colors, making\n  it diffult to focus on what's really relevant. I'd suggest to tone it down.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0rOaP7GvJJ", "reviewer": "Reviewer_AnRn", "summary": "The authors address a relevant and widespread problem in GNN (and SE-GNN in particular), which is the consistency and reliability of the explanations being extracted, and propose an approach to encourage self-consistency of explanations at training time.", "questions": "Can you clearly motivate the performance difference between SC and EE? aren't they basically optimizing for the same objective?\n\nCan you add FID+ results and comment them to better understand the interplay between SC and CR?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Trying to directly enforce consistency at training time seems like a sensible direction.\n\nExperimental results confirm improvements and provide insights into explanations quality and robustness.", "weaknesses": "The problem of consistency of explanations was already addressed in the work by Tai et al, where they propose a simple esamble strategy (EE). The novelty of the work is thus not dramatic. The authors show that their approach improves over EE (apart from being clearly much faster), and their combination further improves results. The rationale for these results is unclear to me. Is there any substantial difference in what the approaches tackle justifying this? Is this a matter of hyperparameter choice? This is important to correctly evaluate the relevance of the contribution.\n\n\nThere are plenty of notions of faithfulness of explanations, but a key aspect is that one should measure both sufficiency (e.g. with FID-) and precision (e.g. with FID+). I thus think the authors should include FID+ (or similar metrics) to get the full picture. Given that the proposed SC component is combined with conciseness regularization (CR), I do not expect this to undermine the utility of the approach, but it would allow to get a better picture of its contribution, especially when seeing the FID+ results without CR (table 3).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YsyGVSGswg", "reviewer": "Reviewer_hKUr", "summary": "The paper studies a common mismatch in self-interpretable GNNs (SI-GNNs): models are trained with classification plus conciseness regularization, but are evaluated by faithfulness (whether the highlighted subgraph alone reproduces the prediction). The authors argue that faithfulness is intrinsically linked to self-consistency (SC)—the explanation produced on a graph should be reproduced when the model is run on its own explanation. They propose a simple, model-agnostic training strategy that adds a self-consistency loss during a short fine-tuning stage where the GNN encoder is frozen and only the explainer and classifier are updated, yielding a final objective.\nThey analyze how enforcing self-consistency drives importance scores toward a few “near fixed levels,” and how this interacts with conciseness regularization (CR) such as sparsity (GISST) or MI constraints (GSAT) to suppress unimportant edges while preserving important ones.\nOn four benchmarks (BA-2MOTIFS, 3MR, BENZENE, MUTAGENICITY) and four SI-GNN families (GISST, GSAT, GAT, CAL), SC improves consistency, faithfulness, explanation accuracy, and informativeness; it also complements explanation ensembling (EE) and is substantially more efficient at inference. SC alone may hurt when CR is absent (GAT/CAL), but adding CR stabilizes and recovers gains. The method requires no architectural change and adds one low-sensitivity hyperparameter ($\\eta$) .", "questions": "1. What happens if the Step 1 SI-GNN misses part of the ground-truth rationale? Please evaluate robustness by ablating a portion of important edges before Step 2 and reporting whether SC can recover faithfulness or simply stabilize incomplete masks.\n\n2. How many models are produced overall, and which one generates explanations? My understanding is that only a single model—the Step 2 checkpoint—is used at inference. If multiple initial explanations are generated in Step 1, are they ensembled before Step 2 or during inference?\n\n3. Consider generating multiple Step 1 explanations and comparing unions or averaged masks before Step 2. This could test whether broader initial coverage yields better results.\n\n4. Since SC alone can hurt for GAT/CAL, what is the weakest form of conciseness regularization ((L_1) or entropy penalty) that stabilizes training?\n\n5. Could encoder-side regularization, such as partial Mixup on masked graphs, reduce encoder distribution shift and improve FID+?\n\n6. Would allowing the last encoder layer to update during Step 2 better adapt to explainer changes without destabilizing learned representations?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Reframes faithfulness optimization as enforcing *self-consistency* during training and connects it to redundancy on unimportant edges; provides a model-agnostic, plug-in loss with a two-stage schedule that freezes the encoder  .\n2. Extensive experiments on four datasets and four SI-GNN families. Clear reporting that SC alone can fail without CR, plus the complementary effect with EE. Solid ablations on ($\\beta$) and ($\\eta$) sensitivity and two-stage training to rule out confounds  .\n3. The training procedure and metrics are well defined; the figures make the stability effect tangible, and the paper carefully explains when SC helps and when it needs CR support .\n4. Practical because it needs no architecture changes, adds a single hyperparameter of low sensitivity, and improves multiple explanation dimensions while being more efficient than EE at inference time .", "weaknesses": "1. The two-stage setup assumes that Step 1 already produces explanations covering the ground-truth rationale. Step 2 then removes redundancy. If Step 1 misses key parts of the rationale, SC may stabilize an incomplete subset rather than recover it. The paper should explicitly state this assumption and test its robustness.\n\n2. Adding more initial explanations (e.g., multiple seeds or ensembling in Step 1) could alleviate this limitation. The gains seen in Table 1 when combining with EE support this interpretation.\n\n3. All datasets are small; results on large-scale or node-level tasks would clarify generality.\n\n4. Limited theory: The fixed-level analysis is illustrative but not formal; it would be valuable to characterize conditions guaranteeing convergence toward faithful explanations.\n\n5. Incomplete metric coverage: Improvements on FID− are strong, while FID+ results remain mixed; additional discussion would improve completeness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "03fJDbMfLW", "reviewer": "Reviewer_7o1e", "summary": "This paper introduce a simple, model-agnostic self-consistency (SC) post-training strategy for pretrained Self-Interpretable GNNs (SI-GNNs). \nRigorous analysis demonstrates that faithfulness can be explicitly optimized through the strategy, and such optimization genuinely improves explanation quality.\nExperiments show that SC consistently improves explanation quality across multiple dimensions and benchmarks, offering an effective and scalable pathway to more trustworthy GNN explanations.", "questions": "Please refer to the weaknesses for suggestions. This article is highly accomplished, and the weaknesses mentioned above will not change my positive evaluation.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "- The paper provides a fresh perspective (Self-Consistency) to improves the trustworthiness of self-interpretable GNNs, , the theoretical and empirical verification of the key assumptions is sufficient, there are no obvious flaws.\n- The method demonstrates strong explanation performance across multiple tasks, improving various types of self-Interpretable GNNs.\n- The paper is well-organized and easy to follow.", "weaknesses": "1. During the fine-tuning phase of SC (Section 3.1), this work freezes the GNN encoder and assumes that \"the encoder representation is already optimal, and only the explainer needs optimization\". However, this work fails to validate this design and could further compare it with a control group where \"the encoder is not frozen\".  \n2. Has the author conducted statistical significance tests? What percentage of the experimental results significantly outperform each baseline? This is crucial for understanding the overall performance of SC.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)