RegExplainer: Generating Explanations for Graph Neural Networks in Regression Tasks
Authors: Jiaxing Zhang
Zhuomin Chen
hao mei
Longchao Da
Dongsheng Luo
Hua Wei
Venue: NeurIPS 2024 poster (2024)
Keywords: graph neural network
explainability
data augmentation
Primary area: interpretability_and_explainability
TL;DR: This work extend the graph information bottleneck from graph classification tasks to the graph regression tasks.

Abstract: Graph regression is a fundamental task that has gained significant attention in
various graph learning tasks. However, the inference process is often not easily
interpretable. Current explanation techniques are limited to understanding Graph
Neural Network (GNN) behaviors in classification tasks, leaving an explanation gap
for graph regression models. In this work, we propose a novel explanation method
to interpret the graph regression models (XAIG-R). Our method addresses the
distribution shifting problem and continuously ordered decision boundary issues
that hinder existing methods away from being applied in regression tasks. We
introduce a novel objective based on the graph information bottleneck theory (GIB)
and a new mix-up framework, which can support various GNNs and explainers
in a model-agnostic manner. Additionally, we present a self-supervised learning
strategy to tackle the continuously ordered labels in regression tasks. We evaluate
our proposed method on three benchmark datasets and a real-life dataset introduced
by us, and extensive experiments demonstrate its effectiveness in interpreting GNN
models in regression tasks.

Reg Generating Explanations for Graph Neural Networks in Regression Tasks
Jiaxing Zhang', Zhuomin Chen², Hao Mei Longchao Da 3 Dongsheng Luo', Hua Wei New Jersey Institute of Technology, "Florida International University, AArizona State University jjz48@njit.eda {zchen051, dluo) ) @ ffu.edu, ((hme77, longchao, hua. wei) @asu.edu
Abstract
Graph regression is a fundamental task that has gained significant attention various graph learning tasks. However, the inference process is often not easily interpretable. Current explanation techniques are limited understanding Graph Neural Network (GNN) behaviors in classification tasks, leaving an explanation gap for graph regression models. In this work, we propose novel explanation method to interpret the graph regression models (XAIG-R), Our method addresses the distribution shifting problem and continuously ordered decision boundary issues that hinder existing methods away from being applied in regression tasks. We introduce a novel objective based on the graph information bottleneck theory (GIB) and new mix-up framework, which can support various GNNs and explainers in a model-agnostic manner. Additionally, we present a self-supervised learning strategy tackle the continuously ordered labels in regression tasks. We evaluate our proposed method on three benchmark datasets and a real-life dataset introduced by us, and extensive experiments demonstrate its effectiveness in interpreting GNN models in regression tasks.
Introduction
Graph Neural Networks [ (GNNs) have become a powerful tool for learning knowledge from graph- structure data and achieved remarkable performance in many areas, including social networks [2] B), molecular structures 4 [5, traffic flows 9. and recommendation systems [2] Despite the
success, their popularity in sensitive fields such fraud detection and drug discovery [13 [4]
requires an understanding of their decision -making processes. To address this challenge, some efforts have been made to explain GNN's predictions in post-hoc manner, which aims find a sub-graph that preserves the information about the predicted label. On top of the intuitive principle, Graph Information Bottleneck (GIB) 15 16 maximizes the mutual information / (G"; Y) between the target prediction label Y and the explanation G* while constraining the size of the explanation.
However, existing methods focus the explanation of the classification tasks, leaving another funda- mental task, explainable regression, unexplored. Graph regression tasks exist widely in nowadays applications, such predicting the molecular property [17] traffic flow volume [8 Explaining the instance-level predictions of graph regression challenging due to two main obstacles. First, in the routinely adopted GIB framework, the mutual information between the explanation sub-graph and label, T(G Y), is estimated with the Prooss-Ettropy between the predictions f(G") from GNN model f and its prediction label Y. However, in the regression task, the regression label is the continuous value, making the approximation unsuitable. Another challenge is the distribution shifting problem in the usage of J(G where the prediction of the explanation sub-graph made by the GNN model f is unsafe. Usually, explanation sub-graphs have different topology and feature information compared the original graph. As result, explanation sub- aub-graphs are out-of- didstrribbtti of the
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

original training graph dataset [1) As shown in Figure T a GNN model f is trained on the original graph training set and cannot be safely used to make predictions for sub-graphs.
To fill the gap, in this paper, we propose
RegExplainer, to generate post-hoc instance- level explanations for graph regression tasks. Specifically, we formulate theoretical-sound objective for explainable regression based on information theory. To further address the distri- bution shifting issue, RegExplainer develops new mix-u up approach with self-supervised learn- ing. Our experiments show that RegExplainer provides consistent and concise explanations of GNN's predictions on regression tasks. We achieved up to 48.0% improvement when com- pared to the alternative baselines in experi- ments. Our contributions can be summarized as follows.
Original graph
hn
Explanation sub-graph
hg
Intuitive illustration of the distribution shifting problem. The 3-dimensional map repre- sents a trained GNN model f. where (h1, h2) rep- resents the embedding distribution of the graph in two dimensions, and Y represents the prediction value of the graph through f. The red and blue lines represent the distribution of the original train- ing graph set and the corresponding explanation sub-graph set, respectively. The distribution of G* shifts away from the original distribution, resulting in shifted prediction values.
Figure 1:
To our best knowledge, we are the first ex- plain GNN predictions on graph regression tasks. We addressed two challenges explaining the graph regression task: the mutual information estimation in the GIB objective and the distribu- tion shifting problem with continuous decision boundaries.
We proposed novel model with self- supervised learning and the mix-up approach, which can address the two challenges more effectively, and better explain the graph model on the regression tasks compared other baselines.
We designed three synthetic datasets, namely BA-Motif- Volume, BA-Motif-Countigg and Triangles, well real-world dataset called Crippen, which can also be used in future works, evaluate the effectiveness of our regression task explanations. Comprehensive empirical studies on both synthetic and real-world datasets demonstrate that method can provide consistent and concise explanations for graph regression tasks.
2 Related Work and Further Discussions
GNN Explainability The explanation methods for GNN models can be categorized into two types based on their granularity: instance-level (22 and model-level (26), where the former methods explain the prediction for each instance by identifying important sub-graphs, and the latter method aims to understand the global decision rules captured by the GNN. These methods can also be classified into two categories based their methodology: self-explainable GNNs [27 28 and post-hoe explanation methods (23)25),, where the former methods provide both predictions and explanations, while the latter methods use additional model strategy to explain the target GNN. Additionally, CGE (29] (cooperative explanation) generates the sub-graph explanation with the sub- net-network simultaneously, by using cooperative learning. However, it has treat the GNN model as a white box, which is usually unavailable in the post-hoc explanation. Existing methods have only partially addressed the explanation of graph regression tasks and have not fully considered two important challenges: the distribution shifting problem and the limitations of the GIB objective, both of which are key areas our work aims tackle.
GIB Objective The Information Bottleneck (IB) 30 31 provides an intuitive principle for learning dense representations that an optimal representation should contain sufficient information for the downstream prediction task with a minimal size. Based on IB, recent work (32 unifies the most existing post-hoc explanation methods for GNN, such as GNNExplainer 23), PGExplainer (2), with the graph information bottleneck (GIB) principle [15] [6] 32). Formally, the objective of explaining

---

the prediction of on G can be represented by
arg mig (GG al al(G))))
where G is the to-be-explained original graph, G is the explanation sub-graph of G. is the original ground-truth label of G, and is a hyper-parameter to get the trade-off between minimal and sufficient constraints. GIB uses the mutual information (G; G*) ) select the minimal explanation that inherits only the most indicative information from G to predict the label Y by maximizing T(G"; Y), where T(;; G*) avoids imposing potentially biased constraints, such as the size or the connectivity of the selected sub-graphs [15]. Through the optimization of the sub-graph G" provides model interpretation. In graph classification task, a widely-adopted approximation to Eq. in
previous methods 23 24 is:
arg min I(G: - aH(Y)G") ss arg (G;G") + oCE(Y,
where Y and Y", approximated by f(G) and f(G"), is the predicted label of G and G* made by the ro-be-xpplained model f, and the cross-entropy CE(Y, between Y and Y is used to approximate - GT( Y). The approximation is based on the definition of mutual information T(G";Y) ((()-H((* HY))")) with entropy H(Y) being static and independent of the explanation process, minimizing the mutual information between the explanation sub-graph G and Y can be reformulated as maximizing the conditional entropy of Y given G* which can be approximated by CE(Y,Y")
3 Preliminary
Notation and Problem Formulation We use G EV,E; X, A) represent graph from an alphabet G, where V equals to (V1, U2,,... ..., Un represents set of n nodes and & E V x V represents the edge set. Each graph has feature matrix X E |Raxd for the nodes, wherein X, X; E Rixd is the d-dimensional node feature of node Ug. is described by an adjacency matrix A {0, 1] juxn, where AG means that there is edge between node and ey; otherwise. Agj = 0. For the graph prediction task, each graph G3 has label Y₂ C. where k E {I,....N] N represents the number of graphs in the dataset, C is the set of the classification categories or regression values in R, with GNN model f trained make the prediction, i.e., f AX C.
Problem 1 (Post-hoc Instance-level GNN Explanation). Given trained GNN model for arbitrary imput graph G = (V, & X, A), the goal of post-hoc instance-level GNN explanation is to find sub- graph G* that explain the prediction of f on G.
In non-graph structured data, the informative feature selection has been well studied (33 as well in traditional methods, such concrete auto-encoder (34), which can be directly extended to explain features in GNNs. In this paper, we focus on discovering the important sub-graph typologies
following the previous work [23 24 Specifically, the obtained explanation G* depicted by binary mask M E 1]"x" on the adjacency matrix. e.g., G = (V. E; X, A O M @ means elements-we wise multiplication. The mask highlights components of G which are essential for f to make the prediction.
4 Methodology
In this section, we first introduce new objective based on GIB for explaining graph regression tasks. Then showcase the distribution shifting problem in the objective for regression and propose novel framework with the mix-up approach to solve the distribution shifting problem, by incorporating the mix-up approach with self-supervised contrastive learning.
4.1
GIB for Explaining Graph Regression
As introduced in Section in the classification task, I(G"; in Eq. 11 is commonly approximated by cross-entropy CE(Y*, 35 However, it is non-trivial to extend it for regression tasks because
Y is continuous variable and it intractable to compute the cross-entropy CE(Y*, Y) or the mutual information (G";Y), where G* is graph variable with continuous variable Y* as its label.
3

---

4.1.1 Optimizing the Lower Bound of I(G"; Y)
To address the challenge of computing the mutual information T(G"; Y) with continuous Y, we propose a novel objective for explaining graph regression.
Instead of minimizing ; Y) directly, we propose to maximize a lower bound for the mutual information by including the prediction label of G", denoted by Y*, and approximate I(G"; Y) in Eq. with (YY;Y):
arg min I (G; G*) (Y"))
(2)
G
(Y) has the following property, upon which we can approximate Eq.
Property I(Y"; Y) is lower bound of [());
Intuitively, the property of YY) is guaranteed by the chain rule for mutual information and the independence between each explanation instance g" in G An intuitive
demonstration is shown in Figure 2 The proof is shown the Appendix B.I
- 10-10"00) (0)
1(0
4.1.2 Estimating I(Y";) Y) with InfoNCE
Figure 2: Intuitive illustration about why T(G"; Y) 2 T(Y"; Y). G contains mutual information having more overlapping area with Y than the over- lapping area between Y* and Y.
Now the challenge becomes the estimation of the mutual information VY) Inspired by the model of Con- trastive Predictive Coding [36), in which InfoNCE loss is interpreted as a mutual information estimator, we further adapt the objective function that it can be applied with InfoNCE loss in explaining graph regression. In our graph explanation scenario, the InfoNCE Loss defined in Eq. (3) can also be utilized lower bound of I(Y"; Y), as shown in the following property with proofs:
Property 2 InfoNCE Loss is a lower bound of the I(Y";
sim (Y*, Y)
T(Y"; Y) 2 log
I Ever sim (Y*, Y")
where Y' is the prediction label of the randomly sampled graph neighbors, Y is the of the neighbors' prediction labels, and sim() estimates the similarity between Y and Y. The proof is shown in the Appendix [.2 Therefore, we have the InfoNCE loss LNCE as the lower bound of the I(Y"; Y). We approximate Eq. ( as:
arg arg (G) aa log sim YY,Y) G-
(4)
IV Ex'er sim (Y*, Y")
4.2 Distribution Shifting Problem in Graph Regression
We include the prediction label Y* in Eq. to estimate similarity, which approximated with J(G)) in previous work [23 24) However, we argue that S(G") cannot be safely obtained due to distribution shift problem
19 classification tasks, small shift may not cross the decision boundaries, which can still lead to correct prediction. However, due to the continuous decision boundarics in regression, the distribution problem would cause serious prediction errors. Here in this paper, the graph distribution is indicated by its regression label in the regression task.
shows ssows the existence of distribution shifts between f f(G*) and fG) in graph regression tasks. For each dataset, we sort the indices of the data samples according to the value of their labels, and visualize the label Y. prediction (G) of the original graph from the trained GNN model f. and prediction f(G") of the explanation sub-gna aub-graph GO from f. As we can see in Figure 3 in all four graph regression datasets, the red points are well distributed around the ground-truth blue points, indicating that /C) is close to Y. In comparison, the green points shift away from the red points, indicating the shifts between j' f(G") ) and (G). Especially dataset Sa--MofffConntiie the sub-graph explanation distribution shifted extremely.
Figure

---

Vatume
Triangles
Crippen
4000
17.5
80
Label
10.0
35000
15.0
70
Explonation "1
gigna
Explanation (0")
3000
12.5
c0
250c
10:0
50
2.5
40
00
4000
30
1500
Original graph
original grapn nGI
20
1030
Explinaation KG")
Explanation nG'1
10
-7.5
25
150
25 50
303
150
205
300
400
500
1230 150
200
softed grach Index
corred graph Indey
corted greph indax
Scrted grapn index
Figure 3: Visualization of distribution shifting problem four graph regression datasets. The points represent the regression value, where the blue points mean ground truth label Y, red points mean prediction f()), and the green points mean prediction f(G*) on the four datasets. The x-axis is the indices of the graph, sorted by the value of the label Y.
Intuitively, this phenomenon indicates the GNN model ʃ can make correct predictions only with the original graph G yet can not predict the explanation sub-graph G* correctly. This is because the GNN model f is trained with the original graph sets, whereas the explanation G as the sub-graph is different from the original graph sets, With the shift between fG) and f(G"), the optimal solution in Eq. unlikely to work well.
4.3 Mix-up Approach with Contrastive Learning
To address this distribution-shifine problem in graph regression, we innovatively incorporate the mix-up approach with a self-supervised contrastive learning strategy. Instead of calculating Y* with f(G*) directly, we approximate with (mix) from f(G which contains similar information as G but is in the same distribution with G. Specifically, our approach includes the following steps:
Step 1 (Neighbor Sampling): Learning through the triplet instances can effectively reinforce the ability of the explainer to learn the explanation self-supervised. For each target graph G with label Y to be explained, we can define two randomly sampled graphs as positive neighbor G and negative neighbor G-,, where G+'s label Y+ closer to Y than G-'s label Y-, i.e., Y YI < -Y--YX - -I. Intuitively, the distance between the distributions of the positive pair (G, G+) should be smaller than the distance between the distributions of the negative pair (G, G-).
Step 2 (Mixup for G'): Then we generate two mixup graphs G(mix)+ and G(mix)- by mixing the sub-gra -obbrraph explanation G with the label irrelevant sub-graph (G+) G (G+ from its positive neighbor G+ and the label irrelevant sub-graph (G-)a )a G (G-)* from negative
neighbor G respectively. Specifically, the label mixup approach calculated from:
(mix))+ G = G + +G+ (G+)*),
( G(mix) G
-
Gmmix)+ and ((tni)) should have the similar information to G because they have the same label- preserving sub-gra sub-graphs G Additionally, considering the following two pairs: (Gúmix)+ G+) and (Gimix) )) The similarity between (Gimix)+ G+ should be larger than the similarity between (Gimix) .G). Intuitively, since (imix)+ and (mmix)- have the same label- preservng sub-graph goappssphhhh G* and Y-- -Y |Y+ - YI. we can have If(G-) [(C(mi)) ((G) represents the prediction label of graph G.
ls(C+) ((Gmii)++ )}, where
Then we can safely estimate the similarity with sim(Y (mmix) .Y.. To save more information, we use the similarity of representation embedding to approximate the similarity of the graph prediction label, where h (mix) represents the embedding for G (mmii) and h represents the embedding for G. We use H to represent the neighbors set accordingly.
Step 3 (InfoNCE Loss Approximation):
Thus, we approximate Eq. as:
I(; - aE log sim (h (mixx) h
min argmin
arg
(5)
H [AT EnEE sim (himix) h!)
Different between Mix-up Approach in Classification Tasks The mix-up approach in previous
work [19] generates mixed graph by simply mixing explanation sub-graph G* with a randomly sampled label- iireeleiteeeeant sub-graph G [19) which can be formally written as Gimix) G + (Go-
5

---

GNNs
Near Neighbor G*
h+
Label = Y
Mixup
G
Imbx+
Graph G
demik
G (G"-("">)
Label =
Mixup
G
(mbx)-
Far Neighbor G
Explainer
Gomik-
G +G (G
Label = Y
Figure 4: Illustration of RegExplainer. G is the to-be-cxplained graph, G and GO are the randomly sampled positive and negative neighbors. The explanation of the graph is produced by the explainer model. Then graph G is mixed with (G+) = G - (G+)* and (G (G-)a - G -(G respectively to produce ( and G Then the graphs
fed into the trained GNN model to retrieve
the embedding vectors h h h (mix)+ and h (mix) where h (mix) z h (mix)- due to the same label- preseeigg sub- uub-graph G We use InfoNCE loss to minimize the distance between (mix) and the positive sample and maximize the distance between G(mix)= and the negative sample. The explainer is trained with the GIB objective and self-supervised contrastive loss.
Gū)). However, it cann't 't tackle the continuous decision boundaries in graph regression tasks. A detailed description of the mix-up approach can be found in Appendix C
4.4
Implementation
InfoNCE Loss After generating the mix-up explanation G(mix), we specify the InfoNCE loss to further train the parameterized explainer with triplet of graphs (G, G G-) In practice, G+ and G are randomly sampled from the graph dataset, upon which we calculate their similarity score with the target graph G. The sample with higher score would be the positive sample and the other
one would be the negative sample. Specifically, we use sim h. ht) h ht' to compute the similarity score, where G' be G+ or G generated by feeding G into the GNN model f and retrieving the embedding vector before the dense layers.
Formally, given target graph G, the sampled positive graph G+ and negative graph G-, formulate the InfoNCE loss in Eq. (5) as the following:
exp((( Th)
(NCE(G, ((GGG,GGG)))
G = -
(6)
exp( (p (mix))+) h+) + exp((((mmix)) h-))
where exp(h h)
is used to instantiate the function sim, the denominator is a sum over the similarities of both positive and negative samples.
Size Constraints We optimize T(G; G*) in Eq. (5) to constraint the size of the explanation sub- graph G*. The upper bound of (G; G*) is optimized as the estimation of the KL-divergence between the probabilistic distribution between the G and G, where the KL-divergence term can be divided into two parts as the entropy loss and size loss [66]. In practice, follow the previous
work (2 24
38 to implement them. Specifically,
Lsize (G, )*)=)) =7 Σ (MG)) -M7))lom loo"
(iJ)EE
where Σ (Mrz) means sum the weights of the existing edges in the edge weight mask M for (K)EEE
the explanation G*: h" extracted from the embedding of the graph G" before the GNN model transforming it into prediction Y*, means the sigmoid function and is the weight for the size of the masked graph. In implementation, we set 7 = (0.0003, 0.3) following previous work [19].
Overall Objective Function In practice, the denominator in Eq. 5 works regularization to avoid trivial solutions. Since the label Y is given and independent of the optimization process, we can also employ the MSE loss between Y* and Y additionally, regarding InfoNCE loss only estimates the

---

Table 1: Illustration of the graph regression datasets together with the explanation faithfulness in terms of AUC-ROC on edges under four datasets on RegExplainer and other baselines. The original graph row visualizes the structure of the complete graph, the explanation row highlights the explanation sub-graph of the corresponding original graph. In the Crippen dataset, different colors of the node represent different kinds of atoms and the node feature is one-hot vector encode the atom type.
Dataset
BA-Motite Volume
BA- HA-Motif-Co Monii-CCuuting
Tringles gee
Crippen
Original Graph G
el 00
Explanation G'
66
Node Feature
Random Flout Vector Sum of Motif Value Fix Size Sub-Graph
Fixed Ones Vector
Fixed Ones Vector
One-hot Vector
Regression Label
Number of Motifs
Number of Triangles
Chemical Property Value
Explanation Type
Dynamic Size Sub-graph
Dynamic Size Sub-graph
Dynamic Size Sub-graph
Explanation AUC
GRAD ATT MixupExplainer
0.418 0.000
0.479 0.000
0.426 + 0.000
0.512 + 0.005
0.521 = 0.003
0.441 0.004
0.502 : 0.006
0.471 = 0.0291
0.868 = 0.127
0.663 0.110
0.499 + 0.002
GNNExplainer
0.501 0.009
0.505 = 0.004
0.500 0.002
0.497 = 0.005
RegExxlaainer
0.588 0.017
0.629 0.001
0.537 0.003
0.541 + 0.011
PGExplainer
0.470 = 0.057
0.798 0.133
0.511 : 0,028
0.448 + 0.005
+RegExplainer
0.758 0.177
0.989 0.003
0.739 0.008
0.553 + 0.013
mutual information between the embeddings. Formally, the overall loss function can be implemented as:
= LGIB + 3fMse(f(G), ( )) where LaIB = Lsizo G*) aLxcE(G, G*, .T)
(8)
Gimio+ means mix G with the positive sample G+ and a and B3 are hyper-parameters. The training algorithm and description of are put Appendix
5
Experiments
In this section, we conduct experiments to demonstrate the performance of our proposed method] These experiments mainly designed to explore the following research questions:
RQI: Can RegExplainer outperforms other baselines explaining GNNs on regression tasks?
RQ2: How does each part of RegExplainer and hyperparameters impact the overall performance in generating explanations?
RQ3: Does the distribution shifting exist in GNN explanation? Can RegExplaiier alleviate it?
5.1
Experiment Settings
We formulate Three synthetic datasets and real-world dataset, as is shown in Table in order to
address the lack of graph regression datasets with ground-truth explanation. The datasets include: BA-Motif-Voluu
and BA-MotiifCounteee
which are based on BA-shapes (23), Triangles [39].
and Crippen 40) We compared the proposed RegExplainer against a comprehensive set of baselines in all datasets, including: GRAD (23). ATT [41), GNNExplainer [23], PGExplainer [24] and Explannel [9. Detailed information about experiment setting are put in the Appendix E
We elaborate on the measurement metric of methods as follows: (1) AUC-ROC: We use the AUC score to evaluate the performance of our proposed methods against baseline methods regarding the ground-truth explanation, which can be treated as a binary classification task. (2) We evaluate the similarity of distribution of the graph with Cosine Similarity and Euclidean Distance.
5.2 Quantitative Evaluation (RQ1)
In this section, we evaluate the performance of our approach with other baselines. For GRAD and GAT, we use the gradient-based and attention-based explanation, following the setting in the
Our data and code available htttp::///ttaab..oo/////////////

---

previous work (23]. We take GCN as our to -o-be-explained model for all post-hoc explainers. For GNNExplainer, PGExplainer, and MixupExplainer, which were previously used for the classification task, we replace the Cross- Cross-Entropy loss with the MSE loss. We run and tune all the baselines on our four datasets. We evaluate the explanation from all the methods with the AUC metric, as done in the previous work. As we can see in [ we take GNNExplainer and PGExplainer as backbones and apply our framework as RegExplainer on both of them. The experiment results demonstrate the effectiveness of our methods in explaining graph regression tasks, where our method achieves the best performance compared to the baselines in all four datasets.
In Table I RegExplainer based on PGExplainer improves the second best baseline with 0.175/344 /34.%% on average and up to 0.246/4 4246/48.0% 48.0%. The comparison between RegExplainer and other baselines indicates the advantages of our proposed approach. This improvement indicates the effectiveness of our proposed method, showing that by incorporating the mix-up approach and contrastive learning, we can generate more faithful explanations in the graph regression tasks. In the following sections, we analyze the RegExplainer with PGExplainer as a backbone.
5.3 Ablation Study and Hyper-parameter Sensitivity Study (RQ2)
We conducted an ablation study show how our proposed components, specifically, the mix- up approach and self-supervised learning, con- tribute to the final performance of RegExplainer RegE
Figure Ablation study of RegExplainer. We evaluated the AUC performance of the original RegExplainer and its variants that exclude the mix- up approach, InfoNCE loss, or MSE loss, respec- tively. The black solid line shows the standard deviation.
To this end, we denote RegExplainer and design three types of variants as follows: (I) RegE mix; We remove the mix-up process- ing after generating the explanations and feed the sub-graph G into the objective function di- rectly. (2) RegE. nce; We remove the InfoNCE loss term but still maintain the mix-up process- ing and MSE loss. (3) RegE-mas We remove the MSE loss computation item from the objec- tive function.
Rogie
legh BeDDaphine
Pagiees
AUC std
Additionally, set all variants with the same configurations as original RegExplainer, includ- ing learning rate, training epochs, and hyper- parameters 7), or and B. We trained them on all four datasets and conducted the results in Fig- ure 5 We observed that the proposed RegExplainer outperforms its variants in all datasets, which indicates that each component is necessary and the combination of them is effective.
SA-Notif-olume
BA Counting
Trangles
Cripgen
We also investigate the hyper-parameters of our approach, which include a and B, across all four datasets. The nyper-parameter & controls the weight of the InfoNCE loss in the GIB objec- tive while the 3 controls the weight of the MSE loss. We determined the optimal values of a and 3 with grid search. The experimental results can be found in Figure 6 We fixed a and | at I and changed another parameter to visual- ize the change in model performance. Figure 6 illustrates that the model's 's performance is ro- bust to changes in hyper parameters within the scope [0.001, 1000). Our findings indicate that our approach. RegExplainer, stable and robust when using different syyper-parametr settings, as evidenced by consistent performance across range.
Figure 6: Jlyper-parameters study of a and ,3 on four datasets with RegExplainer. In both figures, the x-axis is the value of different hyper-parameter settings, and the y-axis is the value of the average AUC score over ten runs with different random seeds.
Tannges
Crippe
0.5
0.4
8

---

Table Prediction shifting study on the RMSE of (f(G),Y), (f(G)) (fG)), respec-
tively.
Dataset
(f
(f(G))
(f(G)
BA-Motif- Volume
131.42
1432.07
1427.07
2.06
7.43
7.22
Triangles
5.28
12.38
12.40
Crippen
1.13
1.54
1.17
5.4 Alleviating Distribution Shifts (RQ3)
In this section, we visualize the regression values of the graphs and calculate the prediction shifting distance for each dataset and analyze their correlations to the distance of the decision boundaries. We put our results into Figure and Table 2
We observed that in Figure 3 red points surround the blue points but green points are shifted away, which indicates that the explanation sub-graph cann't help GNNs make correct predictions. As shown in Table 2 calculate the RMSE score between the f(G) and Y. f(G") and Y. f(G) and f(G*) respectively, where f(G) is the prediction the original graph, f(G is the prediction of the explanation sub-graph, and Y is the regression label. We can observe that f(G") shows significant prediction shifting from f(G) and Y, indicating that the mutual information calculated with (f(G)), Y)) would be biased.
We further explore the relationship of the prediction shifting against the label value with dataset which represents the semantic decision boundary. This additional experiment with Figure can be found in Appendix F.
BA-Motif-
We also design experiments to illustrate how RegExplainer corrects the deviations: we calculate the graph embeddings and predictions p of the explanation sub-graphs and the mix-up graph. Then we compare them to the ground truth and calculate the Euclidean or Cosine distance between the vectors and RMSE between prediction labels. From the results in Table 3 we can observe that all the performances of COS(Ug, Um), EUC(L', vm) and prediction errors are better than those of (Vg. ve), which indicates RegExplainer can effectively fix the distribution of sub-graph explanation G and reduce the embedding distance and prediction error.
Table 3: Table for measuring distribution repairing. lgg. Ve and Um are the embeddings from f of
original graph G, explanation subgraph G" and the mix-up explanation ((mii)+, Pgs Pe and pm the predicted labels for the original graph, explanation subgraph and the mix-up explanation. EUC means Euclidean distance (L. the smaller the better) and COS means cosine distance (1. the larger the better). RMSE means Root Mean Square Error (4. the smaller the better).
Vo-Mooif-Volume Volume BA-Motif- BA-Motif-Counting
Triangles
Crippen
COS(Ei ve)
0.95
0.80
0.97
0.89
COS(Ugr Um)
0.98
0.89
0.99
0.92
EUC(Ug+ Vie)
0,46
0.68
0.19
0.67
EUC(Ug: Um)
0.37
0.52
0.08
0.63
RMSE(Pg. Pe)
1427.07
7.22
12.40
1.17
R.SE((pg Pm)
393.26
2.73
8.22
0.68
Conclusion
We addressed the challenges in the explainability of graph regression tasks and proposed the Reg- Explainer, novel method for explaining the predictions of GNNs with the post-hoc explanation sub-graph on graph regression task without requiring modification of the underlying GNN archi- tecture or re-training. We showed how RegExplainer can leverage the mix-up approach to solve distribution shifting problem and adopt the GIB objective with the InfoNCE loss migrate from graph classification tasks to graph regression tasks, while these existing challenges seriously affect the performances other explainers. We formulated four new datasets: BA-Motif- Volume, PA-Motif-Counting Triangles, and Crippen for evaluating the explainers the graph regression

---

task, which are aligned with the design of datasets in previous work. They can also benefit future studies on the XAIG-R. While we acknowledge the effectiveness of our method, we also recognize its limitations. Specifically, although our approach can be applied to explainers for graph regression tasks in an explainer-agnostic manner, it cannot be easily applied to explainers built for explaining the spatio-temporal graph due to the dynamic topology and node features of the STG. To overcome this challenge, a potential solution is to incorporate cached dynamic embedding memories into the framework.
Ethics Statement
This work is primarily foundational in GNN explainability, focusing expanding the GIB objective function of the explainer framework from graph classification tasks to graph regression tasks. Its primary aim is contribute the academic community by enhancing the explanation graph regression. We do not foresee any direct, immediate, or negative societal impacts stemming from the outcomes of our research.
8 Acknowledgments
The work was partially supported by NSF awards #2421839 and #2331905 The views and conclu- sions contained in this paper those of the authors and should not be interpreted as representing any funding agencies.
References
[1] Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabricle Monfardini. The graph neural network model. IEEE Transactions on Neural Networks, 20(1):61-88) 2009.
[2] Wenqi Fan, Yao Ma, Qing Li, Yuan He, Eric Zhao, Jiliang Tang, and Dawei Yin. Graph neural networks for social recommendation, 2019.
[3] Shengjie Min, Zhan Gao, Jing Peng. Liang Wang. Ke Qin, and Bo Fang. Stgsn - a spa- tial-temporal graph neural network framework for time-evolving social networks. Knowledge- Based Systems, 214:106746, 2021.
[4] Hryhorii Chereda, Annalen Bleckmann, Frank Kramer, Andreas Leha, and Tim Beissbarth. Utilizing molecular network information via graph convolutional neural networks to predict metastatic event in breast cancer. In GMDS, pages 181-186, 2019.
[5] E. Mansimov, O. Mahmood, and S. Kang. Molecular geometry prediction using deep generative graph neural network, 2019.
[6] Xiaoyang Wang, Yao Ma, Yiqi Wang, Wei Jin, Xin Wang, Jiliang Tang, Caiyan Jia, and Jian Yu. Traffic flow prediction via spatial temporal graph neural network. In Proceedings of The Web Conference 2020, WWW '20, page 1082-1092, New York, NY, USA, 2020. Association for Computing Machinery.
[7] Mengzhang Li and Zhanxing Zhu, Spatial-temporal fusion graph neural networks for traffic flow forecasting. Proceedings of the AAAI Conference on Artificial Intelligence, 35(5):4189 4196, May 2021.
[8] Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, and Chengqi Zhang. Graph wavenet for deep spatial-temporal graph modeling. In Proceedings of the 28th International Joint Conference on Artificial Intelligence, IJCAP19, page 1907-1913. AAAI Press, 2019.
[9] Bing Yu, Haoteng Yin, and Zhanxing Zhu. Spatio-temporal graph convolutional networks: deep learning framework for traffic forecasting. In Proceedings of the Twenty-Sevendh Interna- tional Joint Conference on Artificial Intelligence, IJCAI-18, pages 3634-3640, International Joint Conferences on Artificial Intelligence Organization, 7 2018.
10

---

[10] Shakila Shaikh, Sheetal Rathi, and Prachi Janrao. Recommendation system in e-commerce websites: graph based approached. In 2017 IEEE 7th International Advance Computing Conference (IACC), pages 931-934. IEEE, 2017.
[11] Shiwen Wu, Fei Sun, Wentao Zhang, Xu Xie, and Bin Cui. Graph neural networks in recom- mender systems: A survey. ACM Comput. Surv., 55(5), dec 2022.
[12] Kaige Yang and Laura Toni. Graph-based recommendation system. In 2018 IEEE Global Conference on Signal and Information Processing (GlobalSIP), pages 798-802. IEEE, 2018.
[13] Zhaoping Xiong, Dingyan Wang. Xiaohong Liu, Feisheng Zhong, Xiaozhe Wan, Xutong Li, Zhaojun Li, Xiaomin Luo, Kaixian Chen, Hualiang Jiang, et al. Pushing the boundaries of molecular representation for drug discovery with the graph attention mechanism. Journal of medicinal chemistry, 63(16))8749-88 87600 2019.
[14] Pietro Bongini, Monica Bianchini, and Franco Scarselli. Molecular generative graph neural networks for drug discovery. Neurocomputinng, 450:242-252, 2021.
[15] Tailin Wu, Hongyu Ren, Pan Li, and Jure Leskovec, Graph information bottleneck. Advances in Neural Information Processing Systems, 33:20437-20448, 2020.
[16] Siqi Miao, Mia Liu, and Pan Li. Interpretable and generalizable graph learning via stochastic attention mechanism. In International Conference Machine Learning, pages 15524-15543. PMLR, 2022.
[17] Mare Brockschmidt. Gnn-film: Graph neural networks with feature-wise linear modulation, 2020.
[18) Krzysztof Rusek, Paul Almasan, José Suárez- Piotr Cholda, Pere Barlet-Ros, and Albert Cabellos-A Apparios-ppariioo. Fast traffic engineering by gradient descent with learned differentiable routing, 2022.
[19] Jiaxing Zhang, Dongsheng Luo, and Hua Wei. Mixupexplainer: Generalizing explanations for graph neural networks with data augmentation. In Proceedings of 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (SIGKDD), 2023.
[20] Xu Zheng, Farhad Shirani, Tianchun Wang, Wei Cheng, Zhuomin Chen, Haifeng Chen, Hua Wei, and Dongsheng Luo. Towards robust fidelity for evaluating explainability of graph neural networks. In The Twelfth International Conference Learning Representations, 2024.
[21] Zhuomin Chen, Jiaxing Zhang. Jingchao Ni, Xiaoting Li, Yuchen Bian, Md Mezbahul Islam, Ananda Mondal, Hua Wei, and Dongsheng Luo. Generating in-distribution proxy graphs for explaining graph neural networks. In Forty-first International Conference on Machine Learning. 2024.
[22] Caihua Shan, Yifei Shen, Yao Zhang, Xiang Li, and Dongsheng Li. Reinforcement learning enhanced explainer for graph neural networks. M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 22523-22533, Curran Associates, Inc., 2021.
[23] Zhitao Ying, Dylan Bourgeois, Jiaxuan You, Marinka Zitnik, and Jure Leskovec. Gnnexplainer: Generating explanations for graph neural networks. Advances in neural information processing systems, 32, 2019.
[24] Dongsheng Luo, Wei Cheng, Dongkuan Xu, Wenchao Yu, Bo Zong, Haifeng Chen, and Xiang Zhang. Parameterized explainer for graph neural network. Advances in neural information processing systems, 19620-1963911 2020.
[25] Hao Yuan, Haiyang Yu, Jie Wang, Kang Li, and Shuiwang Ji. On explainability of graph neural networks via subgraph explorations. International Conference on Machine Learning, pages 12241-12252. PMLR, 2021.

---

[26] Hao Yuan, Jiliang Tang, Xia Hu, and Shuiwang Ji. XGNN: towards model-level explanations of graph neural networks. In KDD '20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, CA, USA, August 23-27, 2020, pages 4300-38.
ACM, 2020.
[27] Federico Baldassarre and Hossein Azizpour. Explainability techniques for graph convolutional networks, 2019.
[28] Enyan Dai and Suhang Wang. Towards self-explainable graph neural network, 2021.
[29] Junfeng Fang, Xiang Wang, An Zhang, Zemin Liu, Xiangnan He, and Tat-Seng Chua. Coop- erative Explanations of Graph Neural Networks, page 616-624. Association for Computing Machinery, New York, NY, USA, 2023.
[30] Naftali Tishby, Fernando C Pereira, and William Bialek. The information bottleneck method. arXiv preprini physics/0004057, 2000.
[31] Naftali Tishby and Noga Zaslavsky. Deep learning and the information bottleneck principle. In 2015 ieee information theory workshop (itw), pages 1-5. IEEE, 2015.
[32] Junchi Yu, Tingyang Xu, Yu Rong. Yatao Bian, Junzhou Huang, and Ran He. Graph information bottleneck for subgraph recognition. arXiv preprini arXiv:2010.0556. 2020.
[33] Jundong Li, Kewei Cheng, Suhang Wang, Fred Morstatter, Robert P. Trevino, Jiliang Tang, and Huan Liu. Feature selection: A data perspective. ACM Comput. Surv., 50(6), dec 2017.
[34] Muhammed Fatih Balm, Abubakar Abid, and James Zou. Concrete autoencoders: Differentiable feature selection and reconstruction. In International conference on machine learning, pages 444 453. PMLR, 2019.
[35] Zhilu Zhang and Mert Sabuncu. Generalized cross entropy loss for training deep neural networks with noisy labels. Advances neural information processing systems, 31, 2018.
[36] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding, 2019.
[37] Junfeng Fang, Xiang Wang, An Zhang, Zemin Liu, Xiangnan He, and Tat-Seng Chua. Coopera- tive explanations of graph neural networks. In Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining, pages 616-626 2023.
[38] Yaochen Xie, Sumeet Katariya, Xianfeng Tang, Edward Huang. Nikhil Rao, Karthik Subbian, and Shuiwang Ji. Task-agnostic graph explanations, 2022.
[39] Zhengdao Chen, Lei Chen, Soledad Villar, and Joan Bruna. Can graph neural networks count substructures?, 2020.
[40] John S Delaney. Esol:
estimating aqueous solubility directly from molecular structure. Journal of chemical information and computer sciences, 44(3):1000-1000) 2004.
[41] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua 2017.
Bengio. Graph attention networks. arXiv preprint
[42] Scott A Wildman and Gordon M Crippen. Prediction of physicochemical parameters by atomic contributions. Journal of chemical information and computer sciences, 39(5)8868-873, 1999.
[43] Benjamin Menchez-Lengeline Jennifer Wei, Brian Lee, Emily Reif, Peter Wang, Wesley Qian, Kevin McCloskey, Lucy Colwell, and Alexander Wiltschko. Evaluating attribution for graph neural networks. Advances in neural information processing systems, 33:5898-500 -9199, 2020.
12

---

A Symbols Table
Symbol Name
Symbol Meaning
G
Original to-be-explained graph Optimized sub-graph explanation Prediction label for G Prediction label for G* Mutual Information Prediction made by be-beexxpliieed GNN model f The 2-dims representation of graph embeddings Hyper parameter for mutual information term in GIB Information entropy
G"
Y
Y
I0)
f()
(hi, h2)
H()
Node set
Edge set
X
Feature matrix
A
Adjacency matrix
9
Graph set
A node in graph
The i-th node in graph Number of nodes Dimension of feature The i-th node index The j-th node index The k-th graph index edge from node to jode A set of classification categories for regression tasks
d
Agj
M
Edge mask which denotes G in G Set of neighbors' prediction labels Prediction label of sampled graph neighbor Graph embedding for Set of h
Y
H
Hyper parameter for leveraging sum of edge weights Sigmoid function
B
Hyper parameter for MSE loss Embedding vectors for graph G
0g
Ve
Embedding vectors for explanation sub-graph G* Embedding vectors for mix-up graph Gimix++
mm
Pa
Prediction labels for graph G
Pe
Prediction labels for explanation sub-graph G* Prediction labels for mix-up graph G(mx)++ Mapping function from G to Y* An instance of Y An instance of G
Pm
h(.))
y"
Table 4: Important notations and symbols table.
13

---

B Proof
B.1
Property 1
Proof. From the definition Y we can make a safe assumption that there is a many-to-one map (function). denoted by h, from G to Y Y is the prediction label for G For simplicity, assume a finite number of explanation instances for each label y" and each explanation instance, denoted by 9", is generated independently. Then, we have p(y") /gre P(g"), where G(y") {glh(g) = )"} the set of explanations whose labels are y²
Based on the definition of mutual information, we have:
T(G [ PI y) log PI , pc- (g" (py dg
= -] g
y1 ((g"),))
log y h(gg), PC- (0' (o)pp())) (v) dg dgy
J.]. """""""''"""""""
P (g", h((gg))))) ",10"))))) 1)
log PIG h(gg })p (y) dg. dy
=I E PiG Po ..' y. grEblyy)
log P.G 35' 8) P , y² by d
Based on many-to-on assumption, while each g* is generated independently, we know that if é G(y"), then we have PiG ()) ,,",))"""))))"""""""""" = O. Thus, we have:
T(G Y) = Y(G
Σ pn
y v gED(v=)
P(G y dy dy log " (y)
J PIG
y y" 1) log P.G yr "r (y)
dg dy- dy
T(G", YY;
With the chain rule for mutual information, we have T(G Y YY = I(Y: + 1(G; Then due the non-negativity of the mutual information, have T(G", Y*;Y) T(P")). L
B.2 Property 2
Proof. As the InfoNCE method, the mutual information between Y* and Y is defined as:
I(Y";Y)= Σ ,(Y Yog -
P(Y)
Y-Y
However, the ground truth joint distribution p(Y (Y*,, Y) is not controllable, so, we turn to maximize similarity
sim (Y*,Y)
(10)
P(Y)
14

---

We want to put the representation function of mutual information into the NCE Loss
sim (Y*, Y)
CN
--
(11)
Ev Y")
where IN denotes the NCE loss. By inserting the optimal sim ,Y, into Eq. we can get:
P01
LNCE - - log
P(Y)
P(V P(X)
-X)) + Ex p(Yn
1 p(
Σ p( V.
E log
p(Y)
p(Y))
Y
p(
(N 1 p(y .Y)
(12)
2 E log
4
p(Y)
p(Y")
((
E log 4
(N (N-1) 1)
p(Y)
p(
2 E log
N
4
p(Y)
T("", Y) + log(
C Graph Mix-up Approach
To address the distribution shifting issue between f(G) and f(G*) in the GIB objective, we introduce the mix-up approach to reconstruct a within-distribution graph, G Gimix), from the explanation graph G" We follow 24 make riceey-accepted assumption that graph can be divided by G G GA, where G* presents the underlying sub-graph that makes important contributions to GNN's predictions, which is the expected explanatory graph, and G consists of the remaining label-independen edges for predictions made by the GNN. Both G and G influence the distribution of G. Therefore, we the prediction of (Xmmi)) made
need a graph (mmix) that contains both G and GA upon which by ff to approximate Y and h.
Specifically, for target graph G. in the original graph set be explained, generate the explanation sub-graph G = G G from the explainer. To generate a graph in the same distribution original G., can randomly sample graph G, from the original set, generate the explanation sub- sub-graph of G7 with the same explainer and retrieve its label-irrelevant graph G = Go - G Then we can merge G together with G and produce the mix-up explanation Gymix), Formally, we can have Gemis) = G (G,- - GG).
Since we are using the edge weights mask to describe the explanation, we can denote G. and G2 with the adjacency matrices A, and A,, their edge weight mask matrices as M, and M,. If G. and G3 are aligned graphs with the same number of nodes, we can simply mix them up by M( = M (I, M,), where M denotes the weight of the adjacency matrix and I, denotes the zero-ones matrix weights of all edges in the adjacency matrix of Go, where represents the existing edge and represents there is no edge between the node pair.
G. and Go are not aligned with the same number of nodes, we can use connection adjacency ma- trix Acom and mask matrix Moonn merge two graphs with different numbers of nodes. Specifically, the mix-up adjacency matrix can be formed as:
Aa AT conn
Açonn A
A (mix)
(13)
And the mix-up mask matrix can be formed as:
M
Moonn M
M (mix)
M
A
(14)
15

---

@ M) where X (mix) = [Xa; Xo). The detailed algorithm for mix-up shown in Algorithm In implantation, 1j |l * 0.03.
Finally, we can form G as (x (mis). Amiix
Algorithm 1 Graph Mix-up Algorithm
Input: Target co-be-explained graph G. = (Xa; Aa), G, sampled from a set of graphs G, the number of random connections 1), explainer model E.
Output: Graph Gimmi)
1: Generate mask matrix M. E(G.)
2: Generate mask matrix M, = E(G))
3: Sample 7) random connections between G, and Gb A
4: Mix-up adjacency matrix ( with Eq
13
5: Mix-up edge mask M(nix)) with Eq.
6: Mix-up node features X(mxx) [
7: return Gimix) (X (imix) A (mix)
(mix)
D Training Algorithm
Algorithm 2 Training Explainer
Input: A of graphs trained GNN model explainer model E.
Output: Trained explainer E.
Initialize explainer model E.
2: for e E epochs do
3:
for G E do
4:
Gb, Ge +- Randomly sample two graphs from g
5:
G+ G+.G G f- Compare similarity(Ge, G.) to G
6:
G(mix) t Mix-up (G, G+)
7:
G(mix) + Mix-up (G.G-
8: Compute LNCE(G, G+ G-) with Eq. (6)
9: Compute LOi and overall loss with Eq.
10: end for
11: Update E with back propagation.
12: end for
13: return Explainer E
Algorithm showws the training procedure for our explainer. For each epoch and each to-be-explained graph G, we first randomly sample two neighbors and decide the positive neighbor G+ and negative neighbor G according to the similarity between their embedding vectors respectively. The graph with higher similarity to G' is the positive neighbor G+. We generate the explanation for graphs and mix with G and G respectively. We calculate the InfoNCE for triplet (G, ( with Eq. (6) and the GIB loss, which contains the size loss and InfoNCE loss. We also calculate the MSE loss between f +) and fG). The overall loss is the sum of size loss, InfoNCE loss, and MSE loss. We update the trainable parameters in the explainer with the overall loss.
E Implantation details
We provided implementation details for experiments in this section. Data and code are available in the Anumuous git repo and supplementary.
All experiments are conducted on Linux machine (Ubuntu 16.04.4 LTS (GNU/Linux 4.4.0-210- generic x86 64)) with 4 NVIDIA TITAN Xp (12 GB) GPUs. CUDA version 11.8 and the Driver version is 520.56.06. All codes are written with the Python version 3.8.13 with Py Torch 1.12.1 and PyTorch Geometric (PyG) 2. 1.0.tt,, torch-scatter 2.0.9, and torch-sparse 0.6.15. We adopt the Adam optimizer throughout all experiments. Overall, for each dataset, GCN regression model is well-trained first, where we take three-layer GCN model as the backbone. Then the explainers take the to-be-explained GNN and original graph and generate explanations for its prediction on the
16

---

dataset. After that, we evaluate the performance of the explanation. We split the dataset into 8:1:1, where we train the GNN base model with 8 folds, and train and test explainer models with fold respectively. The hyper-parameters are illustrated in the paper correspondingly.
E.1 Datasets
We formulate Three synthetic datasets and real-world dataset, is shown in Table I in order to address the lack of graph regression datasets with ground-trun explanations, (1) BA-Motif- Volume: This dataset based on the BA-shapes [23] and makes a modification, which is adding random float values from [0.00, 100.00] as the node feature. We then sum the node values on the motif as the regression label of the whole graph. which means the GNNs should recognize the [house] motif and then sum features to make the prediction. (2) BA-Morif- BA-MorifCCounting: Different from BA-Motif- Volume, where node features are summarized, in this dataset, we attach various numbers of motifs to the base BA random graph and pad all graphs to equal size. The number of motifs is counted as the regression label. Additionally, we pad base graphs to dynamic size to prevent the GNNs from making trivial predictions based the total number of nodes. (3) Triangles: We follow the previous work [39 to construct this dataset. The dataset is set 5000 Erdōs-Rényí random graphs denoted as ER(m, p). where 30 is the number of nodes in each graph and p 0.2 is the probability for edge to exist. The size of 5000 was chosen to match the previous work. The regression label for this dataset the number of triangles in graph and GNNs
trained count the triangles. (4) Crippen: The Crippen dataset is a real-life dataset that was initially used to evaluate the graph regression task. The dataset has 1127 graphs reported in the Delaney solubility dataset [40] and has weights of each node assigned by the Crippen model (42), which an empirical chemistry model predicting the
water-actual partition coefficient. We adopt this dataset, firstly shown in the previous work [43]. and
construct edge weights by taking the average of the two connected nodes' weights.
E.2 Baselines
We compared the proposed RegExplainer against comprehensive set of baselines in all datasets, including: (1) GRAD [3 GRAD is a gradient-based method that learns weight vectors of edges by computing gradients of the GNN's 's objective function. (2) ATT 41 ATT is graph attention network (GAT) that learns attention weights for edges in the input graph. These weights can be utilized as a proxy measure of edge importance. (3) GNNExplainer [23]: GNNExplainer is model-agnostic method that learns an adjacency matrix mask by maximizing the mutual information between the predictions of the GNN and the distribution of possible sub-graph structures. (4) PGExplainer [24]: PGExplainer adopts a deep neural network to parameterize the generation process of explanations, which facilitates a comprehensive understanding of the predictions made by GNNs. It also produces sub-graph explanations with edge importance masks. (5) MixupExplainer [19]: MixupExplainer adopts the graph mix-up approach with PGExplainer and address the Dfitttr-ittiittiionn problem in graph classification tasks.
F
Additional Experiments
F.1
Correlation between Prediction Shifting and the Label Value
In Figure 7 each point represents graph instance, where Y represents the ground-truth label, and A represents the absolute value difference. It's clear that both the (ff(G"), and (ff ((G)) strongly correlated to Y with statistical significance, indicating the prediction shifting problem is related to the continuous ordered decision boundary, which present regression tasks.
17

---

4000
3000
2000
1000
A(f(G), Y)
-500
-0.99
0.28
A(f(G) ) Y)
2000
A(fG), f(G ))
r
-0.96
== 0.04
0.97
2000
0-
2000
4000-500
2000 A(f(G ") n
0
2000 A(f(G), f(G
A(f(G),
Figure 7: Correlations between the predictions and their shifting on BA-Motif- r indicates the Pearson Correlation Coefficient, and the values with * indicate statistical significance for correlation, where indicates the value for testing non-com correlalaatoon - 0.001. Each point represents one graph instance.
The value of
18

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justifiicton right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]", ", it is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS paper checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We discussed the background scope and introduced our contributions in abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
19

---

Justification:
We provided the discussion about the limitations at the end of conclusion.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model vell-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification:
We provided assumptions and proofs to our theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We provided the flyper-prameee setting.
Guidelines:
The answer NA means that the paper does not include experiments.
20

---

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. w
For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
We provided code and data in supplementary.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (nttps://nips.co
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ /ttppp.cc/pblic/guideeee http://oeeuuiiissooooooooooo
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
21

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g.. data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The experimental setting is presented in the paper and The full details can be found in the appendix and supplemental material.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We conduct experiments in 10 random seeds and report the mean and std values in the performance table.
Guidelines:
The answer NA means that the paper does include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report I-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provided the information of computing resources in the section of imple- mentation details in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
22

---

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethies
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [NA]
Justification: Our research doesn't have concern about the ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations in their jurisdiction).
Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is no socictal impact of the work performed, other than helping human beings better understand the black-box GNNs models.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path any negative applications, the authors should point it out. For example, it is legitimate point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
functioning correctly, harms that could arise when the
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (c.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
23

---

Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging. and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We produce the data by ourselves and the original papers we refer to are cited. The baseline model we used are also cited.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi wwwowwthhde...m caree.com/aee com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We provided the details about our new proposed datasets in the paper.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/mooel as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects

---

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with899 human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
25


Reviews:
[{"review_id": "xT0n1JX4Q1", "reviewer": "Reviewer_KBtu", "summary": "The paper addresses the challenge of interpreting graph regression models, a fundamental yet less explored task in graph learning compared to classification tasks. Existing explanation techniques are predominantly designed for classification, resulting in a gap for regression tasks. Based on the recent advances on information bottleneck and mix-up framework, the author proposes a novel objective to interprete GNN models in regression tasks.", "questions": "1. In Figure 6, why the selection of $\\alpha$ seems do not affect the overall performance? It seems the InfoNCE loss is negeligile in the optimization of the proposed method", "limitations": "N/A", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The Performance of RegExplainer is exceptionally good compared with existing baselines in Table 1, which supports the claim of the paper well. \n\n2. The background knowledge and related work summarization is comprehensive and easy to follow.", "weaknesses": "1. Some of the model designs are not moviated consistently over a pronounced challenge. The paper seems to be a combination of Mixupexplainer, GNNExplainer and G-Mixup. \n\n2. The challgenges of graph regression explainer and distribution shift seem to be independent. The author doesn't make any justification on the graph topology and only embedding/predicted values are reported in Figure 3.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer KBtu, thank you for taking the time to review our work and providing feedback. In the following, we aim to address your questions and concerns.\n\n> Some of the model designs are not moviated consistently over a pronounced challenge. The paper seems to be a combination of Mixupexplainer, GNNExplainer and G-Mixup.\n\nA1: In this work, our goal is to provide reliable explanations for graph regression tasks. Previous methods [1, 2, 3], particularly the vanilla GIB used for graph classification, cannot be trivially applied to graph regression tasks. Therefore, we introduced InfoNCE loss into GIB and theoretically validated its effectiveness. We then recognized that the OOD (out-of-distribution) problem is more severe in graph regression tasks, so we incorporated the mixup method and combined it with InfoNCE loss to develop a contrastive learning objective function that includes contrastive loss. Our model design is cohesive and aims to address a specific problem, rather than simply combining Mixupexplainer, GNNExplainer, and G-Mixup.\n\n> The challgenges of graph regression explainer and distribution shift seem to be independent. The author doesn't make any justification on the graph topology and only embedding/predicted values are reported in Figure 3.\n\nA2: The explainability of graph regression and distribution shift are not independent issues. We found that the distribution shift problem is more severe in graph regression tasks. To generate better explanation subgraphs, we need to correct the subgraph distribution during the explanation process to avoid biased interpretations. Figure 3 illustrates why addressing the distribution shift problem is necessary. It shows that the GNN makes completely incorrect predictions for the BA-Motif-Volume samples, which significantly affects our estimation of $I(Y^*, Y)$ and leads the objective function in the wrong direction. Therefore, we need to address the explainability of graph regression and the distribution shift of subgraphs together.\n\n> In Figure 6, why the selection of \\alpha seems do not affect the overall performance? It seems the InfoNCE loss is negeligile in the optimization of the proposed method\n\nA3: This is because our method exhibits strong robustness to the selection of hyperparameters. However, this does not imply that the InfoNCE loss is negligible. As shown in the ablation study in Figure 5, the performance of the method significantly decreases when the InfoNCE loss module is removed. Therefore, Figures 5 and 6 together demonstrate the robustness and effectiveness of the InfoNCE loss.\n\n[1] Zhitao Ying, Dylan Bourgeois, Jiaxuan You, Marinka Zitnik, and Jure Leskovec. Gnnexplainer: Generating explanations for graph neural networks. Advances in neural information processing systems, 32, 2019.\n\n[2] Dongsheng Luo, Wei Cheng, Dongkuan Xu, Wenchao Yu, Bo Zong, Haifeng Chen, and Xiang Zhang. Parameterized explainer for graph neural network. Advances in neural information processing systems, 33:19620–19631, 2020.\n\n[3] Hao Yuan, Haiyang Yu, Jie Wang, Kang Li, and Shuiwang Ji. On explainability of graph neural networks via subgraph explorations. In International Conference on Machine Learning, pages 12241–12252. PMLR, 2021.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LxbbjNr5ZY", "reviewer": "Reviewer_LTFm", "summary": "The paper introduces XAIG-R, a novel explanation method for interpreting graph regression models. It addresses distribution shifting and decision boundary issues, leveraging the graph information bottleneck theory (GIB) and self-supervised learning.", "questions": "- In this work, GIB is used as Explainer, have you also studied other instance-based approach like GNNExplainer etc.\n-  How does the proposed method handle dynamic graph topology changes, and what are the implications for real-world applications with evolving graph structures? (Directed Graph, Hyper Graph)\n- Any other general-used datasets are tested? Especially those real-world graph datasets.", "limitations": "The authors adequately addressed the limitations.", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Intuitive and clear presentation and illstrration.\n- Previous works have primarily focused on explaining GNN models in classification tasks, leaving a gap in understanding graph regression models. This paper specifically targets the explanation of graph regression tasks, addressing a previously unexplored area.\n- Extensive Experimental Validation with well-designed settings.", "weaknesses": "- Limited Discussion on Computational Efficiency", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer LTFm, thank you for taking the time to review our work and providing feedback. In the following, we aim to address your questions and concerns.\n\n> Limited Discussion on Computational Efficiency\n\nA1: Thank you for pointing this out. We are glad to supplement our analysis of computational complexity, which will also be included in future versions.\nIn the implementation, we transform the structure of the graph data from the sparse adjacency matrix representation into the dense edges list representation. We analyze the computational complexity of our mix-up approach here. \n\nGiven a graph $G_a$ and a randomly sampled graph $G_b$, assuming $G_a$ contains $M_a$ edges and $G_b$ contains $M_b$ edges, the complexity of graph extension operation on edge indices and masks, which extend the size of them from $M_a$, $M_b$ to $M_a+M_b$, is $O(2(M_a+M_b))$, where $M_a>0$ and $M_b>0$. To generate $\\eta$ cross-graph edges, the computational complexity is $O(\\eta)$. For the mix-up operation, the complexity is $O(2(M_a+M_b)+\\eta)$. \nSince $\\eta$ is usually a small constant, the time complexity of our mix-up approach is $O(2*M_a+2*M_b)$.\n\nWe use $M$ to denote the largest number of edges for the graph in the dataset and the time complexity of mix-up could be simplified to $O(M)$.\n\n> In this work, GIB is used as Explainer, have you also studied other instance-based approach like GNNExplainer etc.\n\nA2: GIB is a widely used theoretical foundation in related work. Both GNNExplainer and PGExplainer are based on GIB. In our experiments, we considered both GNNExplainer and PGExplainer and included a comparative study applying our method to GNNExplainer. The results are presented in Table 1 in the paper, as rows \"GNNExplainer\", \"PGExplainer\" and \"+RegExplainer\".\n\n> How does the proposed method handle dynamic graph topology changes, and what are the implications for real-world applications with evolving graph structures? (Directed Graph, Hyper Graph)\n\nA3: Thank you for pointing this out. We also discuss the issue of dynamic graphs in the limitations section. This is a very valuable problem, and we plan to further investigate the explainability of dynamic graph topology, evolving graph structures, and spatio-temporal graphs. This is our direction for future work.\n\n> Any other general-used datasets are tested? Especially those real-world graph datasets.\n\nA4: We would very much like to include more real-world datasets. However, the fact is that graph regression datasets containing ground truth explanation subgraphs are very rare. In this work, we created Crippen[1] as a real-world dataset. We will strive to discover, generate, and use more real-world datasets in future work.\n\n[1] John S Delaney. Esol: estimating aqueous solubility directly from molecular structure. Journal of chemical information and computer sciences, 44(3):1000–1005, 2004.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FsLnaZizza", "reviewer": "Reviewer_B4J7", "summary": "The authors propose an explanation method to interpret the graph regression models. The techniques are built upon the information bottleneck theory and contrastive learning. The authors show that their explanations are accurate in five graph regression datasets. \n\nNote: If authors address my concerns in questions and limitations, I am willing to upgrade my allocated score.", "questions": "Where are the ground truth vectors for your evaluation? How are they obtained? How can ground truth for explanations be in the datasets? This is the most important part of your paper, and it is left to the imagination of readers. \n\nI argue that your technique is better because you add more samples, and the surrogate is learning just that. So basically, you can add this sampling to those explanations, and you don't need the rest of your method (evidence in Section 5.3 Figure 5 except for the BA-motiv dataset). Can you argue against this concern? \n\nWhy are the ablation results different for BA-Motiv-Volume? \n\nWhat is the real limitation of your technique? You have written, \"Instead, they write, \" Specifically, although our approach can be applied to explainers for graph regression tasks in an explainer-agnostic manner, it cannot be easily applied to explainers built for explaining the spatio-temporal graph due to the dynamic topology and node features of the STG.\" What does this mean?", "limitations": "I personally think the main limitation is that this approach is an add-on approach. I am not in favor of add-on model agnostic approaches. What I mean is explanations that sit on top of other explanation techniques. I think this makes the design of these techniques extra complicated, and finding faults in explanations becomes harder. I think if the authors take their sampling and improve it, it can replace the GNN and PGE and become a more general approach. \n\nThe authors have also not stated that the graph regression task is not a very popular task in graph settings. Based on this, I also would like to see how this approach can be extended for use in other graph tasks: node classification etc.", "rating": 5, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "Explanation of graph neural networks and, particularly, explaining graph regression is both a relevant and interesting problem. \n\nThe major parts of the proposed technique are clearly explained and easily understood in detail.\n\nThe proposed method relies on some solid theoretical properties like information bottleneck.\n\nThe code is released with the paper, which helps the work to be reproducible.", "weaknesses": "The paper is densely written in Section 4 and is hard to follow. Instead of explaining each part of the algorithm separately and clearly explaining why these components exist, the authors rely too much on theorizing the problem. For example, I am unsure if a reader is particularly interested in knowing about lower and upper bounds and distribution shifts before seeing the proposed method, and maybe moving this section away as motivation can solve this problem. \n\nThe authors have mentioned that their evaluation relies on ground truth, but the measures for evaluating explanations are barely discussed. See questions below.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer B4J7, thank you for taking the time to review our work and providing feedback. We appreciate your thorough review and aim to address your questions and concerns in detail. Due to character limitations, we will provide a detailed response to the remaining points in the official comment section. Please let us know if you have any further questions or need additional clarification, we are happy to discuss with you.\n\n> The paper is densely written ... problem. \n\nA1: Thank you very much for your suggestion. In this work, our goal is to provide reliable explanations for graph regression tasks.  Previous methods [1, 2, 3] used the vanilla GIB for graph classification, which cannot be trivially applied to graph regression tasks. Therefore, we introduced InfoNCE loss into GIB and theoretically validated its effectiveness. Next, we explain that the OOD problem is more severe in graph regression tasks. On this basis, we introduce the mixup method and combine it with InfoNCE loss to propose our method and model, which includes a contrastive learning objective function with contrastive loss. We aim to balance the effectiveness of the model design with the reliability of the theoretical foundation. In future versions, we will improve the organization of our paper, provide more detailed explanations of the model design, and move some of the theoretical derivations to the appendix.\n\n> Where are the ground truth vectors for your evaluation? How are they obtained? How can ground truth for explanations be in the datasets? This is the most important part of your paper, and it is left to the imagination of readers.\n\nA2: (1). What the explanation subgraphs look like: As introduced in Section 3 Preliminary, line 119 - line 120, we follow the setup from previous work[1, 2], using a binary edge mask to represent the explanation subgraph of the original graph. Specifically, for each edge in the original graph, our explainer produces a prediction value. If this value is 1, it indicates that the edge is part of the explanation; if it is 0, it means the edge is not relevant to the explanation. For each graph, the mask containing these edge weight values is a vector.\n\n(2)&(3). How the ground truth for explanation subgraphs is obtained: Establishing the ground truth for explanation subgraphs is a critical step. We adopt the approach from previous work, including both synthetic and real datasets. In synthetic datasets like BA-motif-volume, we designed the dataset such that the label is related to the motifs and their corresponding features within the graph. Therefore, the corresponding motif subgraph is the explanation subgraph and is used to evaluate the explainer's performance. Additionally, we designed a graph regression task with ground truth explanation subgraphs based on the chemical dataset Crippen. More detailed information can be found in Appendix E.1.\n\n(4). How we evaluate the explainer’s performance and what metrics are used: As mentioned earlier, the explainer produces a weight for each edge in the graph and determines whether the edge belongs to the explanation subgraph based on this weight. In practice, the edge weights are floating-point numbers, and we assess the accuracy of the explanation by calculating the AUC-ROC with respect to the ground truth, thus evaluating the explainer’s performance.\n\n(5). Additional Information: In some other works, evaluation methods without ground truth are used, such as fidelity/sparsity scores to estimate the quality of the explanation subgraph. However, these evaluation methods face issues with out-of-distribution (OOD) problems. Therefore, we did not use datasets and metrics without ground truth in this work. We plan to introduce more datasets, both with and without ground truth, in future work to better evaluate our method and facilitate related research.\n\n> I argue ... concern?\n\nA3: From my understanding, your concern means that you believe the improvement in explainer performance comes from sampling and mixup, and thus the contrastive learning and contrastive loss components are unnecessary. Let me clarify this point; if my understanding is incorrect, I would be happy to further discuss and clarify the issue with you.\n\nFirst, as mentioned in A1, our method aims to explain graph regression tasks, whereas previous methods were based on vanilla GIB and focused on explaining graph classification tasks. By introducing InfoNCE loss, we can better explain graph regression tasks. The sampling and mixup are employed to address the OOD (out-of-distribution) problem during the explanation process. The combination of these components is crucial for effectively improving the model's performance.\n\nAs shown in Figure 5 of the ablation study, simply using the mixup method and sampling (dropping the InfoNCE module) leads to a decrease in model performance. Therefore, just sampling is not sufficient to fully address the problem.\n\n[1] Zhitao Ying, Dylan Bourgeois, Jiaxuan You, Marinka Zitnik, and Jure Leskovec. Gnnexplainer: Generating explanations for graph neural networks. Advances in neural information processing systems, 32, 2019.\n\n[2] Dongsheng Luo, Wei Cheng, Dongkuan Xu, Wenchao Yu, Bo Zong, Haifeng Chen, and Xiang Zhang. Parameterized explainer for graph neural network. Advances in neural information processing systems, 33:19620–19631, 2020.\n\n[3] Hao Yuan, Haiyang Yu, Jie Wang, Kang Li, and Shuiwang Ji. On explainability of graph neural networks via subgraph explorations. In International Conference on Machine Learning, pages 12241–12252. PMLR, 2021.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3P9OdRWP2b", "reviewer": "Reviewer_X7ek", "summary": "This work proposes a method to generate instance-level GNN prediction explanations specifically for graph regression tasks. This method addresses distribution shifting, a problem in regression, by using mix-up for contrastive learning. The work is evaluated on four datasets, both synthetic and real-world.", "questions": "How does this method perform with/against classification explanation models? As the explainer can be modularly applied to existing trained GNNs, this would be interesting to see if it can improve the accuracy of explanation generations. This work [1] is a good work on instance and model-level explanations for graph classification tasks.\n\nIs graph mix-up performed on the graphs explicitly or within latent space? If the mix-up occurs in graph space, then how is the subgraph G* combined with (G^+)^{Delta} and (G^-)^{Delta}? How are edges added between these different subgraphs? Furthermore, for more complex regression datasets, naively adding edges can drastically change the label for each graph. Is this strategy then limited to graph regression datasets which inherently rely on graph structures to derive labels? If the mix-up occurs within latent space, then how does this work differentiate itself from graph rationalization works [2, 3]? If the mix-up occurs in latent space, then additional studies to compare against graph rationalization baselines should be included.\n\n[1] Xuanyuan, Han, et al. \"Global concept-based interpretability for graph neural networks via neuron analysis.\" Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 37. No. 9. 2023.\n\n[2] Wu, Ying-Xin, et al. \"Discovering invariant rationales for graph neural networks.\" arXiv preprint arXiv:2201.12872 (2022).\n\n[3] Liu, Gang, et al. \"Graph rationalization with environment-based augmentations.\" Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2022.", "limitations": "The work requires an accurate trained prediction model. The model explanations are not used to retrain the GNN in any way.\n\nAdditional limitations are addressed in the conclusion.", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The work provides thorough theoretical justification for the objective function in Eq. (4). Additionally, it creates a clever mix-up-based contrastive learning approach that directly addresses the distribution shift problem specific to regression.", "weaknesses": "More real-world datasets would demonstrate this method’s more general applicability. The evaluation is limited by the inclusion of only one real-world dataset, Crippen.\n\nSee questions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer X7ek, thank you for taking the time to review our work and providing feedback. In the following, we aim to address your questions and concerns.\n\n> How does this method perform with/against classification explanation models? As the explainer can be modularly applied to existing trained GNNs, this would be interesting to see if it can improve the accuracy of explanation generations. This work [1] is a good work on instance and model-level explanations for graph classification tasks.\n\n\nA1: Yes, we have compared our method with several existing methods for graph classification, including GRAD, ATT, MixupExplainer, GNNExplainer, and PGExplainer, as shown in Table X. It is important to note that these methods were originally designed for graph classification tasks and use cross-entropy loss in their objective functions, which cannot be directly applied to graph regression tasks. Therefore, we adapted them for comparison by using MSE loss.\n\nWork [1] is an excellent study that explores graph neural network interpretability from a novel perspective. Our method could potentially be adapted to the framework presented in their Equation 6 and might improve the performance of their approach. We look forward to incorporating and citing this method as a baseline in our future work. \n\n> Is graph mix-up performed on the graphs explicitly or within latent space? If the mix-up occurs in graph space, then how is the subgraph G* combined with $(G^+)^{\\Delta}$ and $(G^-)^{\\Delta}$? How are edges added between these different subgraphs? \n\nA2: We describe the mix-up process in detail in Appendix C. The mix-up is performed on the graph explicitly by mixing edge weights, and different subgraphs are connected through randomly sampled connection edges.\n\n> Furthermore, for more complex regression datasets, naively adding edges can drastically change the label for each graph. Is this strategy then limited to graph regression datasets which inherently rely on graph structures to derive labels? \n\nA3: Yes, in this work, we follow the settings of previous studies and focus primarily on the graph structure without incorporating edge features. We are interested in exploring the impact of edge features on our method's performance in future research.\n\n> If the mix-up occurs within latent space, then how does this work differentiate itself from graph rationalization works [2, 3]? If the mix-up occurs in latent space, then additional studies to compare against graph rationalization baselines should be included.\n\nA4: Our mix-up primarily occurs on edge weights. We will consider more related works[2, 3] and incorporate their strengths and citations in future studies.\n\n>Limitations: The work requires an accurate trained prediction model. The model explanations are not used to retrain the GNN in any way.\n\nA5: Thank you for your suggestion. This work primarily focuses on post-hoc explanations. In future work, we will explore methods that involve retraining the GNN to enhance both the GNN and the explainer's performance.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eqSzPagt3f", "reviewer": "Reviewer_4dqr", "summary": "The authors focus on the interpretability of GNN in graph regression tasks. They propose a novel explanation method called RegExplainer as a plug-in to existing explanation methods, such as GNNExplainer and PGExplainer. They also tackle mutual information estimation (graph information bottleneck), distribution shifting and continuously ordered decision boundaries that hinder current explanation techniques.", "questions": "Please address the concerns in the \"Weakness\".", "limitations": "The authors show the limitations on Sec. 6. \n\nHowever, as mentioned above, the assumption and proof of Property 2 should be carefully considered.", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The writing flow is clear and easy to follow.\n\n2. The illustration in Figure 3 is clear and straightforward.", "weaknesses": "1. The **contribution** is confusing:\n\n1a. In the introduction (Lines 64-68), the authors mention *datasets* as a key contribution. However, the paper lacks detailed descriptions of these datasets in later sections. More information about the datasets, including their structure and significance, should be included to clarify this contribution.\n\n2. Concerning the **methods** and corresponding **experimental settings**:\n\n2a. The *subgraph connectivities* are not considered. In other words, RegExplainer could result in isolated nodes rather than \"a subgraph\".\n\n2b. **Property 2 is overclaimed**. Lines 469-471 dismiss the ground truth joint distribution $p(Y^*,Y)$, and only focus on the remaining parts. The assumption weakens the paper, especially since \"distribution shifting\" is a key discussion of the paper.\n\n2c. Section 4.3, especially Step 2, is hard to follow. The process for obtaining $(G^+)^{\\Delta}$ and $G^*$ needs more detail. Specifically, Step 2 is used to approximate $I(G, G^*)$ and find the optimal $G^*$ as in Eq. (4) or (5). However, $G^{(mix)}$ relies on $(G^+)^{\\Delta}$ and $G^*$. Given that $G^*$ is the variable to be optimized, how to pick $(G^+)^{\\Delta}$ is missing.\n\n2d. Whether $G^*$, $(G^+)^{\\Delta}$ ,as well as $G^{mix}$ are binary {0,1} or continuous in [0,1] is unclear. If they are binary, the optimization for $G^*$ needs more explanation. If they are continuous, the relaxation and recovery to the final outcomes should be specified. \n\n2e. The method of measuring distances between graphs or graph distributions is not clearly defined. The authors should clarify and justify whether MSE/RMSE are used, or another metric is needed. \n\nBesides, the authors evaluate the distribution shifting by similarity/distance between embeddings (Table 3) rather than other distribution-based metrics.\n\n2f. The last column in Table 2 shows a large distance from the original prediction, suggesting the *subgraph may not be faithful*. \n\n2g. The problem definition in Lines 113-115 is unclear, how to define \"can\" explain?\nThe problem definition in Lines 113-115 is unclear, particularly the criteria for an explanation to be considered valid (e.g., “can explain the prediction”).\n\n3. **Others**:\n\n3a. The method appears suited for graph-level regression tasks. Could it apply to the node-level regression tasks, such as traffic prediction?\n\n3b. \"N\" in Line 110 is undefined.\n\n3c. What does \"regression label in regression task\" mean on Line 164?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer 4dqr, thank you for taking the time to review our work and providing feedback. In the following, we aim to address your questions and concerns.\n\nA-1a: Thank you for your suggestion. We will include detailed descriptions of the datasets in future versions of the paper. In the appendix, we describe how these datasets were created and the rationale behind their creation.\n\nA-2a: Thank you for pointing this out. Subgraph connectivity is indeed an important issue. However, in this work, our main goal was to extend GIB-based explainers to graph regression tasks, following the settings of previous works in other aspects. We are keen to explore and address the connectivity of explanatory subgraphs in future research.\n\nA-2b: We incorporate the InfoNCE loss into the GIB objective with reference to previous work[1], where InfoNCE is a lower bound of mutual information (MI), and there is no constraint on the distribution of variables. Thank you for pointing out the issue in the appendix. We will provide a more detailed explanation in the next version.\n\nA-2c: We are happy to address your concern. In Step 1 of Section 4.3, we describe in detail how to select $G^+$ and $G^-$. “We can define two randomly sampled graphs as positive neighbor $G^+$ and negative neighbor $G^−$, where $G^+$’s label $Y^+$ is closer to $Y$ than $G^−$’s label $Y^−$, i.e., $|Y^+ − Y| < |Y^− − Y|$.”\n\nA-2d: They are continuous. The final explanatory subgraph is obtained by selecting the top-k edges based on their weights.\n\nA-2e: Thank you for pointing this out. We will correct the typo in Table 3 in the next version of the paper. We use RMSE as the metric for measuring distances between graphs or graph distributions. Besides, metrics like KL-divergence can be challenging to compute in practice. Therefore, we use graph embeddings and prediction labels to measure distribution shifts between original graphs and explanation subgraphs.\n\nA-2f: The purpose of Table 2 is to demonstrate that the explanatory subgraphs can indeed be out-of-distribution (OOD). The results in the last column indicate that these subgraphs are not sufficiently faithful and can introduce bias in the explaining. To address this issue, we introduced the mixup method to mitigate the OOD problem of the explanatory subgraphs.\n\nA-2g: \"Can explain\" means that the explanation should identify the reasons behind the GNN's prediction. For example, in the defined datasets, different motifs and their corresponding features lead to different labels. The GNN relies on these motifs to make predictions, and the goal of the explainer is to identify these motifs as part of the explanation.\n\nA-3a: Yes, our method can be extended to node-level regression tasks. In the case of a three-layer GCN, node-level tasks essentially involve graph tasks centered on 3-hop subgraphs around each node. Therefore, our method can be easily adapted for node-level tasks. We look forward to incorporating more datasets in future work.\n\nA-3b: Thank you for pointing this out. \"N\" represents the number of graphs in the dataset. We will include this clarification in the next version of the paper.\n\nA-3c: The term \"regression label\" refers to the prediction made by the GNN for a given graph or subgraph.\n\n[1] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation Learning with Contrastive\nPredictive Coding.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Dear Reviewers,

We sincerely appreciate your time, consideration, and valuable comments, which have been instrumental in refining our work. If you have any further questions or concerns regarding our response or the current draft, please let us know. We are more than happy to discuss them in detail.

Meta-review: This paper carefully crafted a new method for explanation of predictions of graph regression neural networks. The method utilizes a novel objective based on information bottle neck theory and new mix-up framework; it is an add-on (post training) model agnostic method. 

This paper is well written and easy to follow. Several reviewers praise the theoretically justified objective that was proposed. While experiments are a weak part of this paper, as mentioned by several reviewers,  synthetic datasets performance seems really convincing. The area of explainers for regression GNN is under-explored and will certainly benefit from this paper. Authors provide the code as an artifact, that will further increase the adoption of this work.
Furthermore, authors did an excellent job rebutting, resulting in several raised scores.  

The paper should be strengthened by adding new experimental results on real world datasets.
Decision: Accept (poster)