SlowGuess/NeurIPS_2023_OCR huggingface dataset:
So far, we have derived the estimizable representation of  $G_{S}$  in Formula (8). However, the optimization is still challenging since the distributions  $p(\hat{y} |G_S)$  and  $p(G_S)$  are intractable.
Fortunately, following the variational approximation proposed in [35], we can derive a tractable variational upper bound of GIB in Formula (8). For the first term  $-\mathrm{MI}(\hat{y},G_S)$ , a parameterized variational approximation  $p_{\phi}(\hat{y}|G_S)$  is introduced for  $p(\hat{y}|G_S)$  to get the upper bound as follows,
$$
- \operatorname {M I} (\hat {y}, G _ {S}) \leq - \mathbb {E} _ {p \left(G _ {S}, \hat {y}\right)} \left[ \log p _ {\phi} (\hat {y} \mid G _ {S}) \right] + H (\hat {y}), \tag {13}
$$
where  $p_{\phi}(\hat{y} | G_S)$  is the GNN model and  $H(\hat{y})$  is an entropy independent of  $G_S$ . For the second term, we introduce  $q(G_S)$  for the marginal distribution  $P(G_S)$  [28], and the upper bound is
$$
\operatorname {M I} \left(G _ {S}, G\right) \leq \mathbb {E} _ {p (G)} \left[ \mathrm {K L} \left(p _ {\alpha} \left(G _ {S} \mid G\right) \| q \left(G _ {S}\right)\right) \right], \tag {14}
$$
where  $p_{\alpha}(G_S|G)$  represents the explanation generator and  $q(G_S)$  represents the prior distribution sharing a similar spirit of assuming standard Gaussian prior [28]. See Appendix A for the detailed derivation. Finally, we obtain the variational upper bound of the GIB Formula (8) as follows,
$$
L _ {\mathrm {G I B}} = - \mathbb {E} _ {p \left(G _ {S}, \hat {y}\right)} \left[ \log p _ {\phi} (\hat {y} \mid G _ {S}) \right] + \beta \mathbb {E} _ {p (G)} \left[ \mathrm {K L} \left(p _ {\alpha} \left(G _ {S} \mid G\right) \| q \left(G _ {S}\right)\right) \right], \tag {15}
$$
where  $q(G_S)$  is usually set as
$$
q \left(G _ {S}\right) = C \cdot \prod_ {i, j = 1} ^ {N} p _ {\pi} \left(e _ {i j}\right), e _ {i j} \sim \operatorname {B e r n} (\pi), \tag {16}
$$
where  $C$  is a constant that decided by the hyper-parameter  $\pi$ .
Note that there are no rigorous constraints imposed in Formula (15). The second term actually functions as an adaptive constraint that inhibits  $G_{S}$  from containing useless information. The risk of ignoring the irregular severe corruptions in the final explanation  $G_{S}$  can also be largely mitigated.
To achieve a robust GNN explainer for all the corruption situations, the two proposed modules are trained jointly by minimizing the following overall objective function,
$$
L _ {\text {j o i n t}} = L _ {\mathrm {V A E}} + L _ {\mathrm {G I B}}. \tag {17}
$$
Following the standard procedure of GNN explanation [15, 2], we conduct two steps in the experiments: (1) training a base GNN model for prediction, and (2) generating explanations for this base model. For the first step, the base GNN model is trained to simulate the third-party predictor which is independent of the GNN explainers. We take the trained GNN model as an oracle during the explanation generation, i.e., input the graph (no matter clean or corrupted) and get the prediction. In the second step, we simulate the corruptions by introducing random noise or adversarial attack to the graph structures. The random noise represents the corruptions that naturally exist in real-world which affects each graph component without any distinction. The noise ratio controls the percentage of randomly flipped edges. The adversarial attack represents the man-made malicious corruptions and the attack budget is the highest percentage of attacked edges.
Metric. For a quantitative explanation evaluation, we report the probability of sufficient  $P_{S}$ , the probability of necessary  $P_{N}$  and the  $F_{NS}$  scores. See Appendix D.1 for the detailed definition.
Dataset. We evaluate the proposed V-InfoR and baseline explanation methods on one synthetic dataset and three real-world datasets. The synthetic dataset is BA-3Motifs introduced in [36]. Three real-world datasets are Mutag, Ogbg-molhiv and Ogbg-ppa.
Baseline. The comparable baseline explainers include gradients-based methods GradCAM [21] and IG [22], surrogate method PGM-Explainer [37], and perturbation-based methods GNNExplainer [15], PGExplainer [2] and ReFine [36].
The detailed descriptions of datasets, baselines, and base GNN models are given in Appendix D. The ablation study is presented in Appendix E. We also report the visualized cases of GNN explanation for qualitative analysis in Appendix F.
4.2 Quantitative Evaluation on Graphs with Random Noise
We report the result on the graphs with  $20\%$  random structural noise corruptions in Table 1. Specifically, we randomly select the edges according to the noise ratio and flip the selected ones.
The result shows that V-InfoR is able to improve the explainer performance on the four datasets, with the overall metric  $F_{NS}$  score improvement at least by  $9.19\%$  for the BA-3Motifs dataset and the highest improvement by  $29.23\%$  for the Mutag dataset. The V-InfoR improves the performance remarkably for Mutag by  $29.23\%$  and Ogbg-molhiv datasets by  $15.65\%$ , since the irregularity of the corruptions in chemical molecule structure is more obvious than the other types of graphs (motifs in BA-3Motifs and biologically associations in Ogbg-ppa), and V-InfoR is able to effectively capture
Table 1: The comparison of V-InfoR and baselines under random structural noise. We use bold font to mark the highest score. The second highest score is marked with underline. The Impro. is defined as ([V-InfoR]-[Best Baseline])/[Best Baseline].
Dataset	Metric	GradCAM	IG	GNNExplainer	PGExplainer	PGM-Explainer	ReFine	V-InfoR	Rank	Impro.
BA-3Motifs	\(P_S\)	0.8725	0.8625	0.8535	0.8510	0.8505	0.8300	0.8820	1	1.09%
\(P_N\)	0.2605	0.2795	0.2410	0.2095	0.2235	0.2625	0.3021	1	8.09%
\(F_{NS}\)	0.4012	0.4222	0.3758	0.3362	0.3540	0.3989	0.4610	1	9.19%
Mutag	\(P_S\)	0.8760	0.8880	0.8916	0.8640	0.8900	0.8900	0.8964	1	0.54%
\(P_N\)	0.0996	0.1068	0.1080	0.1260	0.1020	0.1260	0.1696	1	34.60%
\(F_{NS}\)	0.1789	0.1907	0.1920	0.2199	0.1830	0.2207	0.2852	1	29.23%
Ogbg-molhiv	\(P_S\)	0.9230	0.9200	0.8925	0.8390	0.8860	0.9105	0.9386	1	1.69%
\(P_N\)	0.0680	0.0400	0.0940	0.1265	0.0980	0.1020	0.1470	1	16.21%
\(F_{NS}\)	0.1267	0.0767	0.1701	0.2198	0.1765	0.1834	0.2542	1	15.65%
Ogbg-ppa	\(P_S\)	0.4340	0.5820	0.6616	0.6260	0.6192	0.6344	0.6700	1	1.27%
\(P_N\)	0.4720	0.4600	0.3480	0.2856	0.3780	0.4406	0.4930	1	4.45%
\(F_{NS}\)	0.4522	0.5139	0.4561	0.3922	0.4694	0.5200	0.5680	1	9.23%
Figure 3: The comparison of V-InfoR and six baselines under different noise ratios in (a) BA-3Motifs and (b) Mutag.
the irregularity. For the BA-3Motifs and Ogbg-ppa datasets with less obvious irregularity, V-InfoR can still achieve a performance improvement, since the information constraint is general enough to cover the rigorous constraint and thus avoid performance degradation.
One can also see that the raw features based explainers, including GradCAM, IG, GNNExplainer, and PGM-Explainer, perform poorly in small/medium-scale datasets (BA-3Mmotifs, Mutag and Ogbg-molhiv). This may be because the raw features are more easily to be corrupted in small/medium-scale graphs. Since the graph representation is more difficult to learn in the large-scale graph, PGExplainer and ReFine that reliance on latent graph representations are unable to well explain large-scale graphs for the Ogbg-ppa dataset. V-InfoR adopts the robust graph representation which is sampled from a variational distribution containing common information shared by large amounts of graphs, and thus achieves an overall better performance. Furthermore, one can note that all baseline GNN explainers achieve a high  $P_{S}$ , while the  $P_{N}$  is relatively low. This phenomenon implies that the sufficient condition is easier to satisfy than the necessary condition.
As shown in Figure 3, we also evaluate V-InfoR and baselines under different noise ratios ranging from 0 to 0.30, which reversals the tendency of explainer performance with the increase of noise ratio. In real application scenarios, it is intractable to separate the minor and the severe corruptions, and both may exist simultaneously. Different noise ratios indicate different mixing ratios of the two corruptions. Note that  $F_{NS}$  scores with zero noise ratio represent the result on raw graphs without any corruption. It shows that V-InfoR still achieves the best performance when no noise is added (noise ratio = 0). This reveals that even without any corruption, the robust representation extractor still extracts vital common graph information and the explanation generator adaptively identifies the explanations with regular structural properties.
Table 2: The comparison of V-InfoR and baselines under GRABNEL attack [18]. We use bold font to mark the highest score. The second highest score is marked with underlines.
Attack budegt	Dataset	Metric	GradCAM	IG	GNNExplainer	PGExplainer	PGM-Explainer	ReFine	V-InfoR	Rank	Impro.
5%	BA-3Motifs	PS	0.6980	0.6925	0.5625	0.6225	0.5950	0.6700	0.7075	1	1.36%
PN	0.3625	0.4675	0.4200	0.3700	0.3925	0.3925	0.5450	1	16.58%
FNNS	0.4772	0.5582	0.4809	0.4641	0.4730	0.4950	0.6157	1	10.30%
Mutag	PS	0.5740	0.6600	0.6140	0.6610	0.5820	0.6340	0.6760	1	2.27%
PN	0.4200	0.3875	0.3800	0.4003	0.4060	0.4100	0.4588	1	9.24%
FNNS	0.4851	0.4883	0.4695	0.4986	0.4783	0.4980	0.5466	1	9.63%
10%	BA-3Motifs	PS	0.8720	0.8495	0.8605	0.9020	0.8125	0.8800	0.9185	1	1.83%
PN	0.0800	0.2105	0.2615	0.2815	0.1925	0.2100	0.3332	1	18.37%
FNNS	0.1466	0.3374	0.4011	0.4291	0.3113	0.3391	0.4890	1	13.96%
Mutag	PS	0.5848	0.7370	0.6616	0.6524	0.6392	0.6140	0.7424	1	0.73%
PN	0.4160	0.3404	0.3284	0.3928	0.3344	0.4040	0.4277	1	2.81%
FNNS	0.4862	0.4657	0.4389	0.4904	0.4391	0.4873	0.5427	1	10.66%
4.3 Quantitative Evaluation on Graphs with Attack Corruption
The adversarial attack corresponds to the malicious corruption painstakingly customized by the attacker. Here we use the SOTA graph classification attack algorithm GRABNEL [18], whose attack budget is the highest percentage of attacked edges. GRABNEL employs a surrogate model to learn the mapping from an attacked graph to its attack loss and optimizes the attacked graph iteratively via an adapted genetic algorithm until successful attack or budget exhaustion. Considering the overhead of executing GRABNEL, we run the attack algorithm only on the BA-3Motifs and Mutag datasets. We report the results with  $5\%$  and  $10\%$  attack budgets in Table 2.
The result reveals that InfoR achieves an overall superior performance over the other baselines in terms of all metrics. As the attack budget grows from  $5\%$  to  $10\%$ , though the GNN explanation task becomes harder (all metrics decrease), the performance improvement of V-InfoR becomes more significant than baselines. This shows that V-InfoR can effectively mitigate the negative influence of the structural corruptions. For the Mutag dataset, the performance improvement on the graphs with GRABNEL attack  $(13.96\%)$  is less significant than that on the graphs with random noise  $(29.23\%)$ . This gap can be attributed to the difference between adversarial attack and random noise. Although the adversarial attack corruption is devastating to change the model prediction, they are subtle and introduce as less irregularity as possible to the raw graphs.
4.4 Hyper-parameter analysis
We further analyse the effect of three parameters on the model performance,  $\tau$ ,  $\beta$  and  $\pi$ .  $\tau$  controls the approximation degree of  $r_{ij}$  distribution to Bernoulli distribution, which ranges in [0.1, 0.5].  $\beta$  controls the balance between the strength of information restoring (i.e.,  $\min - \mathrm{MI}(\hat{y}, G_S)$ ) and the strength of information filtering (i.e.,  $\min \mathrm{MI}(G_S, G)$ ).  $\pi$  represents the prior Bernoulli probability, which controls the distribution of  $q(G_S)$ .
Figure 4 shows the influence of the three hyper-parameters on V-InfoR for the BA-3Motifs and Mutag datasets. One can roughly archive the following three conclusions. First, V-InfoR is not so sensitive to  $\beta$  that controls the strength of AGI constraint, which verifies the adaptability of our proposed constraint. Second, a suitable value of  $\tau$  is around 0.3, which means the best balance between the continuity of Formula (10) and the approximation degree is achieved when  $\tau = 0.3$ . Third, there is no obvious pattern shown in Figure 4(c) for the choice of  $\pi$  in Formula (16), which means that a reasonable  $\pi$  may largely depend on the specific dataset.
Early attempts to explain GNN simply transfer gradients-based methods to graph data, and they regard the gradients of nodes and edges as significance scores [14]. Though efficient and intuitive, the explanations based on gradients are unavoidably suffered from gradients saturation [25]. Surrogate methods [37, 23] are also adopted in GNN explanations. Yet limited by simple models that function as surrogates, these approaches are unable to capture topological structure which plays an important role in GNN predictions [13, 27]. Currently, the most concerned GNN explanation methods are perturbation-based. By intervening components in the graph, such as deleting (or adding) edges
Figure 4: The influence of hyper-parameters  $\beta$  in Formula (8),  $\tau$  in Formula (10) and  $\pi$  in Formula (16) on V-InfoR.
(or nodes), and monitoring the change of the corresponding prediction, perturbation-based GNN explanation methods [15, 13, 24, 38] optimize the significance scores matrix round after round.
As a classic perturbation-based GNN explanation method, GNNExplainer [15] determines the significant components by maximizing the mutual information between the intervened and the original graphs. It calculates significance scores for both node features and edges, and the components whose scores are below-threshold will be removed. PGExplainer [13] introduces a parameterized neural network to provide significance scores. It demands a training procedure to endow the internal neural network with multi-categorical predicting behavior, and the trained PGExplainer can explain any new graph without retraining. While the previous methods aim to preserve the components that make the prediction invariant, CF-GNNExplainer aims to find the components that will make the prediction change if they are missing [24]. XGNN [39] formulates the GNN explanation problem as a reinforcement learning task. Starting with an empty graph, XGNN gradually adds components until the generated graph belongs to the specified class.
In this paper, we propose a robust GNN explainer V-InfoR for the structurally corrupted graphs. V-InfoR employs the variational inference to learn the robust graph representations and generalizes the GNN explanation exploration to a graph information bottleneck (GIB) optimization task without any predefined rigorous constraints. The robust graph representations are insensitive to minor corruptions when extract the common information shared by the observed graphs. By introducing an adaptive graph information constraint, V-InfoR can effectively capture both the regularity and irregularity of the explanation subgraphs. Extensive experiments demonstrate its superior performance and the ablation study further illustrates the effectiveness of two proposed modules.
[40] Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: datasets for machine learning on graphs. In Proceedings of NeurIPS, 2020.
[41] Ekagra Ranjan, Soumya Sanyal, and Partha Pratim Talukdar. Asap: Adaptive structure aware pooling for learning hierarchical graph representations. In Proceedings of AAAI, 2020.
[42] Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In Proceedings of ICLR, 2019.
[43] Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In Proceedings of ICLR, 2017.
[44] Negin Entezari, Saba A. Al-Sayouri, Amirali Darvishzadeh, and Evangelos E. Papalexakis. All you need is low (rank): Defending against adversarial attacks on graphs. In Proceedings of WSDM, 2020.
[45] Wei Jin, Yao Ma, Xiaorui Liu, Xianfeng Tang, Suhang Wang, and Jiliang Tang. Graph structure learning for robust graph neural networks. In Proceedings of KDD, 2020.
A Detailed Derivation
First, we give the detailed derivation of Formula (4). To maximize the similarity between the variational distribution and the true posterior distribution, we minimize the Kullback-Leibler divergence,
$$
\begin{array}{l} \operatorname {K L} [ q (\mathbf {z}; \mathbf {G}) | | p (\mathbf {z} _ {\mathbf {i}} | \mathbf {G}) ] \\ = \sum_ {\mathbf {z} _ {\mathrm {i}}} q (\mathbf {z} _ {\mathrm {i}}; \mathbf {G}) \log \frac {q (\mathbf {z} _ {\mathrm {i}} ; \mathbf {G})}{p (\mathbf {z} _ {\mathrm {i}} | \mathbf {G})} \\ = \sum_ {\mathbf {z} _ {\mathrm {i}}} q \left(\mathbf {z} _ {\mathrm {i}}; \mathbf {G}\right) \log \frac {q \left(\mathbf {z} _ {\mathrm {i}} ; \mathbf {G}\right)}{p \left(\mathbf {z} _ {\mathrm {i}} , \mathbf {G}\right)} + \sum_ {\mathbf {z} _ {\mathrm {i}}} q \left(\mathbf {z} _ {\mathrm {i}}; \mathbf {G}\right) \log p (\mathbf {G}) \tag {18} \\ = \sum_ {\mathbf {z} _ {\mathrm {i}}} q (\mathbf {z} _ {\mathrm {i}}; \mathbf {G}) \log \frac {q (\mathbf {z} _ {\mathrm {i}} ; \mathbf {G})}{p (\mathbf {z} _ {\mathrm {i}} , \mathbf {G})} + \log p (\mathbf {G}). \\ \end{array}
$$
Since  $p(\mathbf{G})$  is independent of  $\mathbf{z}$ , we get the following derivation,
$$
\begin{array}{l} \sum_ {\mathbf {z} _ {\mathrm {i}}} q (\mathbf {z} _ {\mathrm {i}}; \mathbf {G}) \log \frac {q (\mathbf {z} _ {\mathrm {i}} ; \mathbf {G})}{p (\mathbf {z} _ {\mathrm {i}} , \mathbf {G})} \\ = \sum_ {\mathbf {z} _ {\mathrm {i}}} q \left(\mathbf {z} _ {\mathrm {i}}; \mathbf {G}\right) \log \frac {q \left(\mathbf {z} _ {\mathrm {i}} ; \mathbf {G}\right)}{p \left(\mathbf {G} \mid \mathbf {z} _ {\mathrm {i}}\right) \cdot p \left(\mathbf {z} _ {\mathrm {i}}\right)} \tag {19} \\ = \sum_ {\mathbf {z} _ {\mathrm {i}}} q (\mathbf {z} _ {\mathrm {i}}; \mathbf {G}) \log \frac {q (\mathbf {z} _ {\mathrm {i}} ; \mathbf {G})}{p (\mathbf {z} _ {\mathrm {i}})} - \sum_ {\mathbf {z} _ {\mathrm {i}}} q (\mathbf {z} _ {\mathrm {i}}; \mathbf {G}) \log p (\mathbf {G} | \mathbf {z} _ {\mathrm {i}}) \\ = \operatorname {K L} [ q (\mathbf {z}; \mathbf {G}) | | p (\mathbf {z}) ] - \mathbb {E} _ {q (\mathbf {z}; \mathbf {G})} [ \log p (\mathbf {G} | \mathbf {z}) ]. \\ \end{array}
$$
Next, we deduce the upper bound of  $-\mathrm{MI}(\hat{y}, G_S)$  in Formula (13),
$$
\begin{array}{l} - \operatorname {M I} (\hat {y}, G _ {S}) = - \mathbb {E} _ {p (\hat {y}, G _ {S})} \Big [ \log \frac {p (\hat {y} , G _ {S})}{p (\hat {y}) \cdot p (G _ {S})} \Big ] \\ = - \mathbb {E} _ {p (\hat {y}, G _ {S})} \left[ \log \frac {p (\hat {y} | G _ {S})}{p (\hat {y})} \right] \\ = - \mathbb {E} _ {p (\hat {y}, G _ {S})} \left[ \log \frac {p _ {\phi} (\hat {y} \mid G _ {S})}{p (\hat {y})} \right] - \mathbb {E} _ {p (\hat {y}, G _ {S})} \left[ \log \frac {p (\hat {y} \mid G _ {S})}{p _ {\phi} (\hat {y} \mid G _ {S})} \right] \tag {20} \\ = - \mathbb {E} _ {p (\hat {y}, G _ {S})} \left[ \log \frac {p _ {\phi} (\hat {y} | G _ {S})}{p (\hat {y})} \right] - \mathbb {E} _ {p (G _ {S})} \left[ \mathrm {K L} \left[ p (\hat {y} | G _ {S}) | | p _ {\phi} (\hat {y}, G _ {S}) \right] \right] \\ \leq - \mathbb {E} _ {p (\hat {y}, G _ {S})} \left[ \log \frac {p _ {\phi} (\hat {y} | G _ {S})}{p (\hat {y})} \right] \\ = - \mathbb {E} _ {p (\hat {y}, G _ {S})} [ \log p _ {\phi} (\hat {y} | G _ {S}) ] + H (\hat {y}). \\ \end{array}
$$
At last, we deduce the upper bound of  $\mathrm{MI}(G_S,G)$  in Formula (14),
$$
\begin{array}{l} \operatorname {M I} (G _ {S}, G) = \mathbb {E} _ {P (G _ {S}, G)} \left[ \log \frac {p (G _ {S} , G)}{p (G _ {S}) \cdot p (G)} \right] \\ = \mathbb {E} _ {P (G _ {S}, G)} \left[ \log \frac {p _ {\alpha} (G _ {S} | G)}{p (G _ {S})} \right] \\ = \mathbb {E} _ {P (G _ {S}, G)} \left[ \log \frac {p _ {\alpha} (G _ {S} | G)}{q (G _ {S})} \right] + \mathbb {E} _ {P (G _ {S}, G)} \left[ \log \frac {q (G _ {S})}{p (G _ {S})} \right] \\ = \mathbb {E} _ {P (G _ {S}, G)} \left[ \log \frac {p _ {\alpha} (G _ {S} | G)}{q (G _ {S})} \right] + \mathbb {E} _ {P (G | G _ {S})} \left[ p (G _ {S}) \log \frac {q (G _ {S})}{p (G _ {S})} \right] \tag {21} \\ = \mathbb {E} _ {P (G _ {S}, G)} \left[ \log \frac {p _ {\alpha} (G _ {S} | G)}{q (G _ {S})} \right] + \mathbb {E} _ {P (G | G _ {S})} \left[ - \mathrm {K L} \left[ p (G _ {S}) | | q (G _ {S}) \right] \right] \\ \leq \mathbb {E} _ {p (G)} \left[ p _ {\alpha} \left(G _ {S} | G\right) \log \frac {p _ {\alpha} \left(G _ {S} | G\right)}{q \left(G _ {S}\right)} \right] \\ = \mathbb {E} _ {p (G)} \left[ \mathrm {K L} \left[ p _ {\alpha} \left(G _ {S} | G\right) | | q \left(G _ {S}\right) \right] \right]. \\ \end{array}
$$
B Basic Notations
The basic notations and descriptions are summarized in Table 3
Table 3: Basic notations and descriptions in the manuscript.
Notation	Description
G, Gk	Graph instance, k-th graph instance
GS	Explanatory subgraph
X, Xs	Node feature matrix
A, As	Adjacency matrix
G	Graph set
φ	GNN prediction model
ŷ	GNN predicted label
c, C	Specific label, label space
ŷS	GNN prediction label of explanatory subgraph
|·|	Cardinality of set
z, zk	Continuous random variable, k-th sampled value
N(μ, σ2)	Gaussian distribution with μ mean and σ standard deviation
rij	Bernoulli variable of edge (i, j)
θij	Bernoulli parameter of edge (i, j)
αij	Latent parameter of edge (i, j)
τ	Temperature parameter in Concrete distribution
ε	Random noise following N(0, 1)
MP	Probability matrix
C Preliminary Experiment
The results of six SOTA GNN explainers on the graphs with minor and severe corruptions is shown in Figure 5. The detailed analysis is reported at Section 2.1.
Figure 5: The  $F_{NS}$  scores of the six SOTA GNN explainers on BA-3Motifs, BA-3Motifs with minor corruptions, and BA-3Motifs with severe corruptions, respectively.
D Experiment supplement
D.1 Metric
We formally define the three metrics for explanation evaluation as follows,
$$
P _ {S} = \frac {\sum \mathbb {I} (\tilde {y} = \hat {y})}{| \mathcal {G} _ {\text {t e s t}} |}, \text {w h e r e} \tilde {y} = \underset {c \in \mathcal {C}} {\operatorname {a r g m a x}} P _ {\phi} (c | \mathbf {X} _ {\mathbf {S}}, \mathbf {A} _ {\mathbf {S}}), \tag {22}
$$
$$
P _ {N} = \frac {\sum \mathbb {I} (\tilde {y} \neq \hat {y})}{| \mathcal {G} _ {\text {t e s t}} |}, \text {w h e r e} \tilde {y} = \underset {c \in \mathcal {C}} {\operatorname {a r g m a x}} P _ {\phi} (c | \mathbf {X} - \mathbf {X} _ {\mathbf {S}}, \mathbf {A} - \mathbf {A} _ {\mathbf {S}}), \tag {23}
$$
$$
F _ {N S} = \frac {2 \cdot P _ {S} \cdot P _ {N}}{P _ {S} + P _ {N}}, \tag {24}
$$
where  $\mathcal{G}_{\mathrm{test}}$  is the test dataset.  $P_{S}$  measures how faithfully the explanations in simulating the full graph,  $P_{N}$  measures the indispensability of explanations, and  $F_{NS}$  score is the Harmonic mean of  $P_{S}$  and  $P_{N}$ , which measures the overall performance of a GNN explanation method.
We next briefly introduce the four datasets. The statistics of the four datasets are shown in Table 4, where the "avg nodes/edges" means the average node/edge number of the graphs and the "split" is the dataset splitting of the training, validation, and testing sets.
- BA-3Motifs [36] contains 3,000 synthetic graphs. Each graph adopts Barabasi Albert graphs as the base and attaches each base with one of three motifs: house, cycle, and grid. The synthetic graphs are classified into three classes according to the type of attached motifs.
- Mutag [6] contains 4,337 molecule graphs which are classified into two categories in accordance with their mutagenic effect on a bacterium.
- Ogbg-molhiv [40] contains 41,127 molecule graphs which are classified into two categories based on whether a molecule inhibits HIV virus replication or not.
- Ogbg-ppa [40] contains 158,100 protein-protein association networks in which nodes represent proteins and edges indicate biologically meaningful associations. These association networks cover 1,581 species that belong to 37 taxonomic groups (e.g., mammals and bacterial families). The prediction task is a 37-classes classification to predict which taxonomic group the network is from.