Title: Evaluating Post-hoc Explanations for Graph Neural Networks via Robustness Analysis
Venue: NeurIPS 2023
Accepted: yes
Keywords: Post-hoc Explainability

Explanation Evaluation

Graph Neural Network

Robustness Analysis
Abstract: This work studies the evaluation of explaining graph neural networks (GNNs), which is crucial to the credibility of post-hoc explainability in practical usage. Conventional evaluation metrics, and even explanation methods -- which mainly follow the paradigm of feeding the explanatory subgraph and measuring output difference -- always suffer from the notorious out-of-distribution (OOD) issue. In this work, we endeavor to confront the issue by introducing a novel evaluation metric, termed **O**OD-resistant **A**dversarial **R**obustness (OAR). Specifically, we draw inspiration from the notion of adversarial robustness and evaluate post-hoc explanation subgraphs by calculating their robustness under attack. On top of that, an elaborate OOD reweighting block is inserted into the pipeline to confine the evaluation process to the original data distribution. For applications involving large datasets, we further devise a **Sim**plified version of **OAR** (SimOAR), which achieves a significant improvement in computational efficiency at the cost of a small amount of performance. Extensive empirical studies validate the effectiveness of our OAR and SimOAR.

Evaluating Post-hoc Explanations for Graph Neural Networks via Robustness Analysis

Abstract

This work studies the evaluation of explaining graph neural networks (GNNs), which is crucial to the credibility of post-hoc explainability in practical usage. Conventional evaluation metrics, and even explanation methods - which mainly follow the paradigm of feeding the explanatory subgraph to the model and measuring output difference - mostly suffer from the notorious out-of-distribution (OOD) issue. Hence, in this work, we endeavor to confront this issue by introducing a novel evaluation metric, termed OOD-resistant Adversarial Robustness (OAR). Specifically, we draw inspiration from adversarial robustness and evaluate post-hoc explanation subgraphs by calculating their robustness under attack. On top of that, an elaborate OOD reweighting block is inserted into the pipeline to confine the evaluation process to the original data distribution. For applications involving large datasets, we further devise a Simplified version of OAR (SimOAR), which achieves a significant improvement in computational efficiency at the cost of a small amount of performance. Extensive empirical studies validate the effectiveness of our OAR and SimOAR. Code is available at https://github.com/MangoKiller/SimOAR_OAR .

1 Introduction

Post-hoc explainability has manifested its extraordinary power to explain graph neural networks (GNNs) [1, 2, 3, 4]. Given a GNN-generated prediction for a graph, it aims to identify an explanatory subgraph, which is expected to best support the prediction and make the decision-making process more credible, fair, and understandable [5, 6, 7]. However, the reliable evaluation of explanation quality remains a key challenge. As a primary solution, Human supervision seeks to justify whether the explanations align with human knowledge [8, 9], but it is often too subjective, thus hardly providing quantifiable assessments. Another straightforward solution is quantitatively measuring the agreement between the generated and ground-truth explanations, such as Precision and Recall [10, 11]. Unfortunately, access to the ground truth is usually unavailable and labor-extensive, thereby limiting the scope of evaluations based on this method.

Recently, a compromised paradigm Feature Removal [12, 13] - has been prevailing to quantitatively evaluate the explanation's predictive power as compared to the full graph, without exploiting the human supervision and ground truth. The basic idea is to first remove the unimportant features and feed the remaining part ( i.e., explanatory subgraph) into the GNN, and then observe how the prediction changes. The prediction discrepancy instantiates Accuracy [8] and Fidelity [9] of the

Input

Explanation

Input

Explanation

Input

Constrainedattack+OODscore

?

(H)

H

?

H

N①

C1)

C1

C

C

CI

?

C

N

H

(H

0.3

?

0.2

GNN

GNN

GNN

GNN

Explanation

'mutagenic'

'mutagenic'

'mutagenic'

'mutagenic'

(H)

H

compare

compare

×

C1

0.2

CI

?

0.3

Metric:feed theexplanationsolely

Metric:complementtheexplanationvia

GNN

GNN

andcomparetheoutcome.

generative model and compare the outcome.

mutagenic'

0.68

0.40

Data

Region where

@S

②

?

train

generative

model

0.54

Distribution

Input

Subgraph

★个

H

C

Dataset

@

HO

?

'mutagenic spurious

C1

C

mutagenic'

'non-

0.68×0.3+0.40×0.2+

0

correlation

0.54x0.2+0.27×0.3=0.46

Caveat:subgraph is out of

Caveat:thegenerativemodel alwaystakes-

Metric:generateinstancesviaconstrained

distributionwheremodel is trained.

NH2 as the infilling part given Cl-.

attackandreweightthembyOODscores.

(a) removal-based evaluation

(b) generation-based evaluation

(c) our OAR evaluation

explanation, reflecting 'how accurate and faithful the explanation is to recover the prediction of the input graph'. Despite the prevalence, these removal-based metrics usually come with the caveat of the out-of-distribution (OOD) issue [14, 13]. Specifically, as the after-removal subgraphs are likely to lie off the distribution of full graphs [15, 16], the GNN is forced to handle these off-manifold inputs and easily gets erroneous predictions [17, 18]. Take Figure 1 (a) as an example. For the full molecular graph, the GNN classifies it as 'mutagenic', which is reasonable due to the presence of mutagenic -NH 2 group; whereas, when taking the subgraph, i.e., non-mutagenic C-Cl group solely as the input, the GNN surprisingly maintains its output 'mutagenic'. Clearly, the prediction on the explanatory subgraph might be skeptical, which easily deteriorates the faithfulness of the removal-based evaluations.

In sight of this, recent efforts [18, 17] are beginning to mitigate the OOD issue via the Generationbased metrics. Instead of directly feeding the to-be-evaluated subgraph into the target GNN, they use a generative model [19, 20] to imagine and generate a new full graph conditioned on the subgraph. These methods believe that the generation process could infill the subgraph and pull it closer to the original graph distribution. As Figure 1 (b) shows, comparing the predictions on this new graph and the original graph could be the surrogate evaluation. While intuitively appealing, the generative models easily inherit the data bias and inject it into the infilling part. Considering Figure 1 (b) again, since molecules in the Mutagenicity dataset comprising non-mutagenic chloride ( -Cl) always carry amino ( -NH 2 ), a generative model is prone to capture this occurrence bias and tend to infill -NH 2 with the -Cl-involved subgraphs. This bias not only exerts on the generations but also makes the evaluation inconsistent with the GNN behavior: in the generative model, -Cl is assigned with more 'mutagenic' scores as it usually accompanies the mutagenic partner -NH 2 ; in contrast, the GNN finds no 'mutagenic' clues in -Cl. In a nutshell, the generation-based metrics show respect to the data distribution somehow but could be inconsistent with GNNs' behavior and lose control of the infilling part.

Scrutinizing these removal- and generation-based metrics (as summarized in Figure 1), we naturally raise a question: ' Can we design a metric that respects the data distribution and GNN behavior simultaneously? ' To this end, we draw inspiration from adversarial robustness [21, 22] and propose a new evaluation framework, OAR (OOD-resistant Adversarial Robustness), to help reliably assess the explanations. As shown in Figure 1 (c), OAR encapsulates two components: constrained attack and OOD reweighting, which respect the GNN behavior and data distribution, respectively. Specifically:

Intuitively, perturbations on label-irrelevant features should be ineffective to the GNN prediction, while those on label-relevant features are supposed to be impactful and destructive to the prediction [22, 21]. Hence, for the original input graph, the attack model performs perturbations constrained on the complementary part of its explanation. This perturbing game aims to naturally take control of the "infilling" process, making the explanatory subgraph less influenced by its infilling.

Having obtained a set of perturbed graphs, the reweighting component estimates the OOD score of each perturbed graph, which reflects the degree of distribution shift from the original data

distribution. Then, we feed these graphs into GNN and reweight their predictions with OOD scores. The sum of weighted predictions can quantitatively evaluate the importance of the target subgraph.

We validate the effectiveness of OAR in evaluation tasks across various start-of-the-art explanation methods, datasets, and backbone GNN models. OAR has manifested surprising consistency with the metrics like Precision , Recall and Human Supervision . Furthermore, to better generalize to the large datasets, we also provide a Simplified version of OAR (SimOAR) achieving significant improvements in computational efficiency at the expense of a small amount of performance degradation. Our main contributions can be summarized as follows:

We propose a novel metric, OAR for evaluating GNNs' explainability, which tries to resolve the limitation of current removal- and generative-based evaluations by taking both data distribution and GNN behavior into account (Section 2.2).

We provide a simplified version of OAR, SimOAR for better generalization to the evaluation tasks involving large datasets, which greatly shortens the execution time while only sacrificing a small amount of performance (Section 2.3).

Experimental results demonstrate that our OAR/SimOAR outperforms the current evaluation metrics by a large margin, and further validate the high efficiency of SimOAR (Section 3).

2 Methodology

In this section, we propose an evaluation method for the explainability of GNNs from the perspective of adversarial robustness. We start with the notation of GNNs and its explainability in Section 2.1. After that, we detail our evaluation metric, OAR via three progressive steps (Section 2.2). In Section 2.3, we provide a simplified version of OAR called SimOAR for applications demanding more efficient execution.

2.1 Problem Formulation

Graph neural networks (GNNs). GNNs have achieved remarkable success due to their powerful representation ability. Without loss of generality, we focus on the graph classification task in this work: a well-trained GNN model f takes a graph G as the input and outputs its probabilities y over classes { 1 , ..., C } , i.e., y = f ( G ) ∈ R C . Typically, G is an undirected graph involving the node set V and the edge set E . We first introduce the feature of node v i ∈ V as a d -dimensional vector and collect the features of all nodes into X ∈ R |V|× d . Then we define an adjacency matrix A ∈ R |V|×|V| to describe graph topology, where A uv = 1 if the edge connecting nodes u and v exists, i.e., ( u, v ) ∈ E , otherwise A uv = 0 . Based on these, G can be alternatively represented as G = ( A , X ) .

Explainability for GNNs. Upon the GNN model f , explanation techniques of GNNs generally study the underlying relationships between their outputs y and inputs G . They focus on explainability w.r.t. input features, aiming to answer ' Which parts of the input graph contribute most to the model prediction? '. Towards this end, explainers typically assign an importance score to each input feature ( i.e. , node v i or edge ( v i , v j ) ) to trace their contributions. Then they select the salient part ( e.g. , a subset of nodes or edges with top contributions) as the explanatory subgraph G s and delete the complementary part G ¯ s = G\G s . We formulate the explanation method as h and yield the above process as G s = h ( G , f ) .

2.2 OOD-Resistant Adversarial Robustness

Retrospecting the removal- and generation-based evaluations, we emphasize that both these classes come with inherent limitations. Specifically, Removal-based metrics pay less heed to the data distribution thus forcing GNNs to handle off-manifold instances, while generation-based metrics are inconsistent with GNN behavior and lose control of the infilling part. Fortunately, in this section, we claim that it is possible to get the best of and avoid the pitfalls of both worlds - removal-based and generation-based metrics - by taking both GNN behavior and data distribution into account.

To meet these challenges, we elaborate our evaluation metric, OOD-resistant adversarial robustness (OAR) via three progressive steps: in the first step , we formulate the adversarial robustness tailored for GNNs explanations, which naturally conforms to the GNN behavior; in the second step , we introduce a tractable and easy-to-implement objective of above adversarial robustness; in the third step , we introduce an elaborate OOD reweighting block which confines the overall evaluation process to the original data distribution.

STEP 1: Formulation of Adversarial Robustness. We prioritize the introduction of adversarial robustness in machine learning [21, 22, 23] that motivates our method. Concretely, given a machine

Input

H

ConstrainedAttack

modelbehavior

datadistribution

Evaluation

H

H

H

CI

H

0.68

0.40

0.46

0.3

0.2

Explanation

GNN

sum

C1

C

NE

fix

0.54

0.27

0.20.3

compare

(H)

H

H

®

H

Dataset

OOD reweighting block

reciprocal of

?

H

encoder

decoder

reconstruction

loss: 1/Lrecon

original

H

H

train

VGAE

learning model, an input x and a subset of the input x s ⊆ x , the adversarial robustness of x s denotes the minimum perturbation leading to the wrong prediction, on the condition that perturbation is only imposed on x s [22]. Inspired by this idea, we define the adversarial robustness of GNN explanation G s , and formulate it as the minimum adversarial perturbation δ on the structure of complementary subgraph G ¯ s . More formally,

Definition 1. Given a GNN model f , an input graph G = ( A , X ) with prediction y and explanation G s , suppose that G ′ = ( A ′ , X ′ ) is the graph generated by adding and deleting edges in G , the adversarial robustness δ of explanation G s is defined as:

̸

where V and V s are the node sets of G and G s , respectively.

Definition 1 identifies the quality of explanation G s as the difficulty of reversing the prediction by perturbing features not belonging to G s solely. That is, when G s is fixed, the more difficult it is to fool the model by perturbing its complementary, the more important G s is. The key intuition behind this inference is: if an explanation comprises most of the label-relevant features, it is conceivably hard to change the prediction by manipulating the remaining features that are label-irrelevant. Thus, according to Definition 1 , we can find that: a good explanation would yield high adversarial robustness δ and vice versa, which naturally conforms to the GNN behavior.

It seems that adversarial robustness is the feasible metric to evaluate the GNNs' explanations. However, there are still two matters standing in the way of its adoption: 1) Is its objective ( i.e. , Equation (1)) tractable and easy to implement? 2) Does it respect the data distribution?

STEP 2: Finding a Tractable Objective. To answer the first question, we argue that Equation (1) is hard to realize and sometimes even intractable owing to two possible reasons:

The primary reason is that adversarial attacks may fail to find any adversarial example, since a solution satisfying two conditions in Equation (1) simultaneously may not exist. In other words, if the explanation G s is precise enough, it is almost impossible to reverse the prediction via manipulating features in the complementary part G ¯ s which are mainly label-irrelevant.

It is notoriously hard to search for the minimum adversarial perturbation δ in most cases. Current attack methods [24, 25] typically turn to find an alternative sub-optimal solution. Thus, leveraging these methods could introduce additional bias and threaten the fairness of evaluation.

To address these issues and make the evaluation objective tractable and easy to implement, we first formulate the inference derived from Definition 1 :

Proposition 1. When the high quality explanation G s is anchored (fixed), perturbations restricted to the complementary part G ¯ s have a weak influence on the model prediction.

While Definition 1 evaluates the explanations via the adversarial robustness δ , Proposition 1 indicates a more straightforward way to the tractable evaluation objective from its dual perspective. To be more specific, Definition 1 quantifies perturbation on G ¯ s causing change of prediction; conversely, Proposition 1 quantifies change of prediction caused by perturbation on G ¯ s . More formally,

Definition 2. Given a GNN model f , an input graph G = ( A , X ) with prediction y and explanation G s , suppose that G ′ = ( A ′ , X ′ ) is the graph generated by adding and deleting edges in G , the approximate adversarial robustness δ ∗ of G s is defined as:

where V s refers to the node set of G s ; c denotes the predicted class of G by model f ; f ( G ′ ) c represents the probability value of f ( G ′ ) for the given class c .

Stemmed from Definition 2 , the evaluation method to quantify the adversarial robustness of GNN explanations is more explicit and computationally convenient. As shown in Figure 2, for the to-beevaluated subgraph G s ( i.e. , C-Cl in red dotted box), we anchor it and randomly perturb the remain part G ¯ s to get graphs G ′ ( i.e. , molecules in green dotted box). Then we compare the expectation of the prediction of G ′ with the prediction of the original graph G . If they are close, most features in G ¯ s must be label-irrelevant. Hence, we can assign high quality for the explanation G s .

So far, there is only one question left: how to ensure that the aforementioned evaluation process respects the data distribution?

STEP 3: OOD Reweighting Block Tailored for GNNs. Before elaborating our OOD block, we first retrospect that in most scenarios, a tiny perturbation would not induce large distribution shifts along the input space, thanks to the approximate continuity of input features ( e.g. , image pixels in computer vision ). Unfortunately, the structural features of GNNs' inputs - adjacency matrix comprising of 0 s and 1 s - are discrete thus only one perturbation ( e.g. , adding or deleting an edge) could induce large distribution shifts, and further violate the underlying properties, such as node degree distribution [26], graph size distribution [27] and domain-specific constraints [28].

Thus, it is pivotal to construct an OOD reweighting block for assessing whether the generated graph G ′ deviates from the data manifold. This block is expected to assign an 'OOD score' - the degree of distribution shift between G ′ and original graph G -to each G ′ . However, it is non-trivial to quantify the degree of OOD [29]. Inspired by the great success of graph anomaly detection [30, 31, 32, 33, 34], we treat the OOD instance as the 'anomaly' since it is isolated from the original data distribution, and naturally employ the common usage module of anomaly detection variational graph auto-encoder (VGAE) [19] containing an encoder and a decoder - to instantiate our OOD reweighting block. Additionally, the great success of diffusion generative models has been recognized recently [35, 36], and thus this domain is deferred for future investigation.

Concretely, as shown in Figure 2, the preparation for evaluations is training the VGAE model on the dataset D where input graph G is sampled. After that, we can leverage the reconstruction loss to approximate the degree of OOD for any generated instance G ′ . To be more specific:

Given G ′ = ( A ′ , X ′ ) , the encoder first learns a latent matrix Z according to A ′ and X ′ , where row z i corresponds to the node v ′ i in G ′ . Note that z i is assumed to follow the independent normal distributions with expectation µ i and variance σ 2 i . Formally:

where µ and σ are parameterized by two two-layer GCNs [37] called GCN µ and GCN σ .

Then, the decoder recovers the adjacency matrix A ′ based on Z :

where σ ( · ) is the logistic sigmoid function.

The OOD score of G ′ is given by the normalized reciprocal of the reconstruction loss L recon ( G ′ ) ,

Since VGAE is trained on the dataset D , G ′ straying far from the data distribution of D would get the high reconstruction loss L recon . Thus, as the reciprocal of L recon , the OOD score of G ′ must be low. Conversely, if G ′ is in distribution, it would gain a high OOD score because it is easy to be reconstructed. Based on this, our OOD block can mitigate the impact of OOD instances. Specifically, the OOD score is utilized as the weight of each prediction when calculating the expectation of the generated graph's prediction. This allows for the marginalization of instances with low OOD scores, as shown in the gray dotted box of Figure 2.

Overall evaluation process. As the last piece of the OAR puzzle, i.e., OOD reweighting block has been instantiated, let's revisit Figure 2 and summarize the overall process of OAR:

Before we evaluate the explanatory subgraph G s , the OOD reweighting block ( i.e., VGAE) is trained on the dataset D where input graph G is sampled.

Then, we fix the G s and randomly perturb the structure of the complementary part G ¯ s to get G ′ .

Each G ′ is fed into GNN f and VGAE simultaneously to audit prediction and OOD score. Both GNN's behavior and data distribution are taken into consideration in this step.

At last, according to the predictions and their weights ( i.e., OODscores), we calculate the weighted average of the generated graphs' predictions. The closer this average is to the original prediction of G , the higher the quality of the explanation G s is.

The pseudocode and the tricks to expedite computations are detailed in Appendix A.

2.3 A simplified version of OAR

To better generalize to large datasets and reduce the computational complexity, we provide a simplified version of OAR called SimOAR in this section. Compared with OAR, SimOAR achieves a significant improvement in computational efficiency at the expense of a small amount of performance degradation. Concretely, SimOAR is mainly motivated by three empirical inferences after executing OAR:

The most time-consuming part of OAR is its preparatory work, i.e., training OOD reweighting block, especially for large datasets. For example, on the dataset MNIST superpixels [38] containing 70,000 graphs, the converged process of VGAE occupies 93 . 7% of OAR's execution time.

In the course of generating G ′ , the number of perturbation operations is roughly proportional to the degree of distribution shift given by the OOD block. For example, the graph G ′ 1 created via deleting one edge typically gets a higher OOD score than the graph G ′ 2 created via deleting five edges.

For two generated graphs generated via the same perturbation times, they generally get similar reconstruction losses and are assigned similar OOD scores.

Based on these, to expedite computations and simplify the OAR, we deactivate the OOD reweighting block ( i.e. , deleting all the sketches in gray dotted boxes in Figure 2) in OAR. As compensation for data distribution, we restrict the ratio of the number of perturbations to the number of edges in the original graph G to a pre-defined minor value R . Since the generated graphs typically share similar reconstruction losses and OOD scores while R is fixed, we directly calculate their average prediction to approximate their excepted prediction. The pseudocode and more implementation details of SimOAR are provided in Appendix A.

It is worth noting that despite the potential existence of a few generated graphs G ′ in SimOAR that fall outside the distribution, the performance of SimOAR still significantly surpasses that of current evaluation methods w.r.t consistency with both metrics based on ground truth and human supervision. Hence, in light of the efficiency of the SimOAR, we strongly advocate for its adoption as a predominant alternative to prevalent removal-based evaluation metrics. At the heart of SimOAR and a central thesis of this paper - is the perspective that, during evaluation, rather than deleting all non-explanatory nodes and then gauging the resultant output variations, it is more insightful to randomly delete a portion of the non-explanatory nodes multiple times and then gauge the average output variations.

3 Experiments

We present empirical results to demonstrate the effectiveness of our proposed methods OAR and SimOAR. The experiments aim to investigate the following research questions:

RQ1: How is the evaluation quality of OAR and SimOAR compared to that of existing metrics?

RQ2: How is the generalization of OAR and SimOAR compared to that of existing metrics?

RQ3: What is the impact of the designs ( e.g. , the OOD reweighting block) on the evaluations?

3.1 Experimental settings

To evaluate the effectiveness of our method, we utilize four benchmark datasets: BA3 [39], TR3 [17], Mutagenicity [40, 41], and MNIST-sp [38], which are publicly accessible and vary in terms of domain and size. Moreover, to generate the explanations of the graphs in the datasets mentioned above, we adopt several state-of-the-art post-hoc explanation methods, i.e., SA [10], GradCAM [42], GNNExplainer [11], PGExplainer [43], CXPlain [44] and ReFine [39]. The prevailing metrics removal-based evaluation (RM for short) and generation-based evaluation, i.e., DSE [17] - are selected as the baselines. Detailed experimental details can be found in Appendix B.

3.2 Measurement metric

We elaborate on the measurement metric of existing evaluation methods in this part since how to fairly define the quality of an evaluation method is critical to our research.

Ground-truth explanations. We first follow the prior studies [11, 45, 43] and treat the subgraphs coherent to data generation procedure or human knowledge as ground truth. Although ground truth might not conform to the decision-making process exactly, it contains sufficient discriminative information to help justify the quality of explanations. Moreover, it's worth emphasizing again that our method does not depend on ground truth, and gathering ground truth is only for fair comparison.

Consistency with metric based on ground-truth explanations. Specifically, given a to-be-evaluated subgraph G s and its corresponding ground-truth explanation G GT s , we use Recall as the gold evaluation metric defined as Recall( G s ) = ∣ ∣ E s ⋂ E GT s ∣ ∣ / ∣ ∣ E GT s ∣ ∣ , where E s and E GT s are the edge set of G s and G GT s ; | · | denotes the cardinal function of set. Hence, for any evaluation method, we can calculate its consistency with Recall to quantify its performance via Kendall correlation τ [46] defined as:

where { r i } n i =1 and { s i } n i =1 are a pair of Recall values and evaluation scores; sgn( · ) is the sign function and I ( · ) is the indicator function. The bigger τ is, the higher the evaluation scores are consistent with Recall values, and thus the better the evaluation method is.

Consistency with human intuition. The consistency between evaluation results and human intuition is also an important reference. In view of the high subjectivity of human intuition, we organized a large-scale user study engaging 100 volunteers. Results are shown in Appendix C.

BA3

TR3

MNIST-sp

0.60

1.0

1.0

RM

DSE

OARSimOAR

+OAR

→RM

+OAR

→RM

0.50

0.8

SimOAR→DSE

0.8

一

SimOAR→DSE

0.6

0.6

0.40

0.4

0.4

0.30

0.2

0.2

0.20

0.0

0.0

SA

GradCAMGNNExplainerPGExplainer CXPlainer

0.00

0.33

0.67

1.00

0.0

0.2

0.4

0.6

0.8

1.0

MNIST-sp

BA3

MUTAG

0.60

1.0

1.0

RM

DSEOARSimOAR

+OAR

+RM

0.8

0.8

+SimOAR→DSE

0.50

0.6

0.6

0.40

0.4

0.4

0.30

0.2

+OAR

→RM

0.2

士

SimOAR→DSE

0.20

SA

GradCAMGNNExplainer PGExplainer CXPlainer

0.0

0.00

0.25

0.50

0.75

1.00

0.0

0.0

0.5

1.0

(a) Correlationbetween evaluation metrics and Recall

(b)ConsistencybetweenRecall andevaluationmetrics

Limited by space, we further exhibit and discuss more detailed implementations and results in Appendix B, including but not limited to the generalization of OAR involving the evaluation of the post-hoc explanations for node classification task and the inherent explanations.

3.3 Study of explanation evaluation (RQ1)

As a preparation for the experiments, we first collect the explanatory subgraphs {G i s } for a set of graphs {G i } N i =1 and the corresponding well-trained GNN model f . We denote the evaluation score on G i s based on RM, DSE and our OAR and SimOAR by s i RM , s i DSE , s i OAR and s i SimOAR , respectively. For more faithful comparison, we present both the explainer-level correlation defined

G i,h s means the subgraph G i s is extracted by explainer h , H is the set of explainers. The explainer-level results, under all evaluation methods, on all datasets, are presented in Table 1. Moreover, considering the instance-level results share a similar tendency, we presented the representative results on BA3 and MNIST-sp in Figure 3 (a). According to Table 1 and Figure 3 (a) we can find that:

Observation 1: OAR outperforms other methods in all cases. Substantially, Kendall rank correlation greatly improves after leveraging the paradigm of adversarial robustness. The most notable case is the explainers' rank on BA3 and MUTAG, where τ ∗ = 1.00 achieves a tremendous increase from the RM and the DSE. It demonstrates the effectiveness and universality of OAR/SimOAR and verifies that OAR/SimOAR can be leveraged to boost the quality of evaluations.

Observation 2: SimOAR performs a little worse than OAR, but still significantly improves over the strongest baselines. To be more specific, the average score of SimOAR is 7.83% less than that of OAR, but still, 42.62% higher than RM and 37.45% higher than DSE. It demonstrates that SimOAR is adequate for the task of explanation evaluations in most cases.

Further analysis of the results presented in Table 1 reveals that:

Observation 3: OAR/SimOAR presents a more fair and faithful comparison among explainers. The rankings provided by the OAR and SimOAR are highly consistent ( i.e. , τ ∗ = 0 . 928 on average) with the references, while the removal- and generation-based rankings unsurprisingly pale by comparison. These empirical results give us the courage to leverage OAR and SimOAR to evaluate emerging explanation methods in the future.

3.4 Study of generalization (RQ2)

Although the experiment provided in 3.3 is detailed and fair, we contend that the generalization of OAR and SimOAR is still unexamined. Specifically, a specific explainer always has a preference

for certain patterns, and thus the explanatory subgraphs extracted by it often have similar structures. Therefore, the experimental results based on limited explainers may not generalize well to other existing or future explainers, especially those based on different lines of thought.

As it is impractical for us to cover all these explainers, we resort to directly generalizing the tobe-evaluated subgraphs. That is, we make use of fake explanatory subgraphs which are randomly sampled from the full graph. The detailed sampling algorithm and settings can be found in Appendix C. In these settings, the best case is that the evaluation score is monotonically increasing w.r.t. the Recall level, which indeed reaches the best consistency. Average normalized scores under all evaluation methods are shown in Figure 3 (b), which indicate that:

Observation 4: OAR/SimOAR greatly improves the consistency between evaluation scores and Recalls, which indicates that our method has tremendous potential to perform well on other explainers. Conversely, removal- and generation-based methods are negatively correlated with the importance involving the set of to-be-evaluated subgraphs which get high Recalls.

3.5 Study of designs (RQ3)

Effectiveness of OOD block. We first focus on the effectiveness of the OOD block. The most immediate impact of OOD block on OAR can be estimated by comparing the performance of OAR and SimOAR, which can be also deemed as the ablation experiments. Hence, we turn to qualitatively analyze the OOD block via some case studies shown in Figure 4, where all the OOD scores belonging to the same datasets are normalized to the range of 0 to 1. Based on the information conveyed in Figure 4, the following observation can be made:

Observation 5: OOD block can assign the lower weights to the subgraph which violent the underlying properties of the full graph. For example, graph properties of chemical molecules, such as the valency rules, impose some constraints on syntactically valid

Original Graph

Generated Graph 1

Generated Graph 2

OOD

OOD

OOD

Score:

Score:

Score:

0.97

0.21

0.64

OOD

OOD

OOD

Score:

Score:

Score:

0.94

0.44

0.72

OOD

OOD

OOD

Score:

Score:

Score:

0.93

0.69

0.85

Figure 4: Case studies for OOD reweighting block with the graphs randomly selected from datasets MNIST-sp, MUTAG, and BA3 arranged from top to bottom. Best viewed in color.

molecules. Hence, the invalid molecular subgraphs, which destroy the integrity of the carbon ring by simply removing some bonds (edges) or atoms (nodes) are assigned low scores by the OOD block.

Time complexity. To further explore the efficiency of our evaluation method and the designed module in it, we count the running time of the evaluation process on every single graph and average the time over the entire test set to obtain per-graph time consumption. The comparison is provided in Table 2. According to Table 2 we can find that:

Observation 6: SimOAR greatly reduces the execution time. Concretely, the execution speed has nearly doubled after leveraging the metric of SimOAR. This significant improvement in efficiency corresponds to the original intention of SimOAR and verifies the success of SimOAR's design.

4 Related Work & Further Discussion

OAR & Contemporary Evaluation Metrics. Apart from the qualitative evaluation methods based on human intuition, recent literature categorizes quantitative metrics into four primary categories: accuracy, faithfulness, stability, and fairness [47, 48, 49, 50, 51, 52]. Notably, Precision and Recall metrics align with the accuracy category, while our proposed OAR and SimOAR fall under the faithfulness category. Among these methods, the most recently proposed faithfulness metric is GEF [49], which however omits quantification of the distribution shift in subgraphs. Nevertheless, we

have exhibited the experimental comparison between GEF and our metrics w.r.t the latest dataset SHAPEGGEN [49] in Appendix B.

OOD & GNNs Explainability The OOD issue is one of the most critical challenges in the post-hoc explainability domain currently [16, 53]. To sidestep this challenge, many studies have pivoted towards the development of inherently explainable GNNs [15, 54]. Notwithstanding the complexity of the task, efforts such as FIDO [55] are making significant successes in addressing the OOD problem within post-hoc explanations. Concurrently, CGE [56] leverages the lottery ticket hypothesis [57, 58, 59, 60, 61] to craft the cooperative explanation for both GNNs and graphs, wherein the OOD challenge is potentially mitigated through the EM algorithm. GIBE [62] delves into the intersection of the OOD issue and regularization, viewing it through the lens of information theory. Furthermore, MixupExplainer [63] and CoGE [64] navigate the OOD problem from the generation and recognition stances, respectively.

Simultaneously, the evaluation of inherent explanations encounters the same hurdles as post-hoc explanations: it's challenging to quantify in the absence of the ground truth. Fortunately, by introducing an additionally well-trained GNN, OAR can be employed to evaluate inherent explanations in a similar way to evaluate post-hoc explanations. The experimental results are shown in Appendix B.

Limitations & Concerns. While we acknowledge the effectiveness of our methods, we also recognize their limitations. Concretely, despite utilizing SimOAR to expedite the evaluation process, our paradigm remains more time-intensive compared to the conventional removal-based metric. To overcome this constraint, a probable solution is summarizing the optimal number of perturbations and implementing a self-adaptive extraction module to select the perturbed features.

Furthermore, we recognize potential apprehensions regarding the migration of trust issues from the black-box GNN to the equally non-transparent VGAE. Nevertheless, we posit that the repercussions of this are substantially mitigated in our streamlined method. Specifically, SimOAR bypasses VGAE in favor of employing transparent heuristics for perturbation generation, effectively addressing the aforementioned trust concerns. It's noteworthy that, while SimOAR's performance may be marginally below or comparable to OAR's, it consistently exceeds other benchmarks. This emphasizes VGAE's restrained impact and reaffirms our recommendation of SimOAR over OAR.

5 Conclusion

In this paper, we explored the evaluation process of GNN explanations and proposed a novel evaluation metric, OOD-resistant adversarial robustness (OAR). OAR gets inspiration from the notion of adversarial robustness and evaluates the quality of explanations by calculating their robustness under attack. It addresses the inherent limitations of current removal- and generation-based evaluation metrics by taking both data distribution and GNN behavior into account. For applications involving large datasets, we introduce a simplified version of OAR (SimOAR), which achieves a significant increase in computational efficiency at the cost of a small amount of performance degradation. This work represents an initial attempt to exploit evaluation metrics for post-hoc GNN explainability from the perspective of adversarial robustness and resistance to OOD.

6 Ethics Statement

This work is primarily foundational in GNN explainability, focusing on the development of a more reliable evaluation algorithm. Its primary aim is to contribute to the academic community by enhancing the understanding and implementation of the evaluation process. We do not foresee any direct, immediate, or negative societal impacts stemming from the outcomes of our research.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (9227010114, 62121002) and the University Synergy Innovation Program of Anhui Province (GXXT-2022-040).

References

Vijay Prakash Dwivedi, Chaitanya K. Joshi, Thomas Laurent, Yoshua Bengio, and Xavier Bresson. Benchmarking graph neural networks. CoRR , abs/2003.00982, 2020.

Tianyi Zhao, Yang Hu, Linda R. Valsdottir, Tianyi Zang, and Jiajie Peng. Identifying drugtarget interactions based on graph convolutional network and deep neural network. Briefings Bioinform. , 22(2):2141-2150, 2021.

Zhiwei Guo and Heng Wang. A deep graph neural network-based mechanism for social recommendations. IEEE Trans. Ind. Informatics , 17(4):2776-2783, 2021.

Kun Wang, Yuxuan Liang, Pengkun Wang, Xu Wang, Pengfei Gu, Junfeng Fang, and Yang Wang. Searching lottery tickets in graph neural networks: A dual perspective. In The Eleventh International Conference on Learning Representations , 2023.

Minh N. Vu and My T. Thai. Pgm-explainer: Probabilistic graphical model explanations for graph neural networks. In NeurIPS , 2020.

Wanyu Lin, Hao Lan, and Baochun Li. Generative causal explanations for graph neural networks. In ICML , volume 139, pages 6666-6679, 2021.

Michael Sejr Schlichtkrull, Nicola De Cao, and Ivan Titov. Interpreting graph neural networks for NLP with differentiable edge masking. In ICLR , 2021.

Hao Yuan, Haiyang Yu, Shurui Gui, and Shuiwang Ji. Explainability in graph neural networks: A taxonomic survey. CoRR , abs/2012.15445, 2020.

Hao Yuan, Haiyang Yu, Jie Wang, Kang Li, and Shuiwang Ji. On explainability of graph neural networks via subgraph explorations. In ICML , volume 139, pages 12241-12252. PMLR, 2021.

Federico Baldassarre and Hossein Azizpour. Explainability techniques for graph convolutional networks. CoRR , abs/1905.13686, 2019.

Zhitao Ying, Dylan Bourgeois, Jiaxuan You, Marinka Zitnik, and Jure Leskovec. Gnnexplainer: Generating explanations for graph neural networks. In NeurIPS , pages 9240-9251, 2019.

Leo Breiman. Random forests. Mach. Learn. , 45:5-32, 2001.

Giles Hooker and Lucas Mentch. Please stop permuting features: An explanation and alternatives. CoRR , abs/1905.03151, 2019.

Haoyang Li, Xin Wang, Ziwei Zhang, and Wenwu Zhu. Out-of-distribution generalization on graphs: A survey. CoRR , abs/2202.07987, 2022.

Siqi Miao, Mia Liu, and Pan Li. Interpretable and generalizable graph learning via stochastic attention mechanism. In ICML , pages 15524-15543, 2022.

Ying-Xin Wu, Xiang Wang, An Zhang, Xiangnan He, and Tat-Seng Chua. Discovering invariant rationales for graph neural networks. CoRR , abs/2201.12872, 2022.

Ying-Xin Wu, Xiang Wang, An Zhang, Xia Hu, Fuli Feng, Xiangnan He, and Tat-Seng Chua. Deconfounding to explanation evaluation in graph neural networks. CoRR , abs/2201.08802, 2022.

Christopher Frye, Damien de Mijolla, Tom Begley, Laurence Cowton, Megan Stanley, and Ilya Feige. Shapley explainability on the data manifold. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021 , 2021.

Thomas N. Kipf and Max Welling. Variational graph auto-encoders. CoRR , abs/1611.07308, 2016.

Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings , 2014.

Thomas Fel, Melanie Ducoffe, David Vigouroux, Rémi Cadène, Mikael Capelle, Claire Nicodeme, and Thomas Serre. Don't lie to me! robust and efficient explainability with verified perturbation analysis. CoRR , abs/2202.07728, 2022.

Cheng-Yu Hsieh, Chih-Kuan Yeh, Xuanqing Liu, Pradeep Kumar Ravikumar, Seungyeon Kim, Sanjiv Kumar, and Cho-Jui Hsieh. Evaluations and methods for explanation through robustness analysis. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021 , 2021.

Yongduo Sui, Tianlong Chen, Pengfei Xia, Shuyao Wang, and Bin Li. Towards robust detection and segmentation using vertical and horizontal adversarial training. In 2022 International Joint Conference on Neural Networks (IJCNN) , pages 1-8. IEEE, 2022.

Daniel Zügner, Amir Akbarnejad, and Stephan Günnemann. Adversarial attacks on neural networks for graph data. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2018, London, UK, August 19-23, 2018 , pages 2847-2856, 2018.

Daniel Zügner and Stephan Günnemann. Adversarial attacks on graph neural networks via meta learning. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 , 2019.

Jure Leskovec, Jon M. Kleinberg, and Christos Faloutsos. Graphs over time: densification laws, shrinking diameters and possible explanations. In Proceedings of the Eleventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Chicago, Illinois, USA, August 21-24, 2005 , pages 177-187, 2005.

Beatrice Bevilacqua, Yangze Zhou, and Bruno Ribeiro. Size-invariant graph representations for graph classification extrapolations. In Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event , pages 837-851, 2021.

Qi Liu, Miltiadis Allamanis, Marc Brockschmidt, and Alexander L. Gaunt. Constrained graph variational autoencoders for molecule design. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada , pages 7806-7815, 2018.

Haoyang Li, Xin Wang, Ziwei Zhang, and Wenwu Zhu. OOD-GNN: out-of-distribution generalized graph neural network. CoRR , abs/2112.03806, 2021.

Xiaoxiao Ma, Jia Wu, Shan Xue, Jian Yang, Quan Z. Sheng, and Hui Xiong. A comprehensive survey on graph anomaly detection with deep learning. CoRR , abs/2106.07178, 2021.

Yuan Gao, Xiang Wang, Xiangnan He, Zhenguang Liu, Huamin Feng, and Yongdong Zhang. Alleviating structural distribution shift in graph anomaly detection. In WSDM , pages 357-365. ACM, 2023.

Xian Teng, Muheng Yan, Ali Mert Ertugrul, and Yu-Ru Lin. Deep into hypersphere: Robust and unsupervised anomaly discovery in dynamic networks. In Proceedings of the TwentySeventh International Joint Conference on Artificial Intelligence, IJCAI 2018, July 13-19, 2018, Stockholm, Sweden , pages 2724-2730, 2018.

Yuan Gao, Xiang Wang, Xiangnan He, Zhenguang Liu, Huamin Feng, and Yongdong Zhang. Addressing heterophily in graph anomaly detection: A perspective of graph spectrum. In WWW , pages 1528-1538. ACM, 2023.

Yuan Gao, Xiang Wang, Xiangnan He, Huamin Feng, and Yong-Dong Zhang. Rumor detection with self-supervised learning on texts and social graph. Frontiers Comput. Sci. , 17(4):174611, 2023.

Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In NeurIPS , 2020.

Prafulla Dhariwal and Alexander Quinn Nichol. Diffusion models beat gans on image synthesis. In NeurIPS , pages 8780-8794, 2021.

Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In ICLR (Poster) , 2017.

Federico Monti, Davide Boscaini, Jonathan Masci, Emanuele Rodolà, Jan Svoboda, and Michael M. Bronstein. Geometric deep learning on graphs and manifolds using mixture model cnns. In CVPR , pages 5425-5434, 2017.

Xiang Wang, Ying-Xin Wu, An Zhang, Xiangnan He, and Tat-Seng Chua. Towards multigrained explainability for graph neural networks. In NeurIPS , pages 18446-18458, 2021.

Jeroen Kazius, Ross McGuire, and Roberta Bursi. Derivation and validation of toxicophores for mutagenicity prediction. Journal of medicinal chemistry , 48(1):312-320, 2005.

Kaspar Riesen and Horst Bunke. IAM graph database repository for graph based pattern recognition and machine learning. In SSPR/SPR , pages 287-297, 2008.

Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, October 22-29, 2017 , pages 618-626, 2017.

Dongsheng Luo, Wei Cheng, Dongkuan Xu, Wenchao Yu, Bo Zong, Haifeng Chen, and Xiang Zhang. Parameterized explainer for graph neural network. In NeurIPS , 2020.

Patrick Schwab and Walter Karlen. Cxplain: Causal explanations for model interpretation under uncertainty. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 32. Curran Associates, Inc., 2019.

Hao Yuan, Jiliang Tang, Xia Hu, and Shuiwang Ji. XGNN: towards model-level explanations of graph neural networks. In KDD , pages 430-438, 2020.

Eulalia Szmidt and Janusz Kacprzyk. The spearman and kendall rank correlation coefficients between intuitionistic fuzzy sets. In EUSFLAT Conf , pages 521-528. Atlantis Press, 2011.

Anna Himmelhuber, Mitchell Joblin, Martin Ringsquandl, and Thomas A. Runkler. Demystifying graph neural network explanations. In PKDD/ECML Workshops (1) , volume 1524, pages 67-75, 2021.

Benjamín Sánchez-Lengeling, Jennifer N. Wei, Brian K. Lee, Emily Reif, Peter Wang, Wesley Wei Qian, Kevin McCloskey, Lucy J. Colwell, and Alexander B. Wiltschko. Evaluating attribution for graph neural networks. In NeurIPS , 2020.

Chirag Agarwal, Owen Queen, Himabindu Lakkaraju, and Marinka Zitnik. Evaluating explainability for graph neural networks. CoRR , abs/2208.09339, 2022.

Chirag Agarwal, Marinka Zitnik, and Himabindu Lakkaraju. Probing GNN explainers: A rigorous theoretical and empirical analysis of GNN explanation methods. In AISTATS , volume 151, pages 8969-8996, 2022.

Lukas Faber, Amin K. Moghaddam, and Roger Wattenhofer. When comparing to ground truth is wrong: On evaluating GNN explanation methods. In KDD , pages 332-341, 2021.

He Zhang, Bang Wu, Xingliang Yuan, Shirui Pan, Hanghang Tong, and Jian Pei. Trustworthy graph neural networks: Aspects, methods and trends. CoRR , abs/2205.07424, 2022.

Yongduo Sui, Xiang Wang, Tianlong Chen, Meng Wang, Xiangnan He, and Tat-Seng Chua. Inductive lottery ticket learning for graph neural networks. Journal of Computer Science and Technology , 2023.

Gang Liu, Tong Zhao, Jiaxin Xu, Tengfei Luo, and Meng Jiang. Graph rationalization with environment-based augmentations. In KDD , pages 1069-1078, 2022.

Chun-Hao Chang, Elliot Creager, Anna Goldenberg, and David Duvenaud. Explaining image classifiers by counterfactual generation. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 , 2019.

Junfeng Fang, Xiang Wang, An Zhang, Zemin Liu, Xiangnan He, and Tat-Seng Chua. Cooperative explanations of graph neural networks. In WSDM , pages 616-624. ACM, 2023.

Ari S. Morcos, Haonan Yu, Michela Paganini, and Yuandong Tian. One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers. In NeurIPS , pages 4933-4943, 2019.

Kun Wang, Yuxuan Liang, Pengkun Wang, Xu Wang, Pengfei Gu, Junfeng Fang, and Yang Wang. Searching lottery tickets in graph neural networks: A dual perspective. In The Eleventh International Conference on Learning Representations , 2022.

Yongduo Sui, Xiang Wang, Jiancan Wu, Min Lin, Xiangnan He, and Tat-Seng Chua. Causal attention for interpretable and generalizable graph classification. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , pages 1696-1705, 2022.

Tianlong Chen, Yongduo Sui, Xuxi Chen, Aston Zhang, and Zhangyang Wang. A unified lottery ticket hypothesis for graph neural networks. In ICML , pages 1695-1706, 2021.

Yanfang Wang, Yongduo Sui, Xiang Wang, Zhenguang Liu, and Xiangnan He. Exploring lottery ticket hypothesis in media recommender systems. International Journal of Intelligent Systems , 37(5):3006-3024, 2022.

Junfeng Fang, Wei Liu, An Zhang, Xiang Wang, Xiangnan He, Kun Wang, and Tat-Seng Chua. On regularization for explaining graph neural networks: An information theory perspective. 2022.

Jiaxing Zhang, Dongsheng Luo, and Hua Wei. Mixupexplainer: Generalizing explanations for graph neural networks with data augmentation. In KDD , pages 3286-3296. ACM, 2023.

Lukas Faber, Amin K. Moghaddam, and Roger Wattenhofer. Contrastive graph neural network explanation. CoRR , abs/2010.13663, 2020.

Matthias Fey and Jan E. Lenssen. Fast graph representation learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds , 2019.

Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. CoRR , abs/1710.10903, 2017.

Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In ICLR . OpenReview.net, 2019.

A Algorithms

Algorithm 1 presents the pseudocode of the evaluation process of our proposed method OAR. The pseudocode of SimOAR can be obtained by removing the line 1, 6, and 8 and simply modifying line 9 into ' s ← 1 N adv ∑ i y ( i ) '. For clarity, we put the step of feeding adversarial graph G ′ ( i ) into the target GNN and the VGAE under the for-loop. However, when implementing in real code, we can batch all those N adv adversarial graphs and feed them at one time, after the sampling process is finished, to expedite computation. Meanwhile, Algorithm 2 presents the sampling process of fake explanatory subgraphs for general evaluation.

Algorithm 1 Evaluation Process of OAR

Input: Trained GNN f ; To-be-evaluated subgraph G s and its corresponding original graph G and dataset D ; Perturbation ratio R ; Number of adversarial graphs N adv

Output: Evaluation score s

1: Train a standard VGAE on D according to [19].

2: c ← arg max i f ( G ) i .

3: for i = 1 , 2 , . . . , N adv do

4: G ′ ( i ) ← randomly deleting ⌊ R · |E G |⌉ edges from G while fixing G s .

5: y ( i ) ← f ( G ′ ( i ) ) c .

6: L ( i ) recon ← calculated according to Equation 5 using the trained VGAE.

7: end for

Algorithm 2 Sampling Fake Explanatory Subgraphs for General Evaluation

Input: Dataset D = {G 1 , G 2 , . . . , G N } ; Number of Recall levels N L ; Number of sampled subgraphs per graph N sub ; Size of sampled subgraph K sub

Output: Pairs of Recall level and corresponding subgraphs ( L k , { G i,j s,k | i = 1 , 2 , . . . , N ; j =

1:

for

k

= 1

,

2

, . . . , N

do

2: L k ← k -1 N L -1

L

3: for i = 1 , 2 , . . . , N do

4: K GT ← the number of edges in the ground-truth explanation of G i

6: K neg ← K sub -K pos

7: for j = 1 , 2 , . . . , N sub do

8: G i,j s,k ← a connected subgraph, randomly sampled from G , with K pos edges in the ground truth explanation and K neg edges not in it.

9: end for

10: end for

11: end for

B Experimentals

B.1 Experimental Details

All experiments are conducted on a Linux machine with 8 NVIDIA GeForce RTX 3090 (24 GB) GPUs. CUDA version is 11.6 and Driver Version is 510.39.01. All codes are written under Python 3.9.13 with PyTorch 1.13.0 and PyTorch Geometric (PyG)[65] 2.2.0. We adopt the Adam optimizer throughout all experiments.

Overall, for each dataset, a target GNN classification model is well-trained first. Then the explainers are built on the GNN and generate explanations for its prediction on the dataset. After that, the explanation evaluation methods evaluate how well the explanations are. Our work stands at the last level.

Study Casesfrom MNIST-sp

class

Original

High Quality

Human Intuition

Low Quality

Graph

Explanation 1

Explanation 2

Explanation 3

Explanation 4

Explanation 5

2

OARSimOAR

RM DSEOAR SimOAR RM DSEOAR SimOAR RM DSE

OARSimOAR RM DSE

OAR SimOAR RM DSE

3

OARSimOAR

RM DSE

OARSimOARRMDSE

OARSimOARRMDSEOARSimOARRM

IDSE

OAR SimOAR RM DSH

OAR

SimOAR RMDSE

OAR

SimOAR RM DSE

OARSimOARRMDSE

OARSimOARRMDSE

OAR SimOAR RM DSE

OARSimOAR RMDSE

OAR SimOAR RM DSEOAR SimOAR RM

DSE

OARSimOARRMDSE

OAR SimOAR RM DSH

OAR

SimOAR RM DSE

OAR

SimOARRMDSEOAR

OARSimOAR

RM DSE

OARSimOAR RM DSE

Target GNNs. GNNs have garnered significant recognition for their prowess in encoding graph data [37, 66, 67, 34, 58]. Amidst the vast landscape of GNNs, GIN [67] distinguishes itself with its superior encoding aptitude. In sight of this, the target GNNs for BA3, TR3, and Mutagenicity have the same structure, which is a two-layered GIN followed by a two-layered MLP with 32 hidden channels. They are trained with max epochs equal to 20, 200, and 200 respectively, batch size equal to 128, and learning rate equal to 0.001. The target GNN for MNIST-sp is adapted from an example code 1 provided by PyG, trained with the number of epochs equal to 20, batch size equal to 64, and initial learning rate equal to 0.01. Before training, we randomly split BA3, TR3, and Mutagenicity into train and test sets with ratios of 90% and 10%, respectively, while adopting the split provided by PyG for MNIST-sp. During training, we reserve data of the same size as the test set from the train set as the validation set and save the model which reaches the highest classification accuracy on the validation set for later use.

Explainers. We have implemented six state-of-the-art post-hoc explainers, namely, SA, GradCAM, GNNExplainer, PGExplainer, CXPlain, and ReFine, as claimed in Section 3.1, to generate explanatory subgraphs. Here we give a brief introduction to them:

SA [10] captures the gradients w.r.t. adjacency matrix of the input features in the process of backpropagation and directly treats them as the importance scores.

GradCAM [42] takes one step further over SA via improving the gradients w.r.t. the input features like edges by using context within the graph convolutional layers.

GNNExplainer [11] directly learns an adjacency matrix mask by maximizing the mutual information between a GNN's prediction and distribution of possible subgraph structures.

PGExplainer [43] adopts a deep neural network to parameterize the generation process of explanations, which makes it a natural approach to explaining multiple instances collectively.

CXPlain [44] treats the explanations as a causal learning task and trains causal explanation models that learn to estimate to what degree certain inputs cause outputs in the to-be-explained model.

ReFine [39] leverages the pre-training explanations to exhibit global explanations and the finetuning explanations to adapt the global explanations in the local context.

The default hyper-parameters suggested by those papers are adopted, as whether the explainers are at their optimal state is secondary to our work. Among these explainers, SA, GradCAM, and GNNExplainer directly take the to-be-explained graph as input, while the rest three need to be trained on a set of graphs, i.e., the train set in our case, in advance. We only use the explanations extracted from graphs in the test set for evaluation.

Evaluation Methods. Finally, we arrive at the explanation evaluation level. There are four evaluation methods, i.e., removal-based evaluation, DSE, OAR, and SimOAR, to be considered. Removal-based evaluation directly feeds the explanatory subgraph into the target GNN and gets its prediction, on the predicted class of the original graph, as the evaluation score, which does not involve any details. For DSE, we make use of its public source code 2 and follow its paper to set hyper-parameters. As for our method OAR/SimOAR, we have summarized the entire process of OAR in Algorithm 1. Here we present more details on how the VGAE is trained. The encoder involves two two-layered GCNs for obtaining µ and σ , each of which is realized with hidden channels equal to 256 and output channels equal to 128, while the two GCNs share the first layer. The dataset split process is the same as the training of the target GNNs. We train the VGAE model on the train set with the number of epochs equal to 100, batch size equal to 256, and learning rate equal to 0.001. The model that reaches the lowest loss on the validation set is saved for later OOD reweighting.

B.2 More Quantitative Results

Here, we sequentially present the experimental results of 1) comparison between GEF [49] and our metrics w.r.t the latest dataset SHAPEGGEN and the other datasets (Table 3), 2) correlation between metrics and Recall across explanatory subgraphs in four node classification dataset (Table 4), and 3) correlation between metrics and Recall while evaluating the explanations generated by the inherent explainable GNN, GSAT [15] via introducing an additionally well-trained GIN [67] as the model f in Algorithm 1 (Table 5).

Note that while evaluating explanations in node classification tasks, for each node in the input graph, we construct an ego graph for it based on the number of layers in the baseline GNN. Then, the explanation task for node classification can be transferred to the explanation task for graph classification. Furthermore, the remaining hyperparameters and methods in our OAR/SimOAR remain unchanged.

C User Study

In order to measure the consistency between evaluation results and human intuition, we organized a large-scale user study engaging 100 volunteers. Each volunteer was asked to check 5 groups of graphs, which contain an instance from MNIST-sp, its predicted class, and 5 randomly sampled subgraphs from this instance, and try to rank these 5 subgraphs according to how well they serve as explanations of the prediction based on intuition. We exhibit partial results in Figure 5. According to these results we can find that our evaluation methods i.e. OAR and SimOAR show the highest consistency with human intuition.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a new metric, OOD-resistant Adversarial Robustness (OAR), for evaluating post-hoc explanation methods for Graph Neural Networks (GNNs). Inspired by adversarial robustness, OAR performs random perturbations on the complementary part of the explanation result, reducing the impact of the Out-Of-Distribution (OOD) issue common in previous removal-based metrics and ensuring consistency with GNNs' behavior compared to generation-based methods. The authors further introduce an OOD reweighting block, which measures the OOD score of each perturbed sample, allowing for the marginalization of OOD instances, and the OOD score is assigned via the reconstruction loss of a Variational Graph Auto-Encoder (VGAE) pre-trained by each dataset, inspired by graph anomaly detection methods. Extensive evaluations and ablation studies demonstrate that OAR aligns more closely with the ground truth.\n\nSTRENGTHS:\n- This work summarizes and addresses a critical and emerging issue in post-hoc explanation methods for GNNs, and the proposed metric is intuitive and can mitigate issues that may happen in previous removal-based and generation-based methods, offering significant benefits to the research community.\n- The authors conduct extensive experiments, demonstrating the potential of the proposed metric as a superior evaluation tool for post-hoc explanation methods.\n- It is interesting to see OOD score can be measured using graph anomaly detection methods (and that works).\n- This paper is well-written and well-motivated.\n\nWEAKNESSES:\n- As an evaluation metric, OAR introduces an additional component that requires training on the original dataset used for the GNN. This process makes things complex, time-consuming, and may accumulate errors, making it tricky for different research groups to compare methods across various datasets.\n\n- SimOAR somewhat addresses the above concern, but it does so at the expense of performance. From current experiments (w/ six backbone methods), the performance degradation seems to be significant.\n\n- Though I am generally satisfied with evaluating six backbone explanation methods, the inclusion of more baselines would provide a more comprehensive statistical overview, given that this paper aims to introduce a metric for future community use. It would also be beneficial to include different types of post-hoc methods, such as decomposition, surrogate, and generation-based methods, among others.\n\nQUESTIONS:\n- I am not quite sure if the usage of \"adversarial\" and \"attack\" in OAR would be clear. I thought there would be some adversarial training in OAR but in fact the authors just got inspiration from them and OAR does not really do adversarial things.\n\n- How is the metric RM calculated mathematically? Is it fidelity? Can the authors provide its formulation?\n\nLIMITATIONS:\n- This paper focuses on the explanation of graph-level tasks. How would this generalize to other types of tasks as a metric?\n- There are works showing post-hoc explanation methods are always suboptimal in terms of finding label-relevant patterns and therefore proposing self-explainable models and pretrain-finetuning framework, e.g., GSAT. This may limit the future impact of OAR if it is only for post-hoc methods. Can the idea in OAR help self-explainable methods?", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-23"}

comments: SUMMARY:
The paper proposes a novel evaluation metric called OOD-resistant Adversarial Robustness (OAR) to address the issue of assessing the credibility of explanations provided by graph neural networks (GNNs). The paper criticizes existing evaluation methods that fail to consider out-of-distribution (OOD) data and can produce inconsistent results. OAR overcomes these limitations by calculating the robustness of explanatory subgraphs under attack and incorporating an OOD reweighting block. The paper also introduces a simplified version of OAR called SimOAR for more efficient evaluation of large datasets. Experimental results show that OAR outperforms current evaluation metrics and demonstrates consistency with metrics like Precision, Recall, and Human Supervision .

STRENGTHS:
1. This work tackles an important research gap on evaluating the explanation for GNN.
2. The paper is well-written and easy to follow. 
2. The authors conduct experiments, and the results demonstrate the advantage of their proposed method.

WEAKNESSES:
1. Figure 1 is kind of hard to follow. It could be better if authors can refine it.
2. Some experiment setting is missing. For example, what is the setting of the GAE? What is the number of the generated graph?
3. Lack of related work, and some important reference is missing. For example, the authors could add a section introduce the explanation methods for GNN according to [1].

[1]  Zhang, He, et al. "Trustworthy graph neural networks: Aspects, methods and trends." arXiv preprint arXiv:2205.07424 (2022).

QUESTIONS:
Listed in Weakness.

LIMITATIONS:
Yes.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-06-30

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a new evaluation metric OAR for evaluating post-hoc explanation methods for GNNs. OAR has two main advantages in that it does not need ground-truth labels and in that it prevents finding spurious explanations based on out-of-data-distribution phenomena. The paper focuses on graph classification: The input for this explanation is a subgraph of the graph for which we want to explain a prediction. The proposed method OAR, creates several permutation of the original graphs, where it is allowed to only permute edges **not** in the explanation. A generative model forces those permutations to not leave the training data distribution. The less the permutations can change the model prediction the better an explanation we consider the subgraph. Experiments validate the effectiveness of OAR.\n\nSTRENGTHS:\nThe paper tackles an important topic, GNN explanation, in an ambitious way: To evaluate different explanation methods unsupervised, without access to ground truth explanations.\n\nThe paper is in a good shape (apart from the introduction+related work): The writing is clear and allows for understanding most of the paper in the first or second read. The figures are helpful for understanding and well connected to the text. The experiments chosen by the authors make sense and I appreciate the authors also running a user study.\n\nWEAKNESSES:\nI do not like the approach the authors have taken for the related work in this paper. The first section is a hybrid of related work and introduction, and I feel it serves neither well. I am missing a concise motivation and a broader context of the paper from this section. The actual related work is in Appendix A, which I would prefer to be in the main paper. I also think that there is an important branch of work missing [1-5]. These works discuss general properties that we may want an explanation method and an evaluation setup to adhere to. I think it would benefit the paper to discuss how OAR is compatible/incompatible with these proposals. These works also offer some alternative benchmarks for evaluation instead of the flawed BA and tree-based (TR) datasets. \n[1] Himmelhuber et al: Demystifying Graph Neural Network Explanations\n[2] Sanchez-Lengeling et al. Evaluating attribution for graph neural networks\n[3] Agarwal et al. Evaluating explainability for graph neural networks\n[4] Agarwal et al. Probing GNN explainers: A rigorous theoretical and empirical analysis of GNN explanation methods\n[5] Faber et al. When comparing to ground truth is wrong: On evaluating GNN explanation methods\n\nThere is also one paper discussing out-of-distribution versus explanation for GNNs [6] that might be worth discussing/comparing to.\n[6] Faber et al. Contrastive graph neural network explanation\n\nI am not convinced about the explanations that the method produces. Generally, we want to create explanations to make humans understand what is happening. For this, every step to create the explanation is ideally human inspectable. Here, it seems like we are transferring the trust problem from the GNN to the VGAE. This becomes the blackbox that somehow determines if the results are good or bad and we cannot inspect this blackbox. For example, using recall on ground-truth data makes for a good explanation metric because us humans can see *why* the explanation is supposedly good.\n\nQUESTIONS:\nCould you expand on the setup used for the user study? The previous years (e.g., https://neurips.cc/Conferences/2021/PaperInformation/PaperChecklist) included a paper checklist of what to consider and the details in Appendix D are quite short.\n\nThe study on MNIST explanations seems to be very dependent on the layout of the explanation subgraph (maybe even more so than the choice of what nodes to put there). Can you expand on how you created the graph layout? Are superpixel nodes positioned where they are placed in the image?\n\nDoes the permutation space for the targeted adversarial attacks allow adding and deleting nodes? Or are the attacks constrained to happen in the given adjacency matrix?\n\nOAR itself makes no assumptions on the size of the explanation, but it seems it is easier to get good explanations the larger the explanation graph is: there is less attack space for adversarial permutation and a larger fixed graph likely helps for low OOD scores. I wonder if OAR would benefit from a component that penalizes surplus nodes in the explanation?\n\nCould the method be expanded to also support node classification? One naive angle might be to extract the receptive field of each node and use OAR on those graphs?\n\nFrom what I understand, an important motivation is the fight of out-of-distribution data. This motivates the generative graph model to find and remove outliers. On the other hand, it seems that SimOAR performs similarly without using such a model but a budget(?) on perturbations. Can we maybe do something simpler than the generative model, for example, looking at spectra?\n\nSome of the works linked above showed that the BA and tree based datasets have flaws and GNNs do not always use all the data present in the ground truth. Could these also influence your experiments?\n\nLIMITATIONS:\nAddressed by the authors", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the explainability evaluation of GNNs, and proposes a new evaluation paradigm. Inspired by adversarial robustness, it uses a generative model (VGAE) to fulfill the explanation subgraph, randomly perturbs the fulfilled parts, and uses the OOD scores of perturbed graphs to measure the importance of explanation subgraph. Experiments are done on several datasets and show the improved evaluation quality w.r.t. diverse criteria.\n\nSTRENGTHS:\n1.\tThis paper summarizes the removal- and generation-based evaluation protocols well, and points out their drawbacks w.r.t. post-hoc explainability. It resolves these drawbacks by calculating the robustness of explanations under attack and OOD-reweighting. This motivation is clear and reasonable.\n2.\tIn terms of technical contributions, the evaluation of explanation robustness is well-designed. It first gives an adversarial robustness-related definition, and then transfers it into a tractable objective and designs an OOD-reweighting block (i.e., an external VGAE) to solve the objective. Moreover, a computationally efficient variant is also proposed.\n3.\tThe experiments are sufficient to demonstrate the effectiveness of the proposed method, w.r.t., explanation evaluation, generalization, model design, and user study.\n4.\tI appreciate the diverse criteria used in the paper, especially the “consistency of ground-truth explanations and human intuition”.\n5.\tThe presentation of the proposed method is clear.\n\nWEAKNESSES:\n1.\tRegarding adversarial robustness, the proposed measurement is based on perturbing the subgraphs. I have two questions: (1) why name the random perturbation as adversarial robustness, which usually performs adversarially perturbation? Does it mean that “How adversarial are random perturbations against the original data distribution, which are measured by OOD score”? (2) How many perturbed subgraphs are needed? I think these concepts are essential to understand the proposed evaluation method. Hence, more clarification is needed.\n2.\tThe OOD reweighting block is implemented by VGAE, hence, the proposed method seems heavily dependent on the VGAE quality. However, many studies show the generative ability of VGAE is suboptimal and degenerated. It would be better to analyze the influence of VGAE quality on the OAR performance.\n\nQUESTIONS:\n1.\tWhy name the random perturbation as adversarial robustness, which usually performs adversarially perturbation? Does it mean that “How adversarial are random perturbations against the original data distribution, which are measured by OOD score”?\n2.\tHow many perturbed subgraphs are needed?\n3.\tWhat is the influence of VGAE quality on the OAR performance?\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}