Title: SAME: Uncovering GNN Black Box with Structure-aware Shapley-based Multipiece Explanations
Venue: NeurIPS 2023
Accepted: yes
Keywords: GNN explainability

Shapley value

Monte Carlo tree search

structure awareness

multi-grained explanation
Abstract: Post-hoc explanation techniques on graph neural networks (GNNs) provide economical solutions for opening the black-box graph models without model retraining. Many GNN explanation variants have achieved state-of-the-art explaining results on a diverse set of benchmarks, while they rarely provide theoretical analysis for their inherent properties and explanatory capability. In this work, we propose $\underline{\text{S}}$tructure-$\underline{\text{A}}$ware Shapley-based $\underline{\text{M}}$ultipiece $\underline{\text{E}}$xplanation (SAME) method to address the structure-aware feature interactions challenges for GNNs explanation. Specifically, SAME leverages an expansion-based Monte Carlo tree search to explore the multi-grained structure-aware connected substructure. Afterward, the explanation results are encouraged to be informative of the graph properties by optimizing the combination of distinct single substructures. With the consideration of fair feature interactions in the process of investigating multiple connected important substructures, the explanation provided by SAME has the potential to be as explainable as the theoretically optimal explanation obtained by the Shapley value within polynomial time. Extensive experiments on real-world and synthetic benchmarks show that SAME improves the previous state-of-the-art fidelity performance by 12.9\% on BBBP, 7.01\% on MUTAG, 42.3\% on Graph-SST2, 38.9\% on Graph-SST5, 11.3\% on BA-2Motifs and 18.2\% on BA-Shapes under the same testing condition. Code is available at https://github.com/same2023neurips/same.

SAME: Uncovering GNN Black Box with Structure-aware Shapley-based Multipiece Explanation

Abstract

Post-hoc explanation techniques on graph neural networks (GNNs) provide economical solutions for opening the black-box graph models without model retraining. Many GNN explanation variants have achieved state-of-the-art explaining results on a diverse set of benchmarks, while they rarely provide theoretical analysis for their inherent properties and explanatory capability. In this work, we propose Structure-Aware Shapley-based Multipiece Explanation (SAME) method to address the structure-aware feature interactions challenges for GNNs explanation. Specifically, SAME leverages an expansion-based Monte Carlo tree search to explore the multi-grained structure-aware connected substructure. Afterward, the explanation results are encouraged to be informative of the graph properties by optimizing the combination of distinct single substructures. With the consideration of fair feature interactions in the process of investigating multiple connected important substructures, the explanation provided by SAME has the potential to be as explainable as the theoretically optimal explanation obtained by the Shapley value within polynomial time. Extensive experiments on real-world and synthetic benchmarks show that SAME improves the previous state-of-the-art fidelity performance by 12.9% on BBBP, 7.01% on MUTAG, 42.3% on Graph-SST2, 38.9% on Graph-SST5, 11.3% on BA-2Motifs and 18.2% on BA-Shapes under the same testing condition. Code is available at https://github.com/same2023neurips/same.

1 Introduction

Graph neural networks (GNNs) have demonstrated a powerful representation learning ability to deal with data in non-Euclidean space. However, the explanation techniques for deep learning models on images and text cannot directly apply to understand GNNs [27, 18, 6, 1]. There is a gap in the understanding of how GNNs work, which largely limits GNNs' application in many fields.

Many GNN explanation techniques aim to examine the extent to which GNNs depend on individual nodes and edges of the graph [22, 35, 19, 24, 37]. However, graph features within nodes and edges contribute different amounts of information when considered individually than contextualized with topology [37, 39]. Therefore, discovering the most important explanation with one or more connected components given an input graph and a well-trained GNN raises the additional challenge of handling structure-aware feature interactions. Recently, a number of studies, including GNNLRP [25], SubgraphX [38] and GStarX [40] have endeavored to address this issue to some extent. Although many of the current GNN explanation techniques have empirically achieved state-of-the-art

explainability performance, the design of new GNN explanation techniques mainly relies upon empirical intuition, iterative experiments, and heuristics principles. To the best of our knowledge, the explanatory capability, potential limitations and inherent properties of GNN explanation techniques have not yet been thoroughly studied from a theoretical perspective.

In this work, we propose a novel Structure-Aware Shapley-based Multipiece Explanation (SAME) technique for fairly considering the multi-level structure-aware feature interactions over the graph by introducing expansion-based Monte Carlo tree search (MCTS). Our construction is inspired by the recently proposed perturbation-based GNN explanation methods [38, 40], which have proven effective for providing explainability from a cooperative game perspective. We summarize the main differences between SAME and previous work in Table 1. The main contributions and novelties of this work include the following. (1) Theoretical aspect: i) We review the characteristics of previous methods [22, 35, 19, 25, 38, 40] and highlight several desired properties (see Table 1) that can be considered by explanation methods for GNNs. ii) We provide the loss bound of the MCTS-based explanation techniques and further verify the superiority of our expansion-based MCTS in SAME compared to previous work [38] in an intuitive manner (Sec. 3.2). (2) Empirical aspect: Our experiments cover both real-world and synthetic datasets. The results show that i) SAME outperforms previous SOTA with fidelity and harmonic fidelity metrics under the same testing condition (Sec. 5.1). ii) SAME qualitatively achieves a more human-intuitive explanation compared to previous methods across multiple datasets (Sec. 5.2).

Note: Feature interactions and structure awareness are discussed in Sec. 3.1. Multipiece explanation is provided in Sec. 3.2. Multi-grained importance (node / substructure / composite), priority-based integration and redundancy consideration are presented in Sec. 4. Detailed mathematical definitions are provided in Appendix B.1.

2 Related Work

Improving the explainability of GNN models in a post-hoc fashion has been a theme in deep graph learning. An intuitive way to explain a well-trained GNN is to trace the gradient in the models [3, 22], where the larger gradient indicates the higher importance of node or edge of the graph. Previous work has also studied the decomposition-based methods [26, 25, 9] which decompose the final prediction into several terms and mark them as the important scores for input features. Another line of GNN explanation techniques lies in perturbation-based methods [19, 31, 11, 34] which usually obtain a mask for the input graph in various ways to identify the important input feature. SubgraphX [38], as one of a perturbation-based method, samples the subgraphs from the input graph by the pruningbased MCTS and finds the most important one via the Shapley value. However, the pruning-based MCTS in SubgraphX leads to a much larger search space and thus causes higher computational costs. Moreover, SubgraphX can only provide a single connected explanation for each graph, which limits its explanatory power in many scenarios that require multipiece explanation. Most recently, GStarX [40] scores node importance based on the Hamiache-Navarro (HN) value. Although GStarX also fairly considers the structure-aware feature interactions, it fails to account for the multi-grained importance, which might result in a suboptimal explanation. For other categories in GNN explanation techniques, including surrogate methods [30, 41, 8, 12], generation-based methods [36, 15, 32], and counterfactual-based methods [17, 2, 16], we refer readers to a recent survey [37].

3 Theoretical Motivation

Notation and Preliminaries . The well-trained target GNN to be explained can be formulated as f : G → Y , where G denotes the space of input graphs and Y refers to the related label space. A graph can be denoted as G = ( V, X, E ) , where V ∈ R n × n represents node set, X ∈ R n × d is the node feature set and E ∈ R n × n denotes edge set. Given a well-trained GNN model f ( · ) and an input graph G , the goal of GNN explanation is to find the most important explanation G ∗ ex from G . Formally, this can be defined as an optimization problem that maximizes the importance of the explanation G ex for a given graph G with n nodes using an importance scoring function I ( f ( · ) , G ex , G ) :

where each explanation G i ex has n i nodes, and the other nodes not in the explanation can be expressed as { G \ G i ex } = { v j } n j = n i +1 . It is noteworthy that each explanation G i ex might contain one or more substructures ( i.e. , connected components).

3.1 Structure-aware Shapley-based Explanations Satisfies Fairness Axioms

Unlike grid-like images or sequence-like texts, graphs have more complex and abstract structures. The importance scoring function of explanations determines the reliability and explanatory power of the GNN explanation method. Therefore, we present the idea that a comprehensive assessment of the importance of an explanation should consider the feature interactions under the constraints of the input graph's topology.

Shapley value [28], originating from cooperative game theory, is the unique credit allocation scheme that satisfies the fairness axioms. This concept is similar to the importance scoring function for explanation with the consideration of feature interactions. Some previous work have brought Shapley value into deep learning explanation methods [18, 5, 14]. In our study, the importance assessment of explanation is treated as a cooperative game, where the explanation G i ex and all nodes not in the explanation { G \ G i ex } are the players in the game. Therefore, when scoring the importance of any explanation G i ex , a set of players participating in the game can be denoted as:

Inspired by the close connection between feature interactions and Shapley value, we define several desirable properties of importance scoring function for explanation according to fairness axioms:

Property 1. (Efficiency). The sum importance of all players p j in P i is the same as the improvement of GNN f ( · ) on P i over an empty set, ∑ | P i | j =1 I ( f ( · ) , p j , G ) = f ( P i ) -f ( ∅ ) .

Property 2. (Symmetry). For any explanation G i ex , if there exist two other players p j , p k ∈ { P i /G i ex } that satisfy f ( G i ex ∪ p j ) = f ( G i ex ∪ p j ) , then I ( f ( · ) , p j , G ) = I ( f ( · ) , p k , G ) .

Property 3. (Dummy). If a player p j makes no contribution to GNN f ( · ) , i.e. f ( G i ex ∪ p j ) = f ( G i ex ) holds for any G i ex , then I ( f ( · ) , p j , G ) = 0 .

Property 4. (Monotonicity). Consider two well-trained GNN models f 1 ( · ) and f 2 ( · ) , given an explanation G i ex , if for any player p j , f 1 ( G i ex ∪ p j ) -f 1 ( p j ) ≥ f 2 ( G i ex ∪ p j ) -f 2 ( p j ) always holds, then I ( f 1 ( · ) , G i ex , G ) ≥ I ( f 2 ( · ) , G i ex , G ) .

Shapley value can well satisfy the above four fairness Properties 1-4. However, the Shapley-based importance scoring function attempts to take all nodes in the graph except explanation G i ex into the cooperation game, which not only ignores the topological information of the input graph but also brings huge computational costs. Fortunately, [38] alleviate this issue by modifying the Shapleybased importance scoring function as 'k-hop Shapley', which also makes it structure-aware :

where I ( f ( · ) , G i ex , G ) denotes the weighted sum of the marginal contribution of explanation G i ex , and P i,khop = { G i ex , v n i +1 , v n i +2 , . . . , v n i + k i } includes the nodes within the G i ex as well as the nodes in the k-hop neighbors of G i ex .

3.2 Structure-aware Shapley-based Multipiece Explanation Provide Strong Explainability

The characteristics and properties within a graph or node tend to be jointly influenced by more than one high-order connected community of the graph. This implies that the appropriate explanation within this context requires the GNN explanation method to provide the multiple connected substructures simultaneously.

To solve the above challenge, we first define the mathematical formalization of the search processes on graphs by utilizing the MCTS-based GNN explanation methods. Then we propose a mathematically coherent framework to explore the explanatory power of MCTS-based GNN explanation methods using computational methods for the multi-armed bandit problem in the MCTS algorithm [20, 13].

Our mathematical framework is based on the hierarchical partitioning of the MCTS search space X . More precisely, the smoothness of the MCTS search space X can be defined by the inequality | f ( x i ) -f ( x j ) | ≤ l ( x i , x j ) , where l ( x i , x j ) refers to the Lipschitz continuity between any two substructures x i and x j in X . It serves as a critical metric to ascertain the boundedness of the change in the explanation method f ( · ) concerning the change in input substructures. Assuming that the function f ( · ) is Lipschitz continuous and the l is given, an evaluation of the function f ( · ) at any point x t enables us to define an upper bounding function B t ( x ) for f ( · ) . This upper bounding function can be refined after each evaluation of f ( · ) :

where metric l satisfies the following Assumptions 1-3. In the context of computational uncertainties associated with MCTS, the evaluation strategy in (3) offers the potential to describe specific numerical estimates within an undefinable space.

Assumption 1. (Local smoothness). There exists at least one stage-optimal substructure x /star ∈ X of f ( · ) (i.e. f ( x /star ) = sup x ∈ X f ( x ) ) and ∀ x ∈ X , f ( x /star ) -f ( x ) ≤ l ( x, x /star ) holds.

Assumption 2. (Decreasing diameters). There exists a decreasing sequence δ ( h ) > 0 , such that for any depth h ≥ 0 and for any cell X h,i of depth h , sup X h,i l ( X , X h,i ) ≤ δ ( h ) holds.

Assumption 3. (Well-shaped cells). There exists ν > 0 such that for any depth h ≥ 0 , any cell X h,i contains a l -ball of radius νδ ( h ) centered in x h,i .

With the given assumptions, the search space X can be partitioned into K H subsets ( i.e. , cells) X h,i using a K -ary tree of depth H , where 0 ≤ h ≤ H, 0 ≤ i ≤ K h -1 . Based on this partitioning method, the MCTS process can be treated as the expansion of this K -ary tree. The root of K -ary tree ( i.e. , cell X 0 , 0 ) corresponds to the whole search space X . Each cell X h,i corresponds to a node ( h, i ) of the tree, where h denotes the depth of the tree and i refers to the index. Each node ( h, i ) possesses K children nodes { ( h +1 , i k ) } 1 ≤ k ≤ K s.t. the associated cells {X h +1 ,i k , 1 ≤ k ≤ K } form a partition of the parent's cell X h,i . Consequently, expanding one node requires adding one of its K children to the current tree, which corresponds to subdividing the cell X h,j into K children cells {X h +1 ,j 1 , . . . , X h +1 ,j K } . Assume that there exist a decreasing sequence δ ( h ) ≥ 0 that satisfies sup X h,i l ( X , X h,i ) ≤ δ ( h ) for any X h,i . The decreasing sequence δ ( h ) ensures that each cell size reduces with increasing depth.

The search space X can be divided through the above partitioning method according to δ ( h ) , with respect to l -open balls. Let T t denote nodes of the current tree, and L t denotes the incoming leaves of T t to be expanded at round t . Recalling our earlier definition of B t ( h ) which was derived from the Lipschitz continuity, we can now generalize it to a new representation that connects to δ ( h ) . Formally, it is expressed as:

Based on this, we now consider which nodes will be expanded during the search. Note that Assumption 2 implies that the b-value of any cell contains x /star upper bounds f /star . In other words, for any cell X h,i such that x /star ∈ X h,i ,

This means that a leaf ( h, i ) of a K -ary tree will never be expanded if f ( x h,i ) + δ ( h ) < f /star . Therefore, under this partitioning strategy, the only set of nodes that will be expanded could be defined as I def = ∪ h ≥ 0 I h , which could be stated as

In order to derive a loss bound, we now define a measure of the quantity of near-optimal states, called near-optimality dimension . For any /epsilon1 > 0 , the set of /epsilon1 -optimal states can be defined as

Definition 1. ( η -near-optimality dimension). The η -near-optimality dimension is the smallest d ≥ 0 such that there exists C ≥ 0 , for all /epsilon1 > 0 , the maximum number of disjoint l -balls of radius η/epsilon1 with centers in X /epsilon1 is less than C /epsilon1 -d .

Definition 1 represents the number of near-optimal states for a function f ( · ) around its optimal solution. It is important to note that d is not an intrinsic property of f ( · ) as we are packing nearoptimal states using l -balls. Instead, it characterizes both f ( · ) and l and depends on the constant. In order to relate this measure to the algorithmic details, we also need to correlate it with the characteristics of the partitioning, specifically the shape of the cells. That is, the near-optimality dimension d is dependent on a particular constant, which will be determined in accordance with the parameter ν as defined in Assumption 3.

Lemma 1. Let d be the ν -near-optimality dimension, and C the corresponding constant. Then | I h | ≤ C δ ( h ) -d .

Building upon Lemma 1, we further analyze the loss of the MCTS-based GNN explanation methods across n iterations in Theorem 1.

Theorem 1. Let us write h ( n ) the smallest integer h such that C Σ h l =0 δ ( l ) ( -d ) ≥ n , then the loss of the MCTS-based GNN explanation methods is bounded as:

The loss r n reflects the gap between the obtained and optimal explanations over n iterations. We aim to bound this loss by δ ( h ( n )) , illustrating that refining the partition of the search space X reduces the loss, thus better approximating the optimal explanation. We provide a complete description of the mathematical properties and theorem proofs of the framework in Appendix B.2.

6KDSOH\GLYPH<c=3,font=/NFUROV+font000000002b5a5e15>YDOXHGLYPH<c=29,font=/NFUROV+font000000002b5a5e15>GLYPH<c=3,font=/NFUROV+font000000002b5a5e15>GLYPH<c=19,font=/NFUROV+font000000002b5a5e15>GLYPH<c=17,font=/NFUROV+font000000002b5a5e15>GLYPH<c=28,font=/NFUROV+font000000002b5a5e15>GLYPH<c=21,font=/NFUROV+font000000002b5a5e15>

6KDSOH\GLYPH<c=3,font=/NFUROV+font000000002b5a5e15>YDOXHGLYPH<c=29,font=/NFUROV+font000000002b5a5e15>GLYPH<c=3,font=/NFUROV+font000000002b5a5e15>GLYPH<c=19,font=/NFUROV+font000000002b5a5e15>GLYPH<c=17,font=/NFUROV+font000000002b5a5e15>GLYPH<c=27,font=/NFUROV+font000000002b5a5e15>GLYPH<c=21,font=/NFUROV+font000000002b5a5e15>

6KDSOH\GLYPH<c=3,font=/NFUROV+font000000002b5a5e15>YDOXHGLYPH<c=29,font=/NFUROV+font000000002b5a5e15>GLYPH<c=3,font=/NFUROV+font000000002b5a5e15>GLYPH<c=19,font=/NFUROV+font000000002b5a5e15>GLYPH<c=17,font=/NFUROV+font000000002b5a5e15>GLYPH<c=26,font=/NFUROV+font000000002b5a5e15>GLYPH<c=27,font=/NFUROV+font000000002b5a5e15>

6KDSOH\GLYPH<c=3,font=/NFUROV+font000000002b5a5e15>YDOXHGLYPH<c=29,font=/NFUROV+font000000002b5a5e15>GLYPH<c=3,font=/NFUROV+font000000002b5a5e15>GLYPH<c=19,font=/NFUROV+font000000002b5a5e15>GLYPH<c=17,font=/NFUROV+font000000002b5a5e15>GLYPH<c=25,font=/NFUROV+font000000002b5a5e15>GLYPH<c=26,font=/NFUROV+font000000002b5a5e15>

6KDSOH\GLYPH<c=3,font=/NFUROV+font000000002b5a5e15>YDOXHGLYPH<c=29,font=/NFUROV+font000000002b5a5e15>GLYPH<c=3,font=/NFUROV+font000000002b5a5e15>GLYPH<c=19,font=/NFUROV+font000000002b5a5e15>GLYPH<c=17,font=/NFUROV+font000000002b5a5e15>GLYPH<c=26,font=/NFUROV+font000000002b5a5e15>GLYPH<c=26,font=/NFUROV+font000000002b5a5e15>

2SWLPDOGLYPH<c=3,font=/TXYVKS+font000000002b5a5e15>H[SODQDWLRQ

6XERSWLPDOGLYPH<c=3,font=/TXYVKS+font000000002b5a5e15>H[SODQDWLRQV

1RGHGLYPH<c=3,font=/TXYVKS+font000000002b5a5e15>WKDWGLYPH<c=3,font=/TXYVKS+font000000002b5a5e15>PDWFKGLYPH<c=3,font=/TXYVKS+font000000002b5a5e15>WKHGLYPH<c=3,font=/TXYVKS+font000000002b5a5e15>JURXQGGLYPH<c=3,font=/TXYVKS+font000000002b5a5e15>WUXWKGLYPH<c=3,font=/TXYVKS+font000000002b5a5e15>H[SODQDWLRQ

1RGHGLYPH<c=3,font=/TXYVKS+font000000002b5a5e15>WKDWGLYPH<c=3,font=/TXYVKS+font000000002b5a5e15>

GRGLYPH<c=3,font=/KFRAVV+font000000002b5a5e15>QRWGLYPH<c=3,font=/KFRAVV+font000000002b5a5e15>

PDWFKGLYPH<c=3,font=/TXYVKS+font000000002b5a5e15>WKHGLYPH<c=3,font=/TXYVKS+font000000002b5a5e15>JURXQGGLYPH<c=3,font=/TXYVKS+font000000002b5a5e15>WUXWKGLYPH<c=3,font=/TXYVKS+font000000002b5a5e15>H[SODQDWLRQ

Leveraging the mathematical underpinnings provided above, as demonstrated in Figure 1, we employ an example to contextualize the theoretical insights within the GNN explanation. The ground truth

explanation is highlighted in yellow which includes two connected components. When searching for explanations starting from any node in the two components through the pruning-based MCTS, the nodes in other components are accessible only via the unimportant node which is highlighted in blue. In this situation, given sparsity constraints, the pruning-based MCTS can only generate the suboptimal explanation. As a consequence, regardless of the search trajectory adopted, the diameter of the l -ball remains unyielding, failing to converge to the η -near optimality numerical solution. Therefore, it is necessary to design an explanation method that can accurately retain important nodes while avoiding irrelevant nodes, thus increasing the likelihood of discovering the optimal explanation. This ambition resonates with previously proposed loss bound r n ≤ δ ( h ( n )) , emphasizing the need for advanced exploration to reduce losses and approximate the optimal explanation with higher precision.

ǤǤǤ

GLYPH<c=9,font=/GMONAG+font000000002b5a5e15>GLYPH<c=3,font=/GMONAG+font000000002b5a5e15>GLYPH<c=3,font=/GMONAG+font000000002b5a5e15>GLYPH<c=5,font=/GMONAG+font000000002b5a5e15>

GLYPH<c=8,font=/GMONAG+font000000002b5a5e15>

GLYPH<c=21,font=/GMONAG+font000000002b5a5e15> GLYPH<c=3,font=/GMONAG+font000000002b5a5e15>GLYPH<c=17,font=/GMONAG+font000000002b5a5e15>

GLYPH<c=15,font=/GMONAG+font000000002b5a5e15>GLYPH<c=3,font=/GMONAG+font000000002b5a5e15>GLYPH<c=17,font=/GMONAG+font000000002b5a5e15>

ǤǤǤ

灤

𝑀ଵ

GLYPH<c=24,font=/GMONAG+font000000002b5a5e15>

GLYPH<c=18,font=/GMONAG+font000000002b5a5e15>

GLYPH<c=8,font=/GMONAG+font000000002b5a5e15>

ǤǤǤ

GLYPH<c=24,font=/GMONAG+font000000002b5a5e15>

ȋȌ

ȋȌ

ȋȌ

GLYPH<c=8,font=/GMONAG+font000000002b5a5e15>GLYPH<c=3,font=/GMONAG+font000000002b5a5e15>GLYPH<c=6,font=/GMONAG+font000000002b5a5e15>GLYPH<c=3,font=/GMONAG+font000000002b5a5e15>GLYPH<c=22,font=/GMONAG+font000000002b5a5e15>

灤

𝑀ଶ

GLYPH<c=21,font=/GMONAG+font000000002b5a5e15> GLYPH<c=3,font=/GMONAG+font000000002b5a5e15>GLYPH<c=17,font=/GMONAG+font000000002b5a5e15>

GLYPH<c=15,font=/GMONAG+font000000002b5a5e15>GLYPH<c=3,font=/GMONAG+font000000002b5a5e15>GLYPH<c=17,font=/GMONAG+font000000002b5a5e15>

GLYPH<c=17,font=/GMONAG+font000000002b5a5e15>ǦGLYPH<c=3,font=/GMONAG+font000000002b5a5e15>

GLYPH<c=23,font=/GMONAG+font000000002b5a5e15> GLYPH<c=3,font=/GMONAG+font000000002b5a5e15>GLYPH<c=8,font=/GMONAG+font000000002b5a5e15>

GLYPH<c=12,font=/GMONAG+font000000002b5a5e15>

GLYPH<c=22,font=/GMONAG+font000000002b5a5e15>GLYPH<c=3,font=/GMONAG+font000000002b5a5e15>

GLYPH<c=22,font=/GMONAG+font000000002b5a5e15>

GLYPH<c=17,font=/GMONAG+font000000002b5a5e15>ǦGLYPH<c=3,font=/GMONAG+font000000002b5a5e15>

GLYPH<c=23,font=/GMONAG+font000000002b5a5e15> GLYPH<c=3,font=/GMONAG+font000000002b5a5e15>GLYPH<c=8,font=/GMONAG+font000000002b5a5e15>

GLYPH<c=22,font=/GMONAG+font000000002b5a5e15>ǦGLYPH<c=3,font=/GMONAG+font000000002b5a5e15>

GLYPH<c=23,font=/GMONAG+font000000002b5a5e15> GLYPH<c=3,font=/GMONAG+font000000002b5a5e15>GLYPH<c=8,font=/GMONAG+font000000002b5a5e15>

GLYPH<c=22,font=/GMONAG+font000000002b5a5e15>ǦGLYPH<c=3,font=/GMONAG+font000000002b5a5e15>

GLYPH<c=23,font=/GMONAG+font000000002b5a5e15> GLYPH<c=3,font=/GMONAG+font000000002b5a5e15>GLYPH<c=8,font=/GMONAG+font000000002b5a5e15>

4 Structure-aware Shapley-based Multipiece Explanation Method

As we discussed in Section 3, the structure-aware Shapley-based multipiece explanation provides a potential way to effectively uncover the GNN black box. In order to approximate the optimal Shapleybased explanation, we propose a two-phase framework, Structure-aware Shapley-based Multipiece Explanation (SAME) method, which is composed of (1) an important substructure initialization phase and (2) an explanation exploration phase, as shown in Figure. 2.

In the first phase (Section 4.1), we extend an expansion-based Monte Carlo tree search as an important substructure initializer to generate connected components not only of high importance but also of multi-grained diversity. In the second phase (Section 4.2), we apply the important substructure set as an action set in another expansion-based Monte Carlo tree search to explore potential explanations.

4.1 Important Substructure Initialization

In this section, we propose an expansion-based MCTS approach for important substructure initialization. It is intuitive that a favorable initialization should not exclude important substructures at any scale and should not include redundant substructures. Formally, these can be defined as:

Property 5. (Node-wise importance). Given an input graph G to be explained, for any node v i ∈ G , its I ( f ( · ) , v i , G ) importance will be considered.

Property 6. (Substructure-wise importance). Given an input graph G to be explained, for any substructure G sub i ⊆ G , its importance I ( f ( · ) , G sub i , G ) will be considered.

Property 7. (Composite-wise importance). Given an explanation G i com ⊆ G consisting of one or more substructures, its importance I ( f ( · ) , G i com , G ) will be considered.

Property 8. (Priority-based integration). Given an explanation G j ex with any size, the node v i ∈ { G \ G j ex } will be added on G j ex to get a new explanation G k ex , if and only if for any v l ∈ { G \ ( G j ex ∪ v i ) } , I ( f ( · ) , G j ex ∪ v i , G ) > I ( f ( · ) , G j ex ∪ v l , G ) holds.

Property 9. (Redundancy consideration). Given an explanation G ex ⊆ G , if I ( f ( · ) , G ex \{ i } , G ) >I ( f ( · ) , G ex , G ) holds, the new explanation G ′ ex = G ex \{ i } will be chosen.

Taking the above properties into account, we propose our expansion-based Monte Carlo tree search for important substructure initialization which aims at providing important connected components of the graph. The detailed Algorithm is presented in Appendix C. Given a graph G to be explained, the node in the MCTS is defined as N i which contains the following variables:

where G i sub denotes the corresponding substructure of graph G for node N i in the search tree. T i is the visiting time of node N i in the search tree, R i refers to the the importance (reward) of substructure G i sub . A i represents the action set of node N i . C i = { N j } | A i | j =1 represents a set of children with respect to node N i , and we denote N j as one of the child nodes obtained through the action a j ∈ A i at parent node N i . W i means the sum of the children's rewards.

The expansion-based MCTS is initialized with an empty set, N 0 = ∅ . At the beginning of each iteration, our method will randomly choose an unvisited node from the graph, or choose the node with the highest reward if all nodes have been visited. Then, the tree will be iteratively expanded according to the node-wise tree expansion until it reaches a leaf node. Specifically, given an MCTS node N i , it will only choose the child node within 1-hop neighbors of the associate G i sub to expand, which means that the action set is topology dependent. Therefore, the substructure of any nodes in the search tree will be a connected component. During the child node selection, the reward of each child node R j of N i will be calculated following Equation (2). Finally, the chosen action is decided following the child selection strategy:

where β is a hyperparameter for balancing exploration-exploitation trade-off [4]. We define the maximum substructure size as γ , and for any G i sub ⊆ G , | G i sub | ≤ γ always holds. After reaching the maximum substructure size, the reward of associated leaf nodes is backpropagated up the tree along the search path, updating all the information stored in each node of the path. The important substructure set includes all nodes in MCTS after performing M 1 times iterations.

4.2 Explanation Exploration

In this section, we propose another expansion-based MCTS for exploring high-explainable explanations. The detailed algorithm is provided in Appendix C. Different from the previous expansion-based MCTS provided in Section 4.1, we propose a slight modification on it such that we change the action set from node level to substructure level. As Section 3.2 discussed, this can be useful not only to obtain more flexible explanation results on real-world cases but also to provide the higher potential to approximate the theoretically optimal Shapley-based explanation.

The action set of MCTS in this phase is built upon the substructure set derived from the important substructure initialization phase. Similar to the previous MCTS, at the beginning of each iteration, it will randomly select an unvisited substructure from the set, or choose the substructure with the highest reward to expand if all substructures have been visited. Afterward, the substructure-wise tree expansion also follows the Equation (2) and (9) to develop the tree and provide the explanation candidate. The action set corresponding to each node of MCTS in the phase is the whole substructure set. To further accelerate the exploration, we filter the unimportant substructures by only keeping the top K important substructure in the action set.

Notice that calculating all possible combinations of either substructures or nodes can definitely obtain the optimal Shapley-based explanation. Nevertheless, such node-wise or substructure-wise brute force methods lead to O (2 | V | ) computational complexity, which is an NP-hard problem. We provide a feasible solution to approximate the optimal Shapley-based explanation in polynomial time O ( M 1 γ | V | 2 + M 1 γ | V | × | E | + M 2 Kt s | V | × (1 -sparsity ) γ ) , where M 1 and M 2 denote maximum number of iterations of the first phase and second phase respectively, γ is the maximum substructure size, K refers to the size of the important substructure set and t s is the time budget of MCTS in the second phase. We leave the derivation of the time complexity in Appendix B.4.

Note: The fidelity results are averaged across different sparsity from 0.5 to 0.8. The quantitative results are presented in the form of mean ± std. The previous SOTA results on different datasets are marked with an underline. Relative Improve denotes the relative improvement of our SAME method over the SOTA methods.

5 Experiments

Our objective of the experiments is to understand the following two questions. 1) Are the explanations provided by SAME more informative and faithful compared to other methods under the same test conditions? 2) Can SAME provide a more human-intuitive explanation than others? To this end, we perform extensive quantitative and qualitative analysis to evaluate the explanatory power of SAME, following previous literature [38, 40]. The SAME is compared with various competitive baselines and shows state-of-the-art (SOTA) results in all the cases.

Dataset . The experiments are conducted on six datasets with diverse categories, including molecular graphs (e.g., BBBP [33] and MUTAG [7]), sentiment graphs (e.g., Graph-SST2 and Graph-SST5 [29]) and synthetic Barabási-Albert graphs (e.g., BA-2Motifs [19] and BA-Shapes [35]). We conduct a node classification task for the BA-Shapes dataset, and graph classification tasks for the rest five datasets. More detailed descriptions of datasets are provided in Appendix D.

Metrics . In this work, we use several criteria [37] to evaluate our approach: (1) Sparsity quantifies how compact are the explanations, and further facilitates fair comparison by restricting the different explanations to the same size. (2) Fidelity determines how informative and faithful are the explanations by removing the selected nodes. (3) Inv-Fidelity measures the explanations from the same aspect as fidelity while it keeps the selected nodes. (4) Harmonic fidelity [40] normalizes fidelity by sparsity and takes a harmonic mean to make different explanations comparable with a single metric. We leave detailed mathematical definitions of the above metrics in Appendix E.

Experimental setup . In the important substructure initialization phase, we set the MCTS iteration number M 1 to 20. The exploration-exploitation trade-off β is set to 5 for BBBP and 10 for other datasets. The substructure size γ has different settings in different datasets. In the explanation exploration phase, we set the hyperparameter K = 7 for important substructure filtering, M 2 = 10 for the MCTS iteration number, and the other hyperparameters of MCTS remain the same as the previous phase. We follow [37, 40] to set other baselines hyperparameters. All methods are implemented in PyTorch [21] and PyG [10]. Our experiments are conducted on a single Nvidia V100 GPU with an Intel Xeon Gold 5218 CPU. We leave the detailed settings in Appendix F.

Note: The PGExplainer needs training before inferring the explanation.

Sentence: 'a carefully structured scream of consciousness that is tortured and unsettling -- but unquestionably alive.'

Grad-CAM

cons.

that

and

but

of

struct.

carefully

tortured

is

unset.

--

alive

unques.

scream

a

SAME(ours)

cons.

that

and

but

of

struct.

carefully

tortured

is

unset.

--

alive

unques.

scream

a

cons.

that

and

but

of

struct.

carefully

tortured

is

unset.

--

alive

unques.

scream

a

cons.

that

and

but

of

struct.

carefully

tortured

is

unset.

--

alive

unques.

scream

a

SubgraphX

GNN-LRP

GStarX

cons.

that

and

but

of

struct.

carefully

tortured

is

unset.

--

alive

unques.

scream

a

cons.

that

and

but

of

struct.

carefully

tortured

is

unset.

--

alive

unques.

scream

a

cons.

that

and

but

of

struct.

carefully

tortured

is

unset.

--

alive

unques.

scream

a

GNNExplainer

PGExplainer

5.1 Quantitative Analysis

To validate the overall explainability performance, we compare the proposed SAME with a series of competitive baselines under different metrics. Table 2 shows the averaged fidelity under different sparsity ( i.e. , sparsity=[0.5,0.6,0.7,0.8]). The proposed SAME significantly outperforms the previous state-of-the-art on both real-world and synthetic datasets. Specifically, the performance improvement of SAME is 12.9% on BBBP, 7.01% on MUTAG, 42.3% on Graph-SST2, 38.9% on Graph-SST5, 11.3% on BA-2Motifs and 18.2% on BA-Shapes. Notably, we also demonstrate reliable improvements of SAME over previous SOTA methods in terms of harmonic fidelity at different sparsities, with an average improvement of 1.92% on the graph classification task. This result implies SAME has a higher ability to discover important components in the graph than all baseline methods. As for the inv-fidelity metric, the explanatory power of SAME is competitive compared with the previous SOTA. Detailed results for harmonic fidelity and inv-fidelity are in Appendix G.1. Since the proposed SAME is a model-agnostic method, it works well with GAT and GIN. More comparisons with GraphSVX [8] and OrphicX [16] are provided in Appendix G.3.

The computational cost of SAME and other baselines on different datasets are summarized in Table 3. We show that SAME consistently achieves much lower computational cost compared to GStarX and SubgraphX, reflecting its efficiency and robustness. This further verifies that expansion-based MCTS in SAME can work effectively in various scenarios.

5.2 Qualitative Analysis

Figure 3 presents the visualization comparison of the explanations on sentiment graphs. The nodes of adjectives and adverbs are considered to be important for they reveal the attitude of a sentence. Thus, the optimal explanation here is therefore the word or phrase with a positive meaning. In this sense,

6$0(GLYPH<c=11,font=/ICQAQI+font000000002b5a5e28>RXUVGLYPH<c=12,font=/ICQAQI+font000000002b5a5e28>

6XEJUDSK;

*6WDU;

*11([SODLQHU

*UDGGLYPH<c=16,font=/MICTZN+font000000002b5a5e28>&$0

3*([SODLQHU

*11GLYPH<c=16,font=/MICTZN+font000000002b5a5e28>/53

E

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

E

K

K

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

K

E

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

E

K

K

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

K

E

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

E

K

K

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

K

E

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

E

K

K

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

K

E

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

E

K

K

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

K

E

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

E

K

K

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

K

E

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

E

K

K

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

K

E

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

E

K

K

GLYPH<c=18,font=/BYUSWN+font000000002b5a5e28>

K

*URXQGGLYPH<c=3,font=/ICQAQI+font000000002b5a5e28>7 UXWK

Fidelity: -0.03

Sparsity:0.81

GNN-LRP

Fidelity: 0.02

Sparsity:0.81

Grad-CAM

Fidelity: 0.28

Sparsity:0.82

SubgraphX

Fidelity: 0.80

Sparsity:0.84

GStarX

Fidelity: 0.99

Sparsity:0.77

SAME(ours)

Fidelity: 0.38

Sparsity:0.70

GNNExplainer

Fidelity: 0.35

Sparsity:0.80

PGExplainer

Fidelity: 0.99

Sparsity:0.77

Ground Truth

SAME can better capture the adjectives-or-adverbs-like graph structures than other baselines. For instance, SubgraphX focuses on adjectives and adverbs but fails to capture the "but" word which bears significant weight under the contrasting relationship. Intuitively, without the "but", the contribution of "tortured" and "unsettling" should be negative. GStarX achieves to identify words that are consistent with the opinion such as "carefully" and "alive," yet overlooks the crucial transitional relationship between "alive" and "unsettling". Figure 4 shows the visualization of explanations on molecular graphs. The ground truth explanations ( i.e. , functional group -NO 2 ) of MUTAG are labelled by human experts. We see that SAME is able to provide the explanations the same as the ground truth. We also illustrate the explanation of the synthetic graph in Figure 5. The ground-truth label of all the graphs in BA-2Motifs is a 5-node-house-structure motif. Results show that SAME exactly finds the ground-truth explanation compared to other baselines. We leave more comparisons in Appendix G.2.

6 Conclusion

Structure-aware Shapley-based Multipiece Explanation provides strong explainability over GNN models, while this ability is limited by only using the single connected substructure. Moving forward from the theoretical deduction, we propose the SAME method for explaining GNN, a novel perturbation-based method that is aware of input graph structure, feature interactions, and multi-grained importance. Experimental results demonstrate that our SAME consistently outperforms SOTA methods on multiple benchmarks by a large margin with various metrics and provides a more understandable explanation.

Limitations. In the implementation of SAME, the Shapley value is obtained through approximation following [38]. Under this approximation, the fairness axioms discussed in Section 3.1 no longer hold. This is also identified as an unsolved issue for the Shapley value in machine learning by [23]. In addition, the scalability of SAME on large graphs can also become a potential challenge. As the time complexity shown in Section 4, the time overhead caused by the increase in the number of nodes will become very expensive when scaling up the size of the input graph.

References

S. Ali, T. Abuhmed, S. El-Sappagh, K. Muhammad, J. M. Alonso-Moral, R. Confalonieri, R. Guidotti, J. Del Ser, N. Díaz-Rodríguez, and F. Herrera. Explainable artificial intelligence (xai): What we know and what is left to attain trustworthy artificial intelligence. Information Fusion , page 101805, 2023.

M. Bajaj, L. Chu, Z. Y. Xue, J. Pei, L. Wang, P. C.-H. Lam, and Y. Zhang. Robust counterfactual explanations on graph neural networks. Advances in Neural Information Processing Systems , 34:56445655, 2021.

F. Baldassarre and H. Azizpour. Explainability techniques for graph convolutional networks. In International Conference on Machine Learning (ICML) Workshops, 2019 Workshop on Learning and Reasoning with Graph-Structured Representations , 2019.

C. B. Browne, E. Powley, D. Whitehouse, S. M. Lucas, P. I. Cowling, P. Rohlfshagen, S. Tavener, D. Perez, S. Samothrakis, and S. Colton. A survey of monte carlo tree search methods. IEEE Transactions on Computational Intelligence and AI in Games , 4(1):1-43, 2012.

I. Covert, S. M. Lundberg, and S.-I. Lee. Understanding global feature contributions with additive importance measures. Advances in Neural Information Processing Systems , 33:17212-17223, 2020.

I. C. Covert, S. Lundberg, and S.-I. Lee. Explaining by removing: A unified framework for model explanation. The Journal of Machine Learning Research , 22(1):9477-9566, 2021.

A. K. Debnath, R. L. Lopez de Compadre, G. Debnath, A. J. Shusterman, and C. Hansch. Structure-activity relationship of mutagenic aromatic and heteroaromatic nitro compounds. correlation with molecular orbital energies and hydrophobicity. Journal of Medicinal Chemistry , 34(2):786-797, 1991.

A. Duval and F. D. Malliaros. Graphsvx: Shapley value explanations for graph neural networks. In Machine Learning and Knowledge Discovery in Databases. Research Track: European Conference, ECML PKDD 2021, Bilbao, Spain, September 13-17, 2021, Proceedings, Part II 21 , pages 302-318. Springer, 2021.

Q. Feng, N. Liu, F. Yang, R. Tang, M. Du, and X. Hu. Degree: Decomposition based explanation for graph neural networks. In International Conference on Learning Representations , 2022.

M. Fey and J. E. Lenssen. Fast graph representation learning with pytorch geometric. arXiv preprint arXiv:1903.02428 , 2019.

T. Funke, M. Khosla, M. Rathee, and A. Anand. Zorro: Valid, sparse, and stable explanations in graph neural networks. IEEE Transactions on Knowledge and Data Engineering , 2022.

Q. Huang, M. Yamada, Y. Tian, D. Singh, and Y. Chang. Graphlime: Local interpretable model explanations for graph neural networks. IEEE Transactions on Knowledge and Data Engineering , 2022.

R. Kleinberg, A. Slivkins, and E. Upfal. Multi-armed bandits in metric spaces. In Proceedings of the fortieth annual ACM symposium on Theory of computing , pages 681-690, 2008.

I. E. Kumar, S. Venkatasubramanian, C. Scheidegger, and S. Friedler. Problems with shapley-value-based explanations as feature importance measures. In International Conference on Machine Learning , pages 5491-5500. PMLR, 2020.

W. Lin, H. Lan, and B. Li. Generative causal explanations for graph neural networks. In International Conference on Machine Learning , pages 6666-6679. PMLR, 2021.

W. Lin, H. Lan, H. Wang, and B. Li. Orphicx: A causality-inspired latent variable model for interpreting graph neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 13729-13738, 2022.

A. Lucic, M. A. Ter Hoeve, G. Tolomei, M. De Rijke, and F. Silvestri. Cf-gnnexplainer: Counterfactual explanations for graph neural networks. In International Conference on Artificial Intelligence and Statistics , pages 4499-4511. PMLR, 2022.

S. M. Lundberg and S.-I. Lee. A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems , 30, 2017.

D. Luo, W. Cheng, D. Xu, W. Yu, B. Zong, H. Chen, and X. Zhang. Parameterized explainer for graph neural network. Advances in Neural Information Processing Systems , 33:19620-19631, 2020.

R. Munos et al. From bandits to monte-carlo tree search: The optimistic principle applied to optimization and planning. Foundations and Trends® in Machine Learning , 7(1):1-129, 2014.

A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer. Automatic differentiation in pytorch. 2017.

P. E. Pope, S. Kolouri, M. Rostami, C. E. Martin, and H. Hoffmann. Explainability methods for graph convolutional neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 10772-10781, 2019.

B. Rozemberczki, L. Watson, P. Bayer, H.-T. Yang, O. Kiss, S. Nilsson, and R. Sarkar. The shapley value in machine learning. arXiv preprint arXiv:2202.05594 , 2022.

M. S. Schlichtkrull, N. De Cao, and I. Titov. Interpreting graph neural networks for nlp with differentiable edge masking. In International Conference on Learning Representations , 2021.

T. Schnake, O. Eberle, J. Lederer, S. Nakajima, K. T. Schütt, K.-R. Müller, and G. Montavon. Higher-order explanations of graph neural networks via relevant walks. IEEE Transactions on Pattern Analysis and Machine Intelligence , 44(11):7581-7596, 2021.

R. Schwarzenberg, M. Hübner, D. Harbecke, C. Alt, and L. Hennig. Layerwise relevance visualization in convolutional text graph classifiers. EMNLP-IJCNLP 2019 , page 58, 2019.

R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE International Conference on Computer Vision , pages 618-626, 2017.

L. S. Shapley et al. A value for n-person games. 1953.

R. Socher, A. Perelygin, J. Wu, J. Chuang, C. D. Manning, A. Y. Ng, and C. Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing , pages 1631-1642, 2013.

M. Vu and M. T. Thai. Pgm-explainer: Probabilistic graphical model explanations for graph neural networks. Advances in Neural Information Processing Systems , 33:12225-12235, 2020.

X. Wang, Y. Wu, A. Zhang, X. He, and T.-S. Chua. Towards multi-grained explainability for graph neural networks. Advances in Neural Information Processing Systems , 34:18446-18458, 2021.

Z. Wang, L. Cao, W. Lin, M. Jiang, and K. C. Tan. Robust graph meta-learning via manifold calibration with proxy subgraphs. In AAAI Conference on Artificial Intelligence , 2023.

Z. Wu, B. Ramsundar, E. N. Feinberg, J. Gomes, C. Geniesse, A. S. Pappu, K. Leswing, and V. Pande. Moleculenet: a benchmark for molecular machine learning. Chemical Science , 9(2):513-530, 2018.

Z. Ye, Y. Qu, Z. Liang, M. Wang, and Q. Liu. Explainable fmri-based brain decoding via spatial temporalpyramid graph convolutional network. Human Brain Mapping , 44(7):2921-2935, 2023.

Z. Ying, D. Bourgeois, J. You, M. Zitnik, and J. Leskovec. Gnnexplainer: Generating explanations for graph neural networks. Advances in Neural Information Processing Systems , 32, 2019.

H. Yuan, J. Tang, X. Hu, and S. Ji. Xgnn: Towards model-level explanations of graph neural networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , pages 430-438, 2020.

H. Yuan, H. Yu, S. Gui, and S. Ji. Explainability in graph neural networks: A taxonomic survey. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2022.

H. Yuan, H. Yu, J. Wang, K. Li, and S. Ji. On explainability of graph neural networks via subgraph explorations. In International Conference on Machine Learning , pages 12241-12252. PMLR, 2021.

H. Zhang, B. Wu, X. Yuan, S. Pan, H. Tong, and J. Pei. Trustworthy graph neural networks: Aspects, methods and trends. arXiv preprint arXiv:2205.07424 , 2022.

S. Zhang, Y. Liu, N. Shah, and Y. Sun. Gstarx: Explaining graph neural networks with structure-aware cooperative games. In Advances in Neural Information Processing Systems , 2022.

Y. Zhang, D. Defazio, and A. Ramesh. Relex: A model-agnostic relational model explainer. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society , pages 1042-1049, 2021.

Appendix to 'SAME: Uncovering GNN Black Box with Structure-aware Shapley-based Multipiece Explanation'

Contents

This appendix comprises both theoretical and experimental materials and is structured as follows. Section A presents a notation table that includes all notations used in this paper. Section B provides the full theoretical analysis. Section C presents detailed algorithms related to Section 3 in the manuscript. Section D-F show the detailed settings of the experiments in this work. Section G provides additional quantitative and qualitative results.

A Notations

Table S1 contains the notations organized into three sections, each pertaining to di ff erent aspects of our paper. These include:

Graph : These symbols are related to graph theory, representing elements such as graph nodes, edges, and other attributes of graphs.

MCTS : These symbols represent elements related to the Monte Carlo Tree Search (MCTS) algorithm, such as MCTS nodes, actions, and rewards.

Importance score : These symbols are used to denote values associated with measuring the importance of explanations in our explanation algorithm.

1

B Theoretical Analysis

B.1 Definitions of Desire GNN Explanation Technique Properties

The following content is a detailed definition of desired properties in Table 1 of the main manuscript.

Graph-level tasks : the GNN explanation method can handle the graph classification/regression tasks.

Node-level tasks : the GNN explanation method can handle the node classification/regression tasks.

Feature interactions : given a graph G to be explained, when measuring the importance of the explanation result G ex , the GNN explanation method can consider the influence of { G/G ex } on the importance of G ex .

Structure awareness : given a graph G to be explained, when measuring the importance of the explanation result G ex , the GNN explanation method is sensitive to the topology of the given input graph G .

Multipiece explanation : the GNN explanation method can provide the explanation G ex that can be composed of one or more connected components.

Node-wise importance : given any node v i ∈ G , its importance I ( f ( · ) , v i , G ) can be considered by the GNN explanation method.

Substructure-wise importance : given any substructure G i sub ⊆ G , it can be directly calculated as a whole through the GNN explanation method to obtain its importance I ( f ( · ) , G i sub , G ) . Note that any substructure is a connected component which has more than one node.

2

Composite-wise importance : given any composite G i com ⊆ G consisting of more than two nodes or substructures, it can be directly calculated as a whole through the GNN explanation method to obtain its importance I ( f ( · ) , G i com , G ) .

Priority-based integration : given an explanation G j ex ⊆ G with any size, the node v i ∈ { G/G j ex } will be added by the GNN explanation method on G j ex to get a new explanation, if and only if for any node v k ∈ { G/G j ex } except v i , I ( f ( · ) , { v i ∪ G j ex } , G ) ≥ I ( f ( · ) , { v k ∪ G j ex } , G ) always holds.

Redundancy consideration : given an explanation G j ex ⊆ G , if there exist a node v i ∈ { G/G j ex } that satisfies I ( f ( · ) , { G j ex /v i } , G ) ≥ I ( f ( · ) , G j ex , G ) , the GNN explanation technique would not choose G j ex as the final explanation.

%UDLQGLYPH<c=3,font=/IHDNFK+font000000002b5a5e2b>1HWZRUNVGLYPH<c=3,font=/IHDNFK+font000000002b5a5e2b> $VVRFLDWHGGLYPH<c=3,font=/IHDNFK+font000000002b5a5e2b> ZLWKGLYPH<c=3,font=/IHDNFK+font000000002b5a5e2b>6SHFLILFGLYPH<c=3,font=/IHDNFK+font000000002b5a5e2b>&RJQLWLYHGLYPH<c=3,font=/IHDNFK+font000000002b5a5e2b>7DVN

)XQFWLRQDOGLYPH<c=3,font=/IHDNFK+font000000002b5a5e2b>*URXSVGLYPH<c=3,font=/IHDNFK+font000000002b5a5e2b> $VVRFLDWHGGLYPH<c=3,font=/IHDNFK+font000000002b5a5e2b> ZLWKGLYPH<c=3,font=/IHDNFK+font000000002b5a5e2b>0ROHFXODUGLYPH<c=3,font=/IHDNFK+font000000002b5a5e2b>3URSHUW\

6RFLDOGLYPH<c=3,font=/IHDNFK+font000000002b5a5e2b>*URXSVGLYPH<c=3,font=/IHDNFK+font000000002b5a5e2b> $VVRFLDWHGGLYPH<c=3,font=/IHDNFK+font000000002b5a5e2b> ZLWKGLYPH<c=3,font=/IHDNFK+font000000002b5a5e2b>6LQJOHGLYPH<c=3,font=/IHDNFK+font000000002b5a5e2b>3HUVRQ

S

O

n

As Figure S1 shows, the characteristics and properties within a graph or node tend to be jointly influenced by more than one high-order connected community of the graph. The "multipiece explanation" property is aware of such an attribute by considering whether multiple disconnected pieces are allowed to appear together in the explanation output. For example, when "A" and "B" are two orthogonal connected components and the ground truth explanation for a prediction is "A and B", then the GNN explanation method has a probability of providing the correct "A and B" explanation.

We call a GNN explanation method that has the "structure-awareness" property if it can process the structural information of the input graph, whether through explicit mechanisms, as demonstrated by GstarX, or implicit modalities, as exemplified by SubgraphX and this study.

B.2 Discussion about Four Fairness Properties

E ffi ciency demonstrates that the aggregated importance of individual substructures is equivalent to the prediction of GNN on the entire substructure set. Symmetry and Dummy take into account the equal importance of substructures with the same interactions and the substructures that are completely unimportant, respectively. Monotonicity ensures that the explanation method is consistent in the trend of results between di ff erent well-trained GNN models given the same dataset.

B.3 Detailed Proof of Regret Bound

The proper definition of the regret phenomenon that the algorithm or the function f encounters must be considered to make the objective clear. Prior studies have recognized two principal definitions: We recall the Lemma 1 in the main manuscript as follows.

Lemma 1. Let d be the ν -near-optimality dimension (where ν is defined in Assumption 3 of the main manuscript), and C is the corresponding constant. Then | I h | ≤ C δ ( h ) -d .

Proof. From Assumption 3 in the main manuscript, each cell ( h, i ) contains a ball of radius νδ ( h ) centered at x h,i . Consequently, if | I h | = |{ x h,i ∈ X δ ( h ) }| exceeds C δ ( h ) -d , this would imply the existence of more than C δ ( h ) -d disjoint l -balls of radius νδ ( h ) , each centered within X δ ( h ) . This assertion would contradict the definition of d .

We recall the Theorem 1 in the main manuscript as follows.

3

Theorem 1. Let us write h ( n ) the smallest integer h such that C ∑ h l =0 δ ( l ) -d ≥ n . Then the loss of algorithm is bounded as r n ≤ δ ( h ( n )) .

Proof. Consider the tuple ( h max , j max ) to be the node with the maximum depth that the algorithm expands up to the n th round. Given that the algorithm's expansion is limited to nodes within the set I , we can provide that the total number of nodes expanded, denoted by n , adheres to the following condition:

from Lemma 1. Now from the definition of h ( n ) we have h max ≥ h ( n ) . Finally, since node ( h max , j max ) has been expanded, we have that ( h max , j max ) ∈ I , thus f ( x ( n )) ≥ f ( x h max ,j max ) ≥ f /star -δ ( h max ) ≥ f /star -δ ( h ( n )) .

Corollary 1. Assume that δ ( h ) = c γ h for some constants c > 0 and γ < 1 , if d > 0 , then the loss decreases polynomially fast:

If d = 0 , then the loss decreases exponentially fast:

Proof. From Theorem 1,whenever d > 0 we have

thus γ -dh ( n ) ≥ n Cc -d (1 -γ d ) , therefore we can also have:

Now, if d = 0 then n ≤ C ∑ h ( n ) l =0 δ ( l ) -d = C ( h ( n ) + 1) , and we deduce that the loss is bounded as r n ≤ δ ( h ( n )) = c γ n/C -1 .

B.4 Analysis of Time Complexity

Given a graph G = ( V, E ) , we take a single iteration as an example to analyze the time complexity of the expansion-based MCTS in our proposed SAME. A single iteration includes four steps: selection, expansion, simulation and backpropagation [1].

B.4.1 Important Substructure Initialization Phase

In this phase, the goal of our SAME is to figure out a group of important substructures whose size is smaller than γ . In the selection step , our method chooses an unvisited node from the graph or chooses the node with the highest reward if all nodes have been visited ( O ( | V | ) ). Then, in the expansion step , our method selects the node within 1-hop neighbors whose value is largest

4

according to G , which requires O (1) , and adds it to the tree. In the simulation step , only the adjacency node will be appended each time. For the graph G , in the worst case SAME will append all the nodes by visiting all the edges in the graph. Thus, the time complexity of simulation is O ( | V | + | E | ) . For the backpropagation step , since the maximum size of a substructure is γ , the depth of the MCTS tree will not exceed γ . The time complexity of reward backpropagation is bounded by O ( γ ) . Since we perform M 1 iterations, according to the law of multiplication, the time complexity of the first phase is O ( M 1 ) × O ( selection ) × O ( expansion ) × O ( simulation ) × O ( backpropagation ) = O ( M 1 γ | V | 2 + M 1 γ | V | × | E | ) .

B.4.2 Explanation exploration phase

In the selection step , the MCTS is initialized with an unvisited substructure from Important Substructure Set (size = K , where K /lessmuch | V | ) or chooses the substructure with the highest reward if all substructures have been visited ( O ( K ) ). The expansion step is O (1) as we only need to append the substructure with the largest value to the current state. In the following simulation step , our method will append other substructures to the current state until the size exceeds the sparsity limit, which is bounded by O (2 K ) for finding all possible combination. Notice that O (2 K ) can be very large, we set a maximum simulation time t s as budget which requires O ( t s ) . Since the final explanation is highly related to the sparsity, the cost of backpropagation step is bounded by O ( | V | × (1 -sparsity ) γ )) , where γ is the maximum size of each substructure. With M 2 iterations, the time complexity in this phase is O ( M 2 ) × O ( selection ) × O ( expansion ) × O ( simulation ) × O ( backpropagation ) = O ( M 2 Kt s | V | × (1 -sparsity ) γ ) .

Overall, time complexity of SAME is O ( M 1 γ | V | 2 + M 1 γ | V | × | E | + M 2 Kt s | V | × (1 -sparsity ) γ ) . As the t s , γ , K , M 1 , M 2 and sparsity is predefined, therefore SAME is a polynomial-time method under these constraints.

C Detailed Pseudo Code of SAME

In this section, we are going to provide a detailed overview of the processes of SAME which includes important substructure initialization and explanation exploration .

Algorithm 1 outlines the process of using Monte Carlo Tree Search (MCTS) to find a set of important substructures, as mentioned in Section 3.1 of the main manuscript. Algorithm 2 and 3 illustrate the method to discover an optimal explanation from these important substructures, as described in Section 3.2. Algorithm 4 provides a detailed description of how the importance of an explanation is assessed using the structure-aware Shapley value, which was proposed in Section 2.1 of your document. This algorithm outlines the steps necessary to approximate the Shapley value for a given explanation, e ff ectively measuring the contribution of each individual component via sampling.

D Dataset Description

We provide details of the datasets used in our experiments, including BBBP [11], MUTAG [2], BA2Motifs [8], BA-Shapes [12], Graph-SST2 [7], and Graph-SST5 [7].

Molecular graphs. We use the BBBP [11] containing approximately 2000 molecular graphs, which are classified into two classes over the property of blood-brain barrier penetration (BBBP). Another dataset MUTAG [2] is a collection of molecules with -NO 2 functional groups. The goal is to predict whether these molecules are mutation-induced.

Barabási-Albert graphs. The BA-2Motifs [8] and the BA-Shapes [12] are used for graph classification and node classification respectively. For each instance in BA-2Motifs, it is a Barabási-Albert graph attached by motifs with a structure either house-like or five-node-cycle-like. The instances are labelled according to the type of motifs they get. The graph in BA-Shapes is a Barabási-Albert graph with house-structured network motifs. The nodes in the graph will be classified into four classes, with labels 0 for the nodes belonging to the original graph and labels 1-3 for the nodes on the middle, bottom, or top of the house-like structures respectively.

Sentiment graphs. The sentiment graphs are built from real-world text sequences, and labelled according to the semantic meanings. Specifically, the nodes in the graph represent the words with an

5

Algorithm 1: Important Substructure Initialization

Algorithm 2: Explanation Exploration Input: f ( · ) , well-trained GNN model; G , graph to be explained; N , set of all the nodes in MCTS; K , number of the top most important structures; K t , threshold to use MCTS; M 2 , max iteration number of MCTS; γ 2 , max explanation size; π , child selection strategy. Output: G ex , the best explanation. 1 Sort the N in descending order of the corresponding reward R i . 2 N K ← top K substructures { N 1 , N 2 , . . . , N K } in N 3 G ex ← { ( G sub i , R i ) } M 2 i =1 // Find explanations through MCTS in Algorithm 3. 4 G ′ ex ← MCTS( f ( · ) , G , M 2 , N max , π , N K ) 5 G ex ← G ex ∪ G ′ ex 6 Sort the G ex in descending order of R . // G ex is the explanation at the first element in the sorted G ex 7 G ex ← G ex [0]

6

Algorithm 3: MCTS Explanation Exploration

Algorithm 4: Importance Scoring via Shapley Value

7

initial embedding from the pre-trained BERT [3] and the edges denote the relationships between the words which are extracted by the Bia ffi ne parser [5]. We take experiments on the Graph-SST2 and Graph-SST5 datasets for the sentiment graphs. Note that both Graph-SST2 and Graph-SST5 are the datasets for graph classification of two classes and five classes respectively.

E Metrics

The Fidelity, Inverse Fidelity, and Sparsity are formally defined as:

Harmonic Fidelity (H-Fidelity) takes a harmonic mean of normalized fidelity (N-Fidelity) and normalized inverse fidelity (N-Fidelity Inv ). According to [15], the H-Fidelity can be formally defined as:

F Detailed Experimental Settings

In this section, we provide the computational details and detailed hyperparameter settings on di ff erent datasets.

F.1 Computation Details

All experiments were performed on a single Nvidia V100 GPU with an Intel Xeon Gold 5218 CPU. Since it takes more than 24 hours for some baselines to explain the whole dataset, our inference time analysis is implemented by randomly sampling from the datasets. For BBBP, BA-2Motifs and BA-Shapes, we use all the test data. And all the data (train, evaluation and test data) is used for MUTAG dataset. For semantic graphs like Graph-SST2 and Graph-SST5, we randomly choose 30 graphs.

F.2 Hyperparameter Settings

We adhere to the hyperparameter settings for GNN training as described in [13, 15], detailed in Table S2. For the qualitative analysis, the GCN in Figures 2, 4, S2, and S3 aligns with the settings depicted in Table S2. Moreover, the GIN featured in Figure 3 is trained for 800 epochs with a batch size of 64 and without adding self-loops, while all other hyperparameters remain constant to Table S2.

8

γ

\

K

1

2

3

4

5

1

0.0990

0.0889

0.0874

0.0932

0.0914

3

0.3300

0.3129

0.3986

0.2846

0.2822

5

1.289

1.137

0.8344

0.8483

0.8427

7

5.178

3.817

2.746

2.296

2.191

9

15.95

11.88

7.047

5.330

5.796

The hyperparameters in SAME include γ and K as we didn't use a time budget in our MCTS. In order to see the e ff ect of these two hyperparameters, we run our SAME by randomly selecting 30 graphs from Graph-SST5 datasets using GCN, the fidelity w.r.t. γ and K are shown in the Table S3, and we report the average inference time in Table S4. Notice that since we only selected 30 graphs from Graph-SST5 to obtain the results, the results provided in Table S3 might be slightly di ff erent from the results given in the main manuscript.

It is noteworthy that γ =1 could lead to optimal fidelity in most benchmarks. However, when γ is small, the final explanation may be composed of disconnected nodes at di ff erent positions in the graph, which usually causes the visualization of explanation results not human-understandable. Therefore, we fine-tuned γ and K according to the visualization of explanation results so that the provided explanations are more human-intuitive. For the inference time, a larger K leads to a larger search space for the explanation exploration phase . When γ is small, the size of a single candidate substructure is small, indicating that we need to select more substructures from the candidates to obtain an explanation with the desired sparsity. Thus, the inference time increases as the depth of searching grows. The detailed hyperparameter settings are provided in Table S5. For the sentiment network Twitter [13] and molecular network BACE [11], we use them to compare SAME with more SOTA explainers in Section G.3

G Additional Results

G.1 Results under Other Metrics

Here, we present the results over inverse fidelity and harmonic fidelity under the same experimental settings in Table S2. Table S6 demonstrates that SAME is competitive over the 5/6 datasets. Table S7 reports the harmonic fidelity quantities. SAME outperforms the baselines in all the graph tasks.

G.2 Additional Visualization Results

We are presenting additional visualization results for BBBP and Graph-SST2 datasets. Figure S2 presents explanations for the BBBP dataset. SAME identifies critical functional groups (for instance, carbonyl group =C=O), which result in a high fidelity score. Figure S3 provides a visualization for the negative label of the Graph-SST2 dataset. In this case, SAME uniquely identifies the word "only," which contributes to the negative prediction of the GNN due to the contrasting relationship it establishes.

9

Note: The previous SOTA results on di ff erent datasets are marked with an underline. Relative Improve denotes the relative improvement of our SAME method over the SOTA methods.

↑

↑

↑

↑

↑

Note: The previous SOTA results on di ff erent datasets are marked with an underline. Relative Improve denotes the relative improvement of our SAME method over the SOTA methods.

SAME(ours)

Grad-CAM

GNNExplainer

SubgraphX

GNN-LRP

PGExplainer

GStarX

Fidelity: 0.70

Fidelity: 0.31

Fidelity: 0.99

Fidelity: 0.23

Fidelity: 0.14

Fidelity: 0.03

Fidelity: 0.22

10

Sentence: 'shocking only in that it reveals the filmmaker's bottomless pit of self - absorption.'

it

the

filmma.

's

that

reveals

only

pit

bottom.

of

absorp.

self

-

shocking

in

it

the

filmma.

's

that

reveals

only

pit

bottom.

of

absorp.

self

-

shocking

in

it

the

filmma.

's

that

reveals

only

pit

bottom.

of

absorp.

self

-

shocking

in

SAME(ours)

Grad-CAM

GNNExplainer

SubgraphX

GNN-LRP

PGExplainer

GStarX

it

the

filmma.

's

that

reveals

only

pit

bottom.

of

absorp.

self

-

shocking

in

it

the

filmma.

's

that

reveals

only

pit

bottom.

of

absorp.

self

-

shocking

in

it

the

filmma.

's

that

reveals

only

pit

bottom.

of

absorp.

self

-

shocking

in

it

the

filmma.

's

that

reveals

only

pit

bottom.

of

absorp.

self

-

shocking

in

G.3 Additional Quantitative Analysis

Here we report more quantitative analysis with alternative SOTA methods and datasets. We tested SAME on GAT and GIN, and compared SAME with other SOTA algorithms (GraphSVX [4] and OrphicX [6]). In table S8, We report the averaged fidelity over a sparsity of [0 . 5 , 0 . 6 , 0 . 7 , 0 . 8] . For the inference time plus fidelity, they are summarized in Table S9. SAME not only surpasses the benchmarks' performance by alternative GNN architectures, but it also exhibits superior e ffi ciency in elucidating explanations.

Note: The previous SOTA results on di ff erent datasets are marked with an underline.

Note: The previous SOTA results on di ff erent datasets are marked with an underline.

11

References

C. B. Browne, E. Powley, D. Whitehouse, S. M. Lucas, P. I. Cowling, P. Rohlfshagen, S. Tavener, D. Perez, S. Samothrakis, and S. Colton. A survey of monte carlo tree search methods. IEEE Transactions on Computational Intelligence and AI in Games , 4(1):1-43, 2012.

A. K. Debnath, R. L. Lopez de Compadre, G. Debnath, A. J. Shusterman, and C. Hansch. Structureactivity relationship of mutagenic aromatic and heteroaromatic nitro compounds. correlation with molecular orbital energies and hydrophobicity. Journal of Medicinal Chemistry , 34(2):786-797, 1991.

J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. ArXiv , abs/1810.04805, 2019.

A. Duval and F. D. Malliaros. Graphsvx: Shapley value explanations for graph neural networks. In Machine Learning and Knowledge Discovery in Databases. Research Track: European Conference, ECML PKDD 2021, Bilbao, Spain, September 13-17, 2021, Proceedings, Part II 21 , pages 302-318. Springer, 2021.

M. Gardner, J. Grus, M. Neumann, O. Tafjord, P. Dasigi, N. Liu, M. Peters, M. Schmitz, and L. Zettlemoyer. Allennlp: A deep semantic natural language processing platform. 2018.

W. Lin, H. Lan, H. Wang, and B. Li. Orphicx: A causality-inspired latent variable model for interpreting graph neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 13729-13738, 2022.

M. Liu, Y. Luo, L. Wang, Y. Xie, H. Yuan, S. Gui, H. Yu, Z. Xu, J. Zhang, Y. Liu, et al. Dig: A turnkey library for diving into graph deep learning research. The Journal of Machine Learning Research , 22(1):10873-10881, 2021.

D. Luo, W. Cheng, D. Xu, W. Yu, B. Zong, H. Chen, and X. Zhang. Parameterized explainer for graph neural network. Advances in Neural Information Processing Systems , 33:19620-19631, 2020.

P. E. Pope, S. Kolouri, M. Rostami, C. E. Martin, and H. Ho ff mann. Explainability methods for graph convolutional neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 10772-10781, 2019.

T. Schnake, O. Eberle, J. Lederer, S. Nakajima, K. T. Schütt, K.-R. Müller, and G. Montavon. Higher-order explanations of graph neural networks via relevant walks. IEEE Transactions on Pattern Analysis and Machine Intelligence , 44(11):7581-7596, 2021.

Z. Wu, B. Ramsundar, E. N. Feinberg, J. Gomes, C. Geniesse, A. S. Pappu, K. Leswing, and V. Pande. Moleculenet: a benchmark for molecular machine learning. Chemical Science , 9(2):513530, 2018.

Z. Ying, D. Bourgeois, J. You, M. Zitnik, and J. Leskovec. Gnnexplainer: Generating explanations for graph neural networks. Advances in Neural Information Processing Systems , 32, 2019.

H. Yuan, H. Yu, S. Gui, and S. Ji. Explainability in graph neural networks: A taxonomic survey. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2022.

H. Yuan, H. Yu, J. Wang, K. Li, and S. Ji. On explainability of graph neural networks via subgraph explorations. In International Conference on Machine Learning , pages 12241-12252. PMLR, 2021.

S. Zhang, Y. Liu, N. Shah, and Y. Sun. Gstarx: Explaining graph neural networks with structureaware cooperative games. In Advances in Neural Information Processing Systems , 2022.

12

View publication stats

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nA GNN-based Shapley value for GNN explanation is proposed, which is a novel Structure-Aware Shapley-based Multi-Explanation (SAME) technique for fairly considering the multi-level structure-aware feature interactions over an input graph It is computed by an expansion-based Monte Carlo tree search (MCTS) and its main advantage over existing methods is its theoretical foundation.\nExperiments on multiple datasets also highlight practical benefits.\n\nSTRENGTHS:\n- **Clarity:**\nTable 1 clearly points out the differences/additional features in comparison to related methods. The figures have been designed with care and are visually appealing. In particular, the appendix contains some beautiful figures. However, some arguments could be made more clear (see weaknesses). Some larger parts of the paper are not well understandable because information is missing (for details see points of minor critique).\n\n- **Originality:**\nTable 1 explains the novelty of SAME relative to competing methods. While all of the listed features are realised by some methods, SAME is able to offer their combination.\nThe authors claim that another distinguishing feature is a theoretical foundation. (However, other Shapley based measures are also benefit from a well established related theory.)\nThe proposed method SAME is quite similar to SubgraphX. Primarily, the pruning-based algorithm is replaced by an expansion-based algorithm. Thus, the main idea and game theoretic framework are not novel.\n\n- **Quality:**\nThe figures are well designed and the text is well structured. The experiments seem standard. The main algorithmic contributions are difficult to understand, however.\n\n\n- **Significance:**\nExperiments suggest a clear improvement over competing methods on multiple datasets.\nOnly some results on synthetic graphs in the appendix are not competitive.\n\nWEAKNESSES:\n- The biggest issue is that considerable parts of the paper are not well understandable. The algorithm and the contribution of its parts are not clear. Neither the game theoretic approach is explained. It would also be important to discuss the detailed differences between SAME and the similar approach SubgraphX.\n\n- The Shapley value is only computed approximately (as common).\n\n- The search space cannot be covered completely. Thus, the algorithm has therefore no guarantees to find a global optimum (as common). \n\n- The implications of the Theorem 1 need to be explained better. Currently, it does not seem to provide clear practical value, since the fact that one bound might be better than another one does not imply that the corresponding loss is also smaller..\n\n- Missing literature?\nGraphSVX: Shapley Value Explanations for Graph Neural Networks. ECML PKDD 2021.\n\n- **Reproducibility:** No code has been submitted.\n\n**Points of minor critique:**\n\n- The rules of the underlying multiplayer game are not explained.\n\n- Lines 110-121, which are supposed to explain the main contribution (the algorithm that explores the search space), are not really understandable. At least a correction of the grammar would be helpful. \n\n- Lines 130-134: It is impossible to follow the argument why the proposed algorithm requires fewer smoothness constraints on the function $f$, since the pruning algorithm is not introduced before.\nIt might help to move the discussion of related work to the beginning of the paper.\n\n- Figure 1 is not sufficient to explain the overall approach. The main components are not explained in the text.\n\nQUESTIONS:\nFor what kind of GNNs does the proposed explainer work? Could attention be considered?\n\nLIMITATIONS:\nThe limitations have been pointed out under weaknesses and were mostly discussed by the authors. I do not foresee a negative societal impact of this work.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a novel method for explaining GNNs called SAME. SAME is theoretically grounded and has some good properties over exisiting methods. SAME uses an expansion-based Monte Carlo tree search algorithm to approximate the optimal Shapley-based explanation, which is proven to be better than pruning-based methods like SubgraphX. SAME has two MCTS stages, first on nodes and then on connected component combinations. SAME is evaluated on six datasets from different sources with diverse categories, including molecular graphs, sentiment graphs, and synthetic BA graphs. The experiments show that SAME outperforms previous state-of-the-art methods and has reasonable inference time.\n\nSTRENGTHS:\n1. A theoretical assessment of the expansion-based vs. pruning-based explanation methods is great.\n2. The two-stage MCTS approach involving nodes and then components is novel to me and intuitively makes sense.\n3. The proposed SAME framework has good empirical performance.\n\nWEAKNESSES:\n1. The level of detail for different parts of the paper can be adjusted\n\nOne suggestion I have is to shrink section 2.1 by moving some content to the Appendix. Although being clear about what Shapley value is and what the good properties of Shapley value have is important, most of the target readers should be comfortable reading the paper without mentioning all the details explicitly in the main text. For example, properties 1 - 4 should easily follow as long as the set of players P_i is clearly defined. Also, \"k-hop Shapley\" is another fancy term for saying only nodes in the computation graph should be considered. This idea was introduced in SubgraphX, and the idea of computation graph was brought up even earlier in GNNExplainer. All of these being said, the core idea of Section 2.1 can be quickly made clear by defining the set of players and the computation graph as k-hop neighbors. However, a whole page listing out all four properties and formulas (2) and (3) can make the paper seem redundant and suspicious, as these are not original contributions of the paper.\n\nI would suggest spending more space discussing other important things. For example, at the end of Section 2, the authors mentioned that \"It is intuitive to conclude ....\". This kind of discussion can be made more detailed when there is more space.\n\n\n2. Required properties of explanation methods can be made more clear\n\nSeveral properties of GNN explanation methods are introduced in Table 1, and the major role of sections 2 and 3 is to show SAME is better than other methods in terms of these properties. However, I think some of these properties are not well-defined or not reasonably named, making them not the best ways to measure explanation methods.\n\na. Structure awareness. The authors claim that 4 methods satisfy this property and 3 do not. However, Section 2.1 discusses that the way SubgraphX and SAME satisfy the structure awareness is through the \"k-hop Shapley\" idea. I would not say \"k-hop\" is structure independent, but it only trivially uses the structure to construct the computation graph. The structure awareness refers to a different concept from the existing structure-aware idea proposed by GStarX, as the computation of HN naturally involves the structure. Thus, the property here can be misleading.\n\nb. Multiple explanations. By multiple explanations, the authors seem to mean whether multiple disconnected pieces are allowed to appear together in the explanation output. However, the name here only makes sense when these pieces have a \"or\" relationship instead of an \"and\" relationship. For example, when the ground truth explanation for a prediction is \"A and B\". It is not proper to say there are multiple explanations, one being \"A\" and the other being \"B\", because neither of them is a complete explanation. Also, this multiple-explanation idea seems more like a limitation of the SubgraphX method instead of a property in general. SubgraphX assumed connectedness so there will only be one piece, which can be a good assumption instead of a limitation in some cases, but other methods do not make this assumption. In general, I do not think it is a good idea to take an assumption only one method makes and claiming not having it is a required property. \n\nc. Node-wise/Substructure-wise/global importance. I don't feel this categorization makes full sense. If we just look at the definition, global importance implies substructure-wise importance, which further implies node-wise importance. This is because one explanation can be one connected component, which can be one node. How could one method have the global importance property but not the other two? Like GNNExplainer and PGExplainer in Table 1. Also, I believe that if any of these three importance properties is strictly satisfied, the computational complexity will be exponential. How could SAME satisfy them and be a polynomial algorithm?\n\nI hope the authors can better organize these properties so that they 1. do not have conflicts with existing concepts 2. are properly named 3. are more clearly defined and categorized.\n\nQUESTIONS:\nMy questions are very related to the weaknesses I mentioned above.\n\n1. Why is it intuitive to conclude the expansion-based method is better than the pruning-based method? Can the authors explain more?\n\n2. If I misunderstood the properties in Table 1 in my weaknesses comment, please point it out.\n\n3. I have some doubts about the time complexity analysis. Can the authors be more detailed about why it is O((M1 + M2)n^2)? Also, other methods, like SubgraphX, do not necessarily have time complexity O(2^n), as SubgraphX considers k-hop neighbors, uses MC approximation, and has a budget for the subgraph size.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 1, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces SAME, a novel method for post-hoc explanation of Graph Neural Networks (GNNs). SAME leverages an expansion-based Monte Carlo tree search to explore structure-aware connected substructures and provides explanations that are as explainable as the theoretically optimal Shapley value. Experimental results show that SAME outperforms previous state-of-the-art methods, improving fidelity performance by 7.01% to 42.3% across various benchmarks. The SAME method contributes to the field of GNN explanation by offering theoretical foundations and improved performance.\n\nSTRENGTHS:\n- Theoretical foundation: The paper introduces the SAME method, which provides a theoretical foundation for post-hoc explanation techniques for Graph Neural Networks (GNNs). This adds credibility and rigor to the proposed approach. \n\n- Structure-aware explanations: SAME addresses the challenge of structure-aware feature interactions in GNN explanations. By leveraging an expansion-based Monte Carlo tree search, SAME explores multi-grained connected substructures, resulting in more informative explanations. \n\n- Improved performance: Experimental results demonstrate that SAME outperforms previous state-of-the-art methods in fidelity performance across various benchmarks. The improvements range from 7.01% to 42.3%, indicating the effectiveness of the proposed method.\n\nWEAKNESSES:\n- Limited discussion of limitations: The paper lacks a comprehensive discussion of the limitations or potential drawbacks of the SAME method. Addressing the potential shortcomings would strengthen the paper and guide future research directions.\n\n- Limited comparison with alternative methods: While SAME is shown to outperform previous state-of-the-art methods, the paper does not provide a thorough comparison with alternative approaches. Including such a comparison would enhance the understanding of how SAME stands against other techniques.\n\n- Generalizability to different GNN architectures: The paper does not explicitly discuss the generalizability of the SAME method to different GNN architectures. Considering and discussing the potential challenges or adaptations needed for different architectures would enhance the applicability of the proposed method.\n\n- Theoretical complexity and computational efficiency: Although SAME claims to provide explanations within polynomial time, the paper does not provide a detailed analysis of the theoretical complexity or computational efficiency. Further investigation into the computational aspects would be valuable for understanding the scalability of the method.\n\nQUESTIONS:\n- Theoretical Analysis: Could you provide more details about the theoretical foundation of the SAME method? Specifically, how does the expansion-based Monte Carlo tree search contribute to the structure-aware explanations, and how does SAME ensure the explanations are as explainable as the theoretically optimal Shapley value?\n\n- Comparative Analysis: In the paper, SAME is shown to outperform previous state-of-the-art methods in terms of fidelity performance. However, could you provide a more thorough comparison with alternative explanation techniques? How does SAME compare to other state-of-the-art methods in terms of interpretability, computational efficiency, and generalizability?\n\n- Limitations and Future Directions: While the results are promising, it would be helpful to have a more comprehensive discussion on the limitations of the SAME method. What are some potential drawbacks or challenges associated with SAME? Additionally, what future research directions do you envision to address these limitations and further improve the method?\n\n- Applicability to Different GNN Architectures: The paper focuses on explaining Graph Neural Networks (GNNs). Could you provide insights into the generalizability of SAME to different GNN architectures? Are there any specific considerations or adaptations required for applying SAME to other GNN models, such as graph attention networks or graph convolutional networks?\n\n- Theoretical Complexity and Scalability: The paper claims that SAME provides explanations within polynomial time. Could you provide more details on the computational complexity of the SAME method? How does the method scale with increasing graph size or complexity? Are there any limitations or considerations regarding the scalability of SAME to larger graphs?\n\n- Insights on Parameters and Hyperparameters: The paper mentions the use of Monte Carlo tree search and distinct single substructures in the SAME method. Could you elaborate on how the parameters and hyperparameters of these components affect the quality of the explanations? Are there any guidelines or insights you can provide to help researchers fine-tune these parameters effectively?\n\nLIMITATIONS:\n- Limited Discussion on Practical Applicability: The paper primarily focuses on the fidelity performance of the SAME method without discussing its practical applicability or real-world constraints. It would be beneficial to address any potential limitations or challenges in applying SAME to real-world scenarios, such as scalability, interpretability, or sensitivity to noise or perturbations in the input graph.\n\n- Lack of User Evaluation: The paper does not include user studies or evaluations to assess the usefulness or understandability of the explanations generated by SAME. Incorporating user feedback or conducting user studies would provide insights into the effectiveness of SAME from a human-centered perspective.\n\n- Absence of Ablation Studies: The paper does not include ablation studies to analyze the contribution of individual components or design choices in the SAME method. Conducting such studies would help isolate and evaluate the impact of different aspects of the method on the overall performance.\n\n- Limited Dataset Coverage: The paper evaluates SAME on a specific set of benchmarks, both real-world and synthetic. However, it would be valuable to assess its performance on a more diverse range of datasets, including those with different graph sizes, properties, and domains.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

comments: SUMMARY:
The paper proposes a novel method for explaining GNNs called SAME. SAME addresses the challenges of structure-aware feature interactions in GNN explanation by using an expansion-based Monte Carlo tree search. The authors evaluate SAME on a variety of benchmarks and show that it outperforms previous state-of-the-art methods. The paper makes contributions to the field of GNN explanation.

STRENGTHS:
- SAME addresses the challenges of structure-aware feature interactions in GNN explanation by using an expansion-based Monte Carlo tree search, which is novel
- The authors evaluate SAME on a variety of benchmarks, and the experiments show that SAME outperforms previous state-of-the-art methods on these benchmarks.
- The theoretical aspects of SAME are well-developed and provide a good foundation for the proposed method.

WEAKNESSES:
- The authors could provide more information on the limitations of SAME, particularly in terms of the types of graphs and datasets for which it may not perform as well.
- The authors could provide more insight into the computational complexity of SAME and how it scales to larger graphs and datasets.
- The technical content is not sufficient, for example, it does not fully use the 9-page limit.

QUESTIONS:
How scalable is SAME? Shapley-based explainability methods are generally more expensive than gradient-based methods.

LIMITATIONS:
Yes.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-10