Title: GOAt: Explaining Graph Neural Networks via Graph Output Attribution
Venue: NeurIPS 2023
Accepted: no
Keywords: Graph Neural Networks

explainability

interpretability

local-level explanation

instance-level explanation
Abstract: Understanding the decision-making process of Graph Neural Networks (GNNs) is crucial to their interpretability. Present methods for explaining GNNs typically rely on training auxiliary models, and may struggle with issues such as overfitting to noise, insufficient discriminability, and inconsistent explanations across data samples of the same class. This paper introduces Graph Output Attribution (GOAt), a novel method to attribute graph outputs to input graph features, creating GNN explanations that are faithful, discriminative, as well as stable across similar samples. By expanding the GNN as a sum of scalar products involving node features, edge features and activation patterns, we propose an efficient analytical method to compute contribution of each node or edge feature to each scalar product and aggregate the contributions from all scalar products in the expansion form to derive the importance of each node and edge. Through extensive experiments on synthetic and real data, we show that our method has consistently outperformed various state-of-the-art GNN explainers in terms of fidelity, discriminability, and stability.

1

Abstract

Understanding the decision-making process of Graph Neural Networks (GNNs) is crucial to their interpretability. Present methods for explaining GNNs typically rely on training auxiliary models, and may struggle with issues such as overfitting to noise, insufficient discriminability, and inconsistent explanations across data samples of the same class. This paper introduces Graph Output Attribution (GOAt), a novel method to attribute graph outputs to input graph features, creating GNN explanations that are faithful, discriminative, as well as stable across similar samples. By expanding the GNN as a sum of scalar products involving node features, edge features and activation patterns, we propose an efficient analytical method to compute contribution of each node or edge feature to each scalar product and aggregate the contributions from all scalar products in the expansion form to derive the importance of each node and edge. Through extensive experiments on synthetic and real data, we show that our method has consistently outperformed various state-of-the-art GNN explainers in terms of fidelity, discriminability, and stability.

1 Introduction

Graph Neural Networks (GNNs) have demonstrated notable success in learning representations from graph-structured data in various fields [14, 9, 30]. However, their black-box nature has driven the need for explainability, especially in sectors where transparency and accountability are essential, such as finance [28], healthcare [1], and security [18]. The ability to interpret GNNs can provide insights into the mechanisms underlying deep models and help establish trustworthy predictions.

Existing attempts to explain GNNs usually focus on local-level or global-level explainability. Locallevel explainers [31, 17, 21, 27, 10, 15, 23, 3] typically train a secondary model to identify the critical graph structures that best explain the behavior of a pretrained GNN for specific input instances. These methods are always optimized for ground-truth explanations or fidelity metrics, yet may not be able to generate consistent explanations for similar graph samples or produce accurate and human-intelligible explanations for class discrimination. Global-level explainers [2, 11] perform prototype learning or random walk on the explanation instances to extract the global explanations over a multitude of graph samples. However, their effectiveness rely heavily on the quality of local-level explanations.

In this paper, we introduce a computationally efficient local-level GNN explanation technique called G raph O utput At tribution ( GOAt ) to overcome the limitations of existing methods. Unlike methods that rely on back-propagation with gradients [20, 4, 22, 8] and those relyinig on hyper-parameters or training complex black-box models [17, 15, 23, 3], our approach enables attribution of GNN output to input features, leveraging the repetitive sum-product structure in the forward pass of a GNN.

Given that the matrix multiplication in each GNN layer adheres to linearity properties and the activation functions operate element-wise, a GNN can be represented in an expansion form as a

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

sum of scalar product terms, involving input graph features, model parameters, as well as activation patterns that indicate the activation levels of the scalar products. Based on the notion that all scalar variables X i in a scalar product term g = cX 1 X 2 . . . X N contribute equally to g , where c is a constant, we can attribute each product term to its corresponding factors and thus to input features, obtaining the importance of each node or edge feature in the input graph to GNN outputs. We present case studies that demonstrate the effectiveness of our analytical explanation method GOAt on typical GNN variants, including GCN, GraphSAGE, and GIN.

Besides the fidelity metric, which is commonly used to assess the faithfulness of GNN explanations, we introduce two new metrics to evaluate the discriminability and stability of the explanation, which are under-investigated by prior literature. Discriminability refers to the ability of explanations to distinguish between classes, which is assessed by the difference between the mean explanation embeddings of different classes, while stability refers to the ability to generate consistent explanations across similar data instances, which is measured by the percentage of data samples covered by topk explanations. Through comprehensive experiments based on on both synthetic and real-world datasets along with qualitative analysis, we show the outstanding performance of our proposed method, GOAt , in providing highly faithful, discriminative, and stable explanations for GNNs, as compared to a range of state-of-the-art methods.

2 Problem Formulation

Graph Neural Networks Let G = ( V , E ) be a graph, where V = { v 1 , v 2 , . . . , v N } denotes the set of nodes and E ⊆ V ×V denotes the set of edges. The node feature matrix of the graph is represented by X ∈ R N × d , and the adjacency matrix is represented by A ∈ { 0 , 1 } N × N such that A ij = 1 if there exists an edge between nodes v i and v j . The task of a GNN is to learn a function f ( G ) , which maps the input graph G to a target output, such as node labels, graph labels, or edge labels. Formally speaking, for a given GNN, the hidden state h ( l ) i of node v i at its layer l can be represented as:

where N i represents the set of neighbors of node v i in the graph. COMEBINE ( l ) ( · ) is a COMBINE function such as concatenation [9], while AGGREGATE ( l ) ( · ) are AGGREGATE functions with aggregators such as ADD . We focus on GNNs that adopt the non-linear activation function ReLU in COMBINE or AGGREGATE functions.

Local-level GNN Explainability Our goal is to generate a faithful explanation for a graph instance G = ( V , E ) by identifying a subset of edges S ⊆ E , given a GNN f ( · ) pretrained on a set of graphs G . The term faithful refers to the explanation's ability to perform well in not only fidelity [33] and robustness [3] metrics, but also stability in identifying consistent patterns. We highlight edges instead of nodes as suggested by [7] that edges have more fine-grained information than nodes while giving human-understandable explanations like subgraphs.

3 Method

This section begins by presenting our fundamental definition of equal contribution in a product term and its application in an example of a toy graph neural network. Then, we mathematically present GOAt method for explaining typical GNNs, followed by a case study on GCN [14]. Additional case studies of applying GOAt to GraphSAGE [9] and GIN [30] are included in the Appendix.

3.1 Definitions

Consider a function g ( X 1 , . . . , X M ) of M variables X = { X 1 , . . . , X M } . If we let a pair of variables ( X i , X j ) be set to ( X i , X j ) = ( x i , x j ) , we will obtain a manifold g X i = x i ,X j = x j ( X \{ X i , X j } ) , which represents g ( · ) when all variables excluding X i and X j can vary. Consider a base manifold g X i = x ′ i ,X j = x ′ j ( X \{ X i , X j } ) . If we can obtain two identical manifolds by setting ( X i , X j ) = ( x i , x ′ j ) and ( X i , X j ) = ( x ′ i , x j ) , it will indicate that changing X i = x ′ i to X i = x i is equivalent to changing X j = x ′ j to X j = x j with respect to the base manifold at ( X i , X j ) = ( x ′ i , x ′ j ) .

For example, a function

g

(

x, y, z

) = 2

xy

+

x

2

z

has three variables x, y, z

, we consider taking

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

g x = -1 ,y = -1 ( z ) = z +2 as the base manifold. Since g x =1 ,y = -1 ( z ) = g x = -1 ,y =1 ( z ) = z -2 , we say that changing x = -1 to x = 1 is equivalent to changing y = -1 to y = 1 with respect to the base manifold at ( x, y ) = ( -1 , -1) .

Definition 1 ( Equal Contribution ) . Given a function g ( X ) where X = { X 1 , . . . , X M } represents M variables, we say that variables X i and X j have equal contribution to function g at ( x i , x j ) with respect to the base manifold at ( x ′ i , x ′ j ) if and only if setting X i = x i , X j = x ′ j and setting X i = x ′ i , X j = x j yield the same manifold, i.e.,

for any values of X excluding X i and X j .

Lemma 2 ( Equal Contribution in a product ) . Given a function g ( X ) defined as g ( X ) = b ∏ M k =1 X k , where b is a constant, and X = { X 1 , . . . , X M } represents M uncorrelated variables. Each variable X k is either 0 or x k , depending on the absence or presence of a certain feature. Then, all the variables in X contribute equally to g ( X ) at [ x 1 , . . . , x M ] with respect to [0 , . . . , 0] .

Proofs of all Lemmas and Theorems can be found in the Appendix. Since all the binary variables have equal contribution, we define the contribution of each variable X k to g ( X ) = b ∏ M k =1 X k for all k = 1 , . . . , M , as

For example, let f ( A,X ) = AXW be a simple 2-node GNN for node classification, where A,X,W are 2 × 2 matrices that denote adjacency matrix, node feature matrix, and weight matrix, respectively. Then, we can represent each entry in the resulting 2 × 2 matrix f ( A,X ) as an expansion form:

where f i,j ( A,X ) represents the prediction of the i -th node for the j -th class. In a pretrained GNN, parameter W is fixed. Thus, only A i,k and X k,l contribute to the value of each scalar product A i,k X k,l W l,j . As A i,k is usually independent of X k,l under proper data cleaning, we can calculate the contributions of A i,k and X k,l to the scalar product A i,k X k,l W l,j by I A i,k = I X k,l = 1 2 A i,k X k,l W l,j based on Lemma 2 and Equation (2). By similar computations for all the scalar products in the expansion form of f ( · ) , we can obtain the contribution of all the input features to each entry of the output matrix.

3.2 Explaining Graph Neural Networks via Attribution

A typical GNN [14, 9, 30] for node or graph classification tasks usually comprises 2-6 messagepassing layers for learning node or graph representations, followed by several fully connected layers that serve as the classifier. With the hidden state h ( l ) i of node v i at the l -th message-passing layer defined as Equation (1), we generally have the hidden state H ( l ) of a data sample as:

where A is the adjacency matrix, ϵ ( l ) refers to the self-loop added to the graph if fixed to 1 , otherwise it is a learnable parameter, σ ( · ) is the element wise activation function, Φ ( l ) and Ψ ( l ) can be Multilayer Perceptrons (MLP) or linear mappings, λ ∈ { 0 , 1 } determines whether a concatenation is required. If the COMBINE step of a GNN requires a concatenation, we have λ = 1 and ϵ ( l ) = 1 ; if the COMBINE step requires a weighted sum, we have ϵ ( l ) set trainable and λ = 0 . Alternatively, Equation (4) can be expanded to:

where K,Q refer to the number of MLP layers in Φ ( l ) ( · ) and Ψ ( l ) ( · ) , and W Φ ( l ) k and W Ψ ( l ) q are the trainable parameters in Φ ( l ) k and Ψ ( l ) q .

Given a certain data sample and a pretrained GNN, for an element-wise activation function we can 121 define the activation pattern as the ratio between the output and input of the activation function: 122

Definition 3 ( Activation Pattern ) . Denote H ( l ) ′ and H ( l ) as the hidden representations before and 123 after passing through the element-wise activation function at the l -th layer, we define activation 124 pattern P ( l ) for a given data sample as 125

̸

where P ( l ) i,j is the element-wise activation pattern for the j -th feature of i -th node at layer l .

126

Hence, the hidden state H ( l ) at the l -th layer for a given sample can be written as 127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

where ⊙ represents element-wise multiplication. Thus, similar to Equation (3), we can expand the expression of each output entry in a GNN f ( A,X ) into a sum of scalar products, where each scalar product is the multiplication of corresponding entries in A , X , W , and P in all layers. Then each scalar product can be written as

where C is a constant, c k refers to the k -th layer of the classifier, ( α l 0 , α l 1 ) , ( β l 0 , β l 1 ) , ( γ l 0 , γ l 1 ) are ( row, column ) indices of the corresponding matrices at layer l . In a classifier with M MLP layers, only ( M -1) layers adopt activation functions. Therefore, we do not have P (c M ) α L 0 ,γ M 1 in Equation (7). For scalar products without factors of A , all A 's are considered as constants equal to 1 in Equation (7). Since the GNN model parameters are pretrained and fixed, we only consider A , X , and all the P terms as the variables in each product term.

Lemma 4 ( Equal Contribution variables in the GNN expansion form's scalar product ) . For a scalar product term z in the expansion form of a pretrained GNN f ( · ) , when the number of nodes N is large, all variables in z have equal contributions to the scalar product z .

Hence, by Equation (2), we can calculate the contribution I ν ( z ) of a variable ν (i.e., an entry in A , X and P matrices) to each scalar product z (given by Equation (7)) by:

where function V ( · ) represents the set of variables in its input, and | V ( z ) | denotes the number of unique variables in z , e.g., V ( x 2 y ) = { x, y } , and | V ( x 2 y ) | = 2 .

Similar to Section 3.1, an entry f m,n ( A,X ) of the output matrix f ( A,X ) can be expressed by the sum of all the related scalar products as

where summation is over all possible ( α l 0 , α l 1 ) , ( β l 0 , β l 1 ) , ( γ l 0 , γ l 1 ) , for message passing layer l = 1 , . . . , L or classifier layer l = 1 , . . . , M , as well as all i, j indices for X . By summing up the contribution of each variable ν among the entries in the A , X and P 's in all the scalar products in the expansion form of f m,n ( · ) , we can obtain the contribution of ν to f m,n ( · ) as:

Theorem 5 ( Contribution of variables in the expansion form of a pretrained GNN ) . Given Equations (8) and (10) , for each variable ν (i.e., an entry in A , X and P matrices), when the number of nodes N is large, we can approximate I ν ( f m,n ( · )) by:

where O ( ν, z ) denotes the number of occurrences of ν among the variables of z . 154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

Recall that | V ( z ) | stand for the number of unique variables in z . Hence the total number of occurrences of all the variables ∑ ρ in z O ( ρ, z ) is not necessarily equal to | V ( z ) | . For example, if all of { A (1) α 10 ,α 11 , . . . , A ( L ) α L 0 ,α L 1 } in z are unique entries in A , then they can be considered as L independent variables in the function representing z . If two of these occurrences of variables refer to the same entry in A , then there are only ( L -1) unique variables related to A .

Although Theorem 5 gives the contribution of each entry in A , X and P 's, we need to further attribute P 's to A and X and allocate the contribution of each activation pattern P ( r ) a,b to node features X and edges A by considering all non-zero features in X a of node v a and the edges within m hops of node v a , as these inputs may contribute to the activation pattern P ( r ) a,b . However, determining the exact contribution of each feature that contributes to P ( r ) a,b is not straightforward due to non-linear activation. We approximately attribute all relevant features equally to P ( r ) a,b . That is, each input feature ν that has nonzero contribution to P ( r ) a,b will share an equal contribution of I P ( r ) a,b ( f m,n ( · )) / | V ( P ( r ) a,b ) | , where | V ( P ( r ) a,b ) | denotes the number of distinct node and edge features in X and A contributing to P ( r ) a,b , which is exactly all non-zero features in X a of node v a and the adjacency matrix entries within r hops of node v a . Finally, based on Equation (11), we can obtain the contribution of an input feature ν in X , A of a graph instance to the ( m,n ) -th entry of the GNN output f ( · ) as:

where ν is an entry in the adjacency matrix A or the input feature matrix X , P ( r ) a,b denotes an entry in all the activation patterns. Thus, we have attributed f ( · ) to each input feature of a given data instance.

Our approach meets the completeness axiom, which is a critical requirement in attribution methods [25, 24, 6]. This axiom guarantees that the attribution scores for input features add up to the difference in the GNN's output with and without those features. Passing this sanity check implies that our approach provides a more comprehensive account of feature importance than existing methods that only rank the top features [3, 17, 20, 31, 27, 23].

3.3 Case Study: Explaining Graph Convolutional Network (GCN)

GCNs use a simple sum in the combination step, and the adjacency matrix is normalized with the diagonal node degree matrix D . Hence, the hidden state of a GCN's l -th message-passing layer is:

where V = D -1 2 ( A + I ) D -1 2 represents the normalized adjacency matrix with self-loops added. Suppose a GCN has three convolution layers and a 2-layer MLP as the classifier, then its expansion form without the activation functions ReLU( · ) will be:

̸

where V ( l ) = V is the normalized adjacency matrix in the l -th layer's calculation. In the actual expansion form with the activation patterns, the corresponding P ( m ) 's are multiplied whenever there is a W ( m ) or B ( m ) in a scalar product, excluding the last layer W ( c 2 ) and B ( c 2 ) . For example, in the scalar products corresponding to V (3) V (2) V (1) XW (1) W (2) W (3) W ( c 1 ) W ( c 2 ) , there are eight variables consisting of four P 's, one X , and three V 's. By Equation (11), an adjacency entry V i,j itself will contribute 1 8 of p ( V (3) V (2) : i V (1) i,j X j : W (1) W (2) W (3) W ( c 1 ) W ( c 2 ) ) + p ( V (3) : i V (2) i,j V (1) j : XW (1) W (2) W (3) W ( c 1 ) W ( c 2 ) )+ p ( V (3) i,j V (2) j : V (1) XW (1) W (2) W (3) W ( c 1 ) W ( c 2 ) ) , where p ( · ) denotes the results with the element-wise multiplication of the corresponding activation patterns applied at the appropriate layers. After we obtain the contribution of V i,j itself on all the scalar products, we can follow Equation (12) to allocate the contribution of activation patterns to V i,j .

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

75

80

85

90

95

Average sparsity (%)

0.2

0.4

0.6

Fidelity

(a) BA-2Motifs

80

85

90

95

Average sparsity (%)

0.0

0.2

0.4

0.6

0.8

Fidelity

(b) Mutagenicity

80

85

90

95

Average sparsity (%)

0.0

0.2

0.4

0.6

Fidelity

(c) NCI1

GNNExplainer

PGExplainer

PGM-Explainer

SubgraphX

CF-GNNExplainer

RCExplainer

GOAt (Ours)

With Equation (14), we find that when both V and X are set to zeros, f ( · ) remains non-zero and is:

where B ( c 2 ) is the global bias, and the other terms have non-zero entries at the activated neurons. In other words, certain GNN neurons in the 3 -rd and c 1 -th layers may already be activated prior to any input feature being passed to the network. When we do feed input features, some of these neurons may remain activated or be toggled off. With Equation (12), we consider taking all 0 's of the X entries, V entries and P entries as the base manifold. Now, given that some of the P entries in GCN are non-zero when all X and V set to zeros, as present in Equation (15), we will need to subtract the contribution of each features on these P from the contribution values calculated by Equation (12). We let P ′ represent the activation patterns of f ( 0 , 0 ) , then the calibrated contribution ̂ I cali V i,j ( f ( · )) of V i,j is given by:

In graph classification tasks, a pooling layer such as mean-pooling is added after the convolution layers to obtain the graph representation. To determine the contribution of each input feature, we can simply apply the same pooling operation as used in the pre-trained GCN.

As we mentioned in Section 2, we aim to obtain the explanations by the critical edges in this paper, since edges have more fine-grained information than nodes. Therefore, we treat the edges as variables, while considering the node features X as constants similar to parameters W or B . This setup naturally aggregates the contribution of node features onto edges. By leveraging edge attributions, we are able to effectively highlight motifs within the graph structure.

4 Experiments

We conduct a series of experiments on the fidelity, discriminability and stability metrics to compare our method with the state-of-the-art methods including GNNExplainer [31], PGExplainer [17], PGMExplainer [27], SubgraphX [33], CF-GNNExplainer [16], RCExplainer [3], RG-Explainer [23] and DEGREE [8]. As outlined in Section 2, we highlight edges as explanations as suggested by [7]. For baselines that identify nodes or subgraphs as explanations, we adopt the evaluation setup from [3].

We evaluate the performance of explanations on three variants of GNNs, which are GCN [14], GraphSAGE [9] and GIN [30]. The experiments are conducted on both the graph classification task and the node classification task. For graph classification task, we evaluate on a synthetic dataset, BA-2motifs [17], and two real-world datasets, Mutagenicity [13] and NCI1 [19]. For node classification task, we evaluate on three synthetic datasets [17], which are BA-shapes, BA-Community and Tree-grid. As space is limited, we will only present the key results here. Fidelity results on GIN and GraphSAGE, as well as the results of node classification tasks can be found in the Appendix. Discussions on the controversial metrics such as accuracy are also moved to the Appendix.

4.1 Fidelity

Fidelity [20, 32, 29, 3] is the decrease of predicted probability between original and new predictions after removing important edges, which are used to evaluate the faithfulness of explanations. It is

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

70

75

80

85

90

Average sparsity (%)

10 -3

10 -2

10 -1

10 0

10 1

Disc)i%inability

(a)BA-2Motifs

70

75

80

85

90

Average sparsity (%)

10 -1

10 0

Disc)i%inability

(b)Mutagenicity

70

75

80

85

90

A-e)age s(a)sity (%)

10 -2

10 -1

10 0

Disc)i%inability

(c)NCI1

GNNEx(laine)

PGEx(laine)

PGM-Explainer

RG-Explainer

SubgraphX

RCExplainer

DEGREE

GOAt (Ours)

Original

Class 0

Class 1

(a)GNNExplainer

(b)PGExplainer

(c)PGM-Explainer

(d)RG-Explainer

(e)SubgraphX

(f)RCExplainer

(g)DEGREE

(h)GOAt (Ours)

(i) Original

defined as fidelity ( S, G ) = f y ( G ) -f y ( G \ S ) . As pointed out by [32], the fidelity may be sensitive to sparsity of explanations. The sparsity of an explanation S ⊆ E for a graph G = {V , E} is given by sparsity ( S, G ) = 1 -| S | |E| . It indicates the percentage of edges that remain in G after the removal of edges in S . Higher sparsity means fewer edges are identified as critical, which may have a smaller impact on the prediction probability. Therefore, we compare fidelity performance under similar levels of average sparsity, as in [33, 29, 3]. Figure 1 displays the fidelity results, with the baseline results sourced from [3]. Our proposed approach, GOAt , consistently outperforms the baselines in terms of fidelity across all sparsity levels, validating its superior performance in generating accurate and reliable faithful explanations. Among the other methods, RCExplainer exhibits the highest fidelity, as it is specifically designed for fidelity optimization. Notably, unlike the other methods that require training and hyperparameter tuning, GOAt offers the advantage of being a training-free approach, thereby avoiding any errors across different runs.

4.2 Discriminability

Discriminability, also known as discrimination ability [5, 12], refers to the ability of the explanations to distinguish between the classes. We define the discriminability between two classes c 1 and c 2 as the L2 norm of the difference between the mean values of explanation embeddings of the two classes. The embeddings used for explanations are taken prior to the last-layer classifier, with node embeddings employed for node classification tasks and graph embeddings utilized for graph classification tasks. In this procedure, only the explanation subgraph S is fed into the GNN instead of G .

We show the discriminability across various sparsity levels on GCN, as illustrated in Figure 2. Due to 248 the significant performance gap between the baselines and GOAt , a logarithmic scale is employed. 249 Our approach consistently outperforms the baselines in terms of discriminability across all sparsity 250 levels, demonstrating its superior ability to generate accurate and reliable class-specific explanations. 251 Notably, at sparsity = 0 . 7 , GOAt achieves higher discriminability than the original graphs on the 252 BA-2Motifs and NCI1 datasets. This indicates that GOAt effectively reduces noise unrelated to 253 the investigated class while producing informative class explanations. Additionally, we observe a 254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

296

1

3

5

7

9

Top k motifs (k)

0

20

40

60

80

100

Coverage (%)

(a)BA-2Motifs

5

105

205

305

405

505

Top k motifs (k)

0

20

40

60

Coverage (%)

(b)Mutagenicity

5

105

205

305

405

505

Top k motifs (k)

0

20

40

60

Coverage (%)

(c)NCI1

GNNExplainer

PGExplainer

PGM-Explainer

RG-Explainer

SubgraphX

RCExplainer

DEGREE

GOAt (Ours)

substantial decrease in discriminability between sparsity levels of 0.75 and 0.8 on BA-2Motifs. This implies that a minimum of approximately 25% of the edges is necessary to distinguish between the classes, which is in line with our expectation, given that a "house" motif, consisting of 6 edges, usually represents 24% of the total edges (on average, the total number of edges in BA-2Motifs is 25).

Furthermore, we present scatter plots to visualize the explanation embeddings generated by various GNN explainers. Figure 3 showcases the explanation embeddings obtained from different GNN explaining methods on the BA-2Motifs dataset, with sparsity = 0 . 7 . More scatter plots on Mutagenicity and NCI1 and can be found in the Appendix. The explanations generated by GNNExplainer fail to exhibit class discrimination, as all the data points are clustered together without any distinct separation. While some of the Class 1 explanations produced by PGExplainer, PGM-Explainer, RG-Explainer, RCExplainer, and DEGREE are noticeably separate from the Class 0 explanations, the majority of the data points remain closely clustered together. As for SubgraphX, most of its Class 1 explanations are isolated from the Class 0 explanations, but there is a discernible overlap between the Class 1 and Class 0 data points. In contrast, our method, GOAt , generates explanations that clearly and effectively distinguish between Class 0 and Class 1, with no overlapping points and a substantial separation distance, highlighting the strong discriminability of our approach.

4.3 Stability of extracting motifs

As we will later show in Section 4.4, it is often observed that datasets contain specific class motifs. For instance, in the BA-2Motifs dataset, the Class 1 motif exhibits a "house" structure. To ensure the stability of GNN explainers in capturing the class motifs across diverse data samples, we aim for the explanation motifs to exhibit relative consistency for data samples with similar properties, rather than exhibiting significant variations. To quantify this characteristic, we introduce the stability metric, which measures the coverage of the topk explanations across the dataset. An ideal explainer should generate explanations that cover a larger number of data samples using fewer motifs. This characteristic is also highly desirable in global-level explainers, such as [2, 11]. We illustrate the stability of the unbiased class as the percentage converge of the topk explanations produced on GCN with sparsity = 0 . 7 in Figure 4. Our approach surpasses the baselines by a considerable margin in terms of the stability of producing explanations. Specifically, GOAt is capable of providing explanations for all the Class 1 data samples using only three explanations. This explains why there are only three Class 1 scatters visible in Figure 3.

4.4 Qualitative analysis

We present the qualitative results of our approach in Table 1, where we compare it with state-of-the-art baselines such as PGExplainer, SubgraphX, and RCExplainer. The pretrained GNN achieves a 100% accuracy on the BA-2Motifs dataset. As long as it successfully identifies one class, the remaining data samples naturally belong to the other class, leading to a perfect accuracy rate. Based on the explanations from GOAt , we have observed that the GNN effectively recognizes the "house" motif that is associated with Class 1. In contrast, other approaches face difficulties in consistently capturing this motif. The Class 0 motifs in the Mutagenicity dataset generated by GOAt represent multiple connected carbon rings. This indicates that the presence of more carbon rings in a molecule increases its likelihood of being mutagenic (Class 0), while the presence of more "C-H" or "O-H" bonds in a molecule increases its likelihood of being non-mutagenic (Class 1). Similarly, in the NCI1 dataset, GOAt discovers that the GNN considers a higher number of carbon rings as evidence of chemical

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

330

BA-2Motifs

Mutagenicity

NCI1

Class0

Class1

Class0

Class1

Class0

Class1

PGExplainer

C

N

C

C

C

C

C

C

C

C

C

C

Cl

C

C

C

C

C

C

×2

×2

×N

×N

4.8%

1.8%

1.2%

1.3%

0.1%

0.5%

SubgraphX

Br

C

H

H

C

H

C

N

0.4%

12.8%

0.2%

0.2%

0.2%

0.1%

RCExplainer

C

C

C

C

C

C

C

×N

H

C

H

C

Br

Br

×N

×N

×N

6.4%

6.2%

0.4%

0.5%

0.05%

0.1%

GOAt

×N

×N

C

C

C

×N

C

C

C

C

C

×N

H

C

×N

H

O

×N

×N

3.8%

3.4%

93.4%

4%

3.5%

2.2%

2.2%

1.2%

3.5%

1.2%

4.3%

4.0%

compounds being active against non-small cell lung cancer. Other approaches, on the other hand, fail to provide clear and human-understandable explanations.

5 Related Work

Local-level Graph Neural Network (GNN) explanation approaches have been developed to shed light on the decision-making process of GNN models at the individual data instance level. Most of them, such as GNNExplainer [31], PGExplainer [17], PGM-Explainer [27], GraphLime [10], RGExplainer [23], CF-GNNExplainer [16], RCExplainer [3], CF 2 [26], RelEx [34] and Gem [15], train a secondary model to identify crucial nodes, edges, or subgraphs that explain the behavior of pretrained GNNs for specific input samples. However, the quality of the explanations produced by these methods is highly dependent on hyperparameter choices. Moreover, these explainers' black-box nature raises doubts about their ability to provide comprehensive explanations for GNN models. Approaches like SA [4], Grad-CAM [20], GNN-LRP [22], and DEGREE [8], which rely on gradient back-propagation, encounter the saturation problem [24]. As a result, these methods may generate explanations that are less faithful. SubgraphX [33] combines perturbation-based techniques with pruning using Shapley values. While it can generate some high-quality subgraph explanations, its computational cost is significantly high due to the reliance on the MCTS (Monte Carlo Tree Search). Additionally, as demonstrated in our experiments in Section 4, existing approaches exhibit inconsistencies on similar data samples and poor discriminability. This reinforces the need for our proposed method GOAt , which outperforms state-of-the-art baselines on fidelity, discriminability and stability metrics. Our work also relates to global-level explainability approaches. GLGExplainer [2] leverages prototype learning and builds upon PGExplainer to obtain global explanations. GCFExplainer [11] generates global counterfactual explanations by employing random walks on an edit map of graphs, utilizing local explanations from RCExplainer and CF 2 . Both GLGExplainer and GCFExplainer heavily rely on local explanations. Integrating local explainers that produce higher-quality local explanations, such as GOAt , has the potential to enhance the performance of these global-level explainers.

6 Conclusion

We propose GOAt , a local-level GNN explainer that overcomes the limitations of existing GNN explainers, in terms of insufficient discriminability, inconsistency on same-class data samples, and overfitting to noise. We analytically expand GNN outputs for each class into a sum of scalar products and attribute each scalar product to each input feature. Although GOAt shares similar limitations with some decomposition methods of requiring expert knowledge to design corresponding explaining processes for various GNNs, our extensive experiments on both synthetic and real datasets, along with qualitative analysis, demonstrate its superior explanation ability. Our method contributes to enhancing the transparency of decision-making in various fields where GNNs are widely applied.

331

332

333

334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

370

371

372

373

374

375

376

377

378

379

380

381

References

Julia Amann, Alessandro Blasimme, Effy Vayena, Dietmar Frey, and Vince I Madai. Explainability for artificial intelligence in healthcare: a multidisciplinary perspective. BMCMedical Informatics and Decision Making , 20(1):1-9, 2020.

Steve Azzolin, Antonio Longa, Pietro Barbiero, Pietro Lio, and Andrea Passerini. Global explainability of GNNs via logic combination of learned concepts. In The Eleventh International Conference on Learning Representations , 2023.

Mohit Bajaj, Lingyang Chu, Zi Yu Xue, Jian Pei, Lanjun Wang, Peter Cho-Ho Lam, and Yong Zhang. Robust counterfactual explanations on graph neural networks. Advances in Neural Information Processing Systems , 34:5644-5655, 2021.

Federico Baldassarre and Hossein Azizpour. Explainability techniques for graph convolutional networks. In International Conference on Machine Learning (ICML) Workshops, 2019 Workshop on Learning and Reasoning with Graph-Structured Representations , 2019.

David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. Network dissection: Quantifying interpretability of deep visual representations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , July 2017.

Alexander Binder, Grégoire Montavon, Sebastian Lapuschkin, Klaus-Robert Müller, and Wojciech Samek. Layer-wise relevance propagation for neural networks with local renormalization layers. In Artificial Neural Networks and Machine Learning-ICANN 2016: 25th International Conference on Artificial Neural Networks, Barcelona, Spain, September 6-9, 2016, Proceedings, Part II 25 , pages 63-71. Springer, 2016.

Lukas Faber, Amin K. Moghaddam, and Roger Wattenhofer. When comparing to ground truth is wrong: On evaluating gnn explanation methods. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining , pages 332-341, 2021.

Qizhang Feng, Ninghao Liu, Fan Yang, Ruixiang Tang, Mengnan Du, and Xia Hu. Degree: Decomposition based explanation for graph neural networks. In International Conference on Learning Representations , 2022.

Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in neural information processing systems , 30, 2017.

Qiang Huang, Makoto Yamada, Yuan Tian, Dinesh Singh, and Yi Chang. Graphlime: Local interpretable model explanations for graph neural networks. IEEE Transactions on Knowledge and Data Engineering , 2022.

Zexi Huang, Mert Kosan, Sourav Medya, Sayan Ranu, and Ambuj Singh. Global counterfactual explainer for graph neural networks. In Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining , pages 141-149, 2023.

Brian Kenji Iwana, Ryohei Kuroki, and Seiichi Uchida. Explaining convolutional neural networks using softmax gradient layer-wise relevance propagation. In 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) , pages 4176-4185, 2019.

Jeroen Kazius, Ross McGuire, and Roberta Bursi. Derivation and validation of toxicophores for mutagenicity prediction. Journal of medicinal chemistry , 48(1):312-320, 2005.

Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations , 2017.

Wanyu Lin, Hao Lan, and Baochun Li. Generative causal explanations for graph neural networks. In International Conference on Machine Learning , pages 6666-6679. PMLR, 2021.

Ana Lucic, Maartje A Ter Hoeve, Gabriele Tolomei, Maarten De Rijke, and Fabrizio Silvestri. Cfgnnexplainer: Counterfactual explanations for graph neural networks. In International Conference on Artificial Intelligence and Statistics , pages 4499-4511. PMLR, 2022.

Dongsheng Luo, Wei Cheng, Dongkuan Xu, Wenchao Yu, Bo Zong, Haifeng Chen, and Xiang Zhang. Parameterized explainer for graph neural network. Advances in neural information processing systems , 33:19620-19631, 2020.

Xinjun Pei, Long Yu, and Shengwei Tian. Amalnet: A deep learning framework based on graph convolutional networks for malware detection. Computers & Security , 93:101792, 2020.

Douglas EV Pires, Tom L Blundell, and David B Ascher. pkcsm: predicting small-molecule pharmacoki382 netic and toxicity properties using graph-based signatures. Journal of medicinal chemistry , 58(9):4066383 4072, 2015. 384

385

386

387

Phillip E Pope, Soheil Kolouri, Mohammad Rostami, Charles E Martin, and Heiko Hoffmann. Explainability methods for graph convolutional neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 10772-10781, 2019.

388

389

390

391

392

393

394

395

396

397

398

399

400

401

402

403

404

405

406

407

408

409

410

411

412

413

414

415

416

417

418

419

420

421

Michael Sejr Schlichtkrull, Nicola De Cao, and Ivan Titov. Interpreting graph neural networks for nlp with differentiable edge masking. In International Conference on Learning Representations , 2021.

Thomas Schnake, Oliver Eberle, Jonas Lederer, Shinichi Nakajima, Kristof T Schütt, Klaus-Robert Müller, and Grégoire Montavon. Higher-order explanations of graph neural networks via relevant walks. IEEE transactions on pattern analysis and machine intelligence , 44(11):7581-7596, 2021.

Caihua Shan, Yifei Shen, Yao Zhang, Xiang Li, and Dongsheng Li. Reinforcement learning enhanced explainer for graph neural networks. Advances in Neural Information Processing Systems , 34:22523-22533, 2021.

Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating activation differences. In International conference on machine learning , pages 3145-3153. PMLR, 2017.

Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International conference on machine learning , pages 3319-3328. PMLR, 2017.

Juntao Tan, Shijie Geng, Zuohui Fu, Yingqiang Ge, Shuyuan Xu, Yunqi Li, and Yongfeng Zhang. Learning and evaluating graph neural network explanations based on counterfactual and factual reasoning. In Proceedings of the ACM Web Conference 2022 , pages 1018-1027, 2022.

Minh Vu and My T Thai. Pgm-explainer: Probabilistic graphical model explanations for graph neural networks. Advances in neural information processing systems , 33:12225-12235, 2020.

Daixin Wang, Zhiqiang Zhang, Jun Zhou, Peng Cui, Jingli Fang, Quanhui Jia, Yanming Fang, and Yuan Qi. Temporal-aware graph neural network for credit risk prediction. In Proceedings of the 2021 SIAM International Conference on Data Mining (SDM) , pages 702-710. SIAM, 2021.

Lingfei Wu, Peng Cui, Jian Pei, and Liang Zhao. Graph Neural Networks: Foundations, Frontiers, and Applications . Springer Nature, 2022.

Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In International Conference on Learning Representations , 2019.

Zhitao Ying, Dylan Bourgeois, Jiaxuan You, Marinka Zitnik, and Jure Leskovec. Gnnexplainer: Generating explanations for graph neural networks. Advances in neural information processing systems , 32, 2019.

Hao Yuan, Haiyang Yu, Shurui Gui, and Shuiwang Ji. Explainability in graph neural networks: A taxonomic survey. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2022.

Hao Yuan, Haiyang Yu, Jie Wang, Kang Li, and Shuiwang Ji. On explainability of graph neural networks via subgraph explorations. In International Conference on Machine Learning , pages 12241-12252. PMLR, 2021.

Yue Zhang, David Defazio, and Arti Ramesh. Relex: A model-agnostic relational model explainer. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society , pages 1042-1049, 2021.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel local-level post-hoc GNN explainer (GOAt) that takes the issue of discriminability and consistency into consideration.  The discriminability and consistency of graph explanation are significant and meaningful for the community’s understanding of the GNN decision-process. The GOAt explainer attributes the prediction of the trained GNN to the input graph features, i.e., the node feature matrix and the adjacency matrix, based on the Equal Contribution technique. The authors implement the specific GOAt with regard to GCN, GraphSAGE, and GIN. Sufficient experiments, including both qualitative and quantitative evaluation, verify that GOAt outperforms the SOTA GNN explainers in terms of both fidelity, discriminability, and stability.\n\nSTRENGTHS:\nS1: The issue of discriminability and consistency (i.e., stability) in GNN explanation methods is thought-provoking. However, it is neglected by previous methods. The GOAt explainer bridges the research gap.\nS2: The mathematical derivation of the equal contribution technique is solid and ingenious, which well support the design of GOAt explainer.\nS3: The experiments to verify the superiority of GOAt are sufficient and the results are discussed in detail.\n\nWEAKNESSES:\nW1: My main concern is about the effectiveness of the attribution strategy based on equal contribution. Though the experimental results have shown it outperformance, the intuitive reason why equal contribution can improve the discriminability and stability of GOAt explanation is still obscure for me. In other words, the motivation behind GOAt is not well-illustrated.\nW2: The organization and writing of this manuscript need to be improved. Some contents are a little confusing. For example, in line 155, the authors claim that the number of occurrences is not necessarily equal to the number of unique variables. However, the example following does not explain this claim well.\n\nQUESTIONS:\nQ1: If GOAt take the node feature matrix X as constant finally (as stated in line 209), introducing this setup at beginning may be able to accelerate the GOAt inference. Moreover, why not take both node feature and topological structure as explanation, as done in GNNExplainer?\nSee WEAKNESSES for more detailed questions.\n\nLIMITATIONS:\nThe authors have discussed some limitation about GOAt, including the requirement of expert knowledge, the failure in explaining deep models (e.g., Transformer). In my regard, as a post-hoc explainer, GOAt has to access the detailed architecture of the GNN model to be explained, which may serve as the main limitation in real-world application.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the local explanation algorithm for Graph Neural Networks (GNN), Graph Output Attribution (GOAt), is proposed. The key idea of the algorithm is decomposing the GNN's forward propagation as sum of product form. After decomposing, authors compute the contributions of nodes and edges by proposed measure. Next, (sum of the contribution including the interest node or edge) - (contribution from global bias) becomes the contribution (explanation) of the node or edge. The experiments show (1) the explanation can capture the importance of edges (fidelity, discriminability) (2) the explanation is consistence (Stability).\n\nSTRENGTHS:\n1. The algorithm is general for graph neural networks.\n2. The examples help the reader understand the explanations.\n3. The experiments show the performance of proposed model with regard to baselines for various criteria.\n\nWEAKNESSES:\n1. The paper used zero-vector as base manifold. It makes that the contributions for all variables are always equal.\n2. The contribution does not consider the value of variables.\n3. It would be better to explain the difference between the baseline algorithms and proposed algorithm.\n\nQUESTIONS:\n1. What is motivation of setting base manifold as zero-vector? It seems to make equal contribution, but I think the other resonable explanation is necessary.\n2. When we consider the 3 variables (x1',x2',x3'), the manifolds at (x1,x2,x3) are {(x1,x2',x3'), (x1',x2,x3'), (x1',x2',x3)}? If so, why {(x1,x2,x3'),(x1,x2',x3),...} are not considered like shapley value?\n3. In lemma 4, why \"when the number of nodes N is large\" is necessary?\n4. In line 144, the |V(x^2y)|={x,y}, this makes x and y have same contribution for x^2y. It is not reasonable.\n5. In line 154, it is hard to understand O(v,z) (the number of occurrences of ν among the variables of z).\n6. In line 217, I think it is unfair to compare the proposed edge explanation and the edge explanation from node explanations of baselines by [3]. Because the proposed algorithm can compute the node explanation like baselines, same process [3] should be used for edge explanation from proposed node explanation.\n7. In figure 3 (h), the plot seems like there are less points than others.\n8. In figure 3, what kinds of algorithms is used for 2D plotting? How the position of green scatter points is consistent for various embeddings? I think the different 2D projections should be used for each embeddings. \n9. In line 282, why the figure 3 (h) explains the data samples use only 3 explanation? If 3 explanations used, I think 2^3 points (existence of each explanation) should exist in figure 3.\n10. In table 1, there is no motifs for \"This indicates that the presence of more carbon rings in a molecule increases\n its likelihood of being mutagenic (Class 0)\" (line 293) and \"GOAt discovers that the GNN considers a higher number of carbon rings as evidence of chemical compounds being active against non-small cell lung cancer\" (line 296)\n\n(minor comment) there is no definition for the j-th feature of i-th node (line 126)\n\nLIMITATIONS:\nThe paper used zero-vector as base manifold. It makes that the contributions for all variables are always equal. Also, the proposed contribution consider absence or presence of variable only without the value of variable.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIt proposes a local GNN explanation method called GOAt by analysing the output attribution of graphs. Specifically, it proposes a novel explanation framework by the notion of equation contribution, which is transparent compared to the existing black-box type explanation models. Experiments on real and simulation data are tested on several metrics, where the results are convincing.\n\nSTRENGTHS:\n1. The point of equation contribution is quite novel and interesting. Especially its transparency compared to current baselines.\n2. The experiments are extensive and convincing.\n\nWEAKNESSES:\n1. In line 28, it seems that \"their effectiveness relies heavily on the quality of local-level explanations\" is the main point of abandoning the global style in this paper, could you explain more about this in the rebuttal phase or say more about it in section 2 - see second con below.\n2. In section 2, the differences between global and local explanatory methods should be introduced in a proper way to show the superiority of global explanations. Also, please consider including the matrices in lines 66-69 as another subsection to introduce the matrices together.\n\nQUESTIONS:\n1. Major:\n    1. In lines 103-107, the reasonableness of the independence between the variables $X$ and $A$ in the contribution for $f(A,X)$ should be further improved. As I understand it, the equal contribution property of $f_{i,j}(A,X)=\\sum_k\\sum_l A_{i,k} X_{k,l} W_{l,j}$ is not identical to $A_{i,k} X_{k,l} W_{l,j}$, because the distribution of $A$ can never be independent of $X$. In line 103 the authors mention that \"since $A_{i,k}$ is usually independent of $X_{k,l}$ under proper data cleaning\" - this is true, while it is not equal to $\\sum_k\\sum_l A_{i,k} X_{k,l} W_{l,j}$.\n    2. Could you please explain more about the reduction of equation (7)?\n    3. For the proof of Lemma 4 and Theorem 5, what's the use of the condition requiring a large number of nodes? It is highly recommended that the authors make this condition more practical or reanable in real-world cases.\n2. Minor:\n    1. In lemma 2, better include the condition of $x_k \\neq 0$;\n    2. Is the number of variables $|V(z)|$ of equation (8) equal to the number of parameters?\n    3. I'm also wondering if there is a gap between node-based and edge-based explanation methods, since in the experiments the node-based approaches are aligned with the edge-based ones by using their induced subgraphs. Can the authors explain more about the correct explanation is symmetric in both types of methods, or provide some basic intuition of this?\n    4. In lines 173-177 the authors mention the property of completeness. A question here is whether in the experiments the top-k edges are still selected to compute the evaluation matrices?\n    5. Following the idea of matrix-based understanding of GNNs, there is a recent paper you might consider citing: https://arxiv.org/abs/2305.06142 Feature expansion for graph neural networks. J Sun, L Zhang, G Chen, P Xu, K Zhang, and Y Yang. ICML'23. You can use it to extend your framework to more GNN approaches.\n\nLIMITATIONS:\nThe authors presented the limitations in the Appendix about wider range of model architectures.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nUnlike existing methods that rely on training auxiliary models, the paper introduces a computationally efficient instance-level GNN explanation technique (called GOAt), which enables attribution of GNN output to input features by direct algebraically computation. More specifically, a GNN is represented in an expansion form as a sum of scalar product term, involving input graph features, model parameters, and activation levels. Based on the assumption that all scalar variables in a scalar product term contribute equally to the term, each product term can be attributed to its corresponding factors and thus to input features, which gives the importance of each node or edge feature in the input graph to GNN outputs. Besides that, the paper also introduce two new metrics called discriminability and stability, which assess the ability of explanations to distinguish between classes and the ability to generate consistent explanations across similar data instances, respectively. Through extensive experiments on synthetic and real data, the paper show that GOAt has consistently outperformed various state-of-the-art GNN explainers in terms of fidelity, discriminability, and stability.\n\nSTRENGTHS:\n1.\tThe assumption that all scalar variables in a scalar product term contribute equally to the term is acceptable to me. And the derived lemmas and theorems are well proved and practically applicable.\n2.\tThe proposed explanation method is computationally efficient, and does not rely on back-propagation with gradients, hyper-parameters or training complex black-box models, which is novel to me.\n3.\tThe work contributes a new attribution method to the field of explaining GNNs (might extend to other types of neural networks, e.g., CNN) .\n\nWEAKNESSES:\n1.\tThe specific calculation expression of the attribution process needs to be carefully derived with expert knowledge for different neural network architectures and becomes very complex as the network goes deeper, which greatly limits its widespread use, though it shows superior explanation ability.\n2.\tThe main argument that existing methods struggle with issues such as overfitting to noise, insufficient discriminability, and inconsistent explanations across data samples of the same class is not well elaborated. Though the experiments seem to have verified this argument, it will be more motivated if some detailed illustrations (e.g., by presenting a toy example) could be provided in the introduction part.\n\nQUESTIONS:\n1.\tThe assumption that the number of nodes of the input graph is large in the key lemma(Lemma 4) and the theorem(Theorem 5) is not well established in the datasets used in experiments.\n\nLIMITATIONS:\na)\tBuild an algorithm to automatically derive the attribution expressions based on the network architecture.\nb)\tAlthough the experiments seem to corroborate the main argument, bolstering the introduction with some detailed illustrations (for instance, by incorporating a simple, illustrative example) could make the paper more compelling.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-24"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work studies the problem of GNN local explanation.  It points out that existing GNN explainers are suffering from a few limitations including insufficient discriminability, inconsistency on same-class data samples, and overfitting to noise, and the aim to address these limitations by proposing the GOAt method. Given a pre-trained GNN model, GOAt defines a contribution score on each node feature and each edge feature (i.e. values on each edge) by expanding the GNN as a sum of scalar products involving these features, and thus would be able to find the edges that contribute more to the GNN's output. The authors evaluate GOAt with the frequently used fidelity metric, as well as the newly proposed discriminability and stability metric in this paper.\n\nSTRENGTHS:\n1. Results are impressive.\n2. Code is provided in the supplementary. \n3. The newly introduced discriminability and stability metric for explanation evaluation makes sense to me.\n\nWEAKNESSES:\n1. The writing for the method section is not easy to follow. \n2. The motivation of why the contribution should be designed in this way is not clearly explained. Why the current design can address the insufficient discriminability, inconsistency on same-class data samples, and overfitting to noise issues also remains unclear to me.\n\nSome minor weaknesses:\n1. There seems to be one typo in the appendix proof lemma 2: line 11 should be X_j not E_j?\n2. The term ``feature`` is used to describe all X_{ij}s and A_{ij}s, which looks a little bit confusing for me. I think usually people use the term ``feature`` to describe the attributes on nodes and edges, but here A_{ij} is more like a weight scaler on edge instead of the edge attributes? I feel it would be clearer to put a note to clarify that the term ``feature`` is used to describe all node attributes and edge weights.\n3. It would be clearer if the authors can explain what does each lemma and theorem imply.\n\nQUESTIONS:\nQ1. I am still very confused about, why the contribution should be defined in this way? In other words, what does this formula imply and why it can act as an evaluation of contribution?\n\nQ2. Why GOAt can achieve better discriminability?\n\nQ3. What is the scope of GNN designs that can be explained by GOAt? (For example, is every GNN model that can be formulated in the form of formula (1) covered in the scope?)\n\nLIMITATIONS:\nPlease refer to ``weaknesses``.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-27"}