Original: Explaining Graph Neural Networks with Structure-Aware Cooperative Games
Revised: Explaining Graph Neural Networks with Structure-Aware Cooperative Games

Original: Anonymous Author(s)

Original: Abstract
Revised: Abstract

Original: Explainingpredictions made by machine learning models is  important and haveattracted an increased interest.
Revised: Explaining machine learning models is an important and increasingly popular area of research interest.

Original: The Shapley value fromcooperative game theoryhas been proposed as a prime approach to compute feature importance towardspredictions, especially for images, text, tabular data, and recently graph neuralnetworks (GNNs) on graphs.
Revised: The Shapley value from game theory has been proposed as a prime approach to compute feature importance towards model predictions on images, text, tabular data, and recently graph neural networks (GNNs) on graphs.

Original: In this work, we revisit the appropriateness of the 
Revised: In this work, we revisit the appropriateness of the Shapley value for

Original: Shapley value for GNN explanation, where the task is to identify the most important subgraph and constituent nodes for GNN predictions.
Revised: GNN explanation, where the task is to identify the most important subgraph and constituent nodes for GNN predictions.

Original: We purport thatthe Shapley value is a non-ideal choice for graph data because it is by definitionnot structure-aware.
Revised: We claim that the Shapley value is a non-ideal choice for graph data because it is by definition not structure-aware.

Original: We propose a Graph Structure-aware eXplanation (GStarX)method to leverage the critical graph structure information to improve the explanation.
Revised: We propose a Graph Structure-aware eXplanation (GStarX) method to leverage the critical graph structure information to improve the explanation.

Original: Specifically, we define a scoring function based on a new structure-awarevalue fromthe cooperative game theory called the HN value.
Revised: Specifically, we define a scoring function based on a new structure-aware value from cooperative game theory proposed by Hamiache and Navarro (HN).

Original: When used to scorenode importance, the HN value utilizes graph structures to attribute cooperationsurplus between neighbor nodes, resembling message passing in GNNs, so thatnode importance scores reflect not only the node feature importance, but also thestructural roles.
Revised: When used to score node importance, the HN value utilizes graph structures to attribute cooperation surplus between neighbor nodes, resembling message passing in GNNs, so that node importance scores reflect not only the node feature importance, but also the node structural roles.

Original: We demonstrate that GStarX produces qualitatively more intuitiveexplanations, and quantitatively improves explanation fidelity over strong baselineson chemical graph property prediction and text graph sentiment classification.
Revised: We demonstrate that GStarX produces qualitatively more intuitive explanations, and quantitatively improves explanation fidelity over strong baselines on chemical graph property prediction and text graph sentiment classification.

Original:  Introduction
Revised:  Introduction

Original: Explainability is crucial for complex machine learning (ML) models in sensitive applications, helpingestablish user trust and providing insights for potential  improvements.
Revised: Explainability is crucial for complex machine learning (ML) models in sensitive applications, helping establish user trust and providing insights for potential model improvements.

Original: Many efforts focus onexplaining models on images, text and tabular data.
Revised: Many efforts focus on explaining models on images, text, and tabular data.

Original: In contrast, the explainability of models ongraph data is yet underexplored.
Revised: In contrast, the explainability of models on graph data is yet underexplored.

Original: Since explainability can be especially critical for many graph taskslike drug discovery and interest in deep graph models is growing rapidly, further investigation ofgraph explainability is warranted.
Revised: Since explainability can be especially critical for many graph tasks like drug discovery, and interest in deep graph models is growing rapidly, further investigation of graph explainability is warranted.

Original: In this work, we study graph ML explanation with graph neuralnetworks (GNNs) as the target models, given their popularity and widespread use.
Revised: In this work, we study graph ML explanation with graph neural networks (GNNs) as the target models, given their popularity and widespread use for graph machine learning tasks [42, 29, 38, 34, 33, 45].

Original: In ML explainability the Shapley value [1] has been deemed as a “fair” scoring function for computingfeature importance.
Revised: In ML explainability, important features are identified, and the Shapley value [30] has been deemed as a “fair” scoring function for computing feature importance.

Original: Originally fromthe cooperative game theory, many values, including the Shapleyvalue, have been proposed for allocating a total payoff to players in a game.
Revised: Originally from cooperative game theory, many values, including the Shapley value, have been proposed for allocating a total payoff to players in a game.

Original: When used for scoringthe feature importance of a data instance, the model prediction is treated as the total payoff andthe features are considered as players.
Revised: When used for scoring the feature importance of a data instance, the model prediction is treated as the total payoff and the features are considered as players.

Original: In particular, for an instance with n features { x 1 , . . .
Revised: In particular, for an instance with n features { x 1 , . . .

Original: x n } ,the Shapley value of its i th feature x i is computed via aggregating m ( i, S ) , which are the marginalcontributions of x i to sets of other features x S ⊆ { x 1 , . . .
Revised: x n } , the Shapley value of its i th feature x i is computed via aggregating m ( i, S ) , which are the marginal contributions of x i to sets of other features x S ⊆ { x 1 , . . .

Original: , x n } \ { x i } .
Revised: , x n } \ { x i } .

Original: Each x S is called a coalition .m
Revised: Each x S is called a coalition .

Original:  ( i, S ) is computed as the difference between model outputs forx S ∪ { x i } and x S , which is meantto capture the interaction between x i and x S . The Shapley value is widely used for explaining MLmodels on images, text, and tabular data, when the features are pixels, words, and attributes [2, 3].
Revised: Each m ( i, S ) is computed as the difference between model outputs for

Original: “is still quite good – natured and not a bad way to spend an hour” spendgood bad still natured and not hour to an is quite a way 3 1   3  31/ 1/ 1/ 1/ 1/ 1/  1/
Revised: “is still quite good – natured and not a bad way to spend an hour” spendgood bad still natured and not hour to an is quite a way 3 1   3  31/ 1/ 1/ 1/ 1/ 1/  1/

Original: HNweights
Revised: HNweights

Original: [Figure]
Revised: [Figure]

Original: weights Marginal Contributions
Revised: x S ∪ { x i } and x S , e.g., difference of probability belonging to a target class for these two set of features, and it is meant to capture the interaction between x i and x S . The Shapley value is widely used for explaining ML models on images, text, and tabular data, when the features are pixels, words, and attributes [22, 24].

Original: The Shapley value has recently been extended to explain GNNs on graphs by feature importancescoring as above, where features are nodes [4] or supernodes [5].
Revised: The Shapley value has recently been extended to explain GNNs on graphs through feature importance scoring as above, where features are nodes [9] or supernodes [44].

Original: We argue that the Shapley valueis a non-ideal choice for (super)node importance scoring because its contribution aggregation isnon-structure-aware .
Revised: We argue that the Shapley value is a non-ideal choice for (super)node importance scoring because its contribution aggregation is non-structure-aware .

Original: The Shapley value aggregation assumes no structural relationship between x iand x S even though they are both parts of the input graph (a review of the Shapley value  in Section2.2).
Revised: The Shapley value aggregation assumes no structural relationship between x i and x S even though they are both parts of the input graph (a review of the Shapley value is in Section 2.2).

Original: Since the graph structure generally contains critical information and is crucial to the success of
Revised: Since the graph structure generally contains critical information and is crucial to the success of

Original: GNNs, we consider properly leveraging the structure with a better structure-aware scoring function.
Revised: GNNs, we consider properly leveraging the structure with a better structure-aware scoring function.

Original: We propose Graph Structure-aware eXplanation (GStarX), where we construct a structure-aware nodeimportance scoring function based on the HN value [6] fromthe cooperative game theory.
Revised: We propose Graph Structure-aware eXplanation (GStarX), where we construct a structure-aware node importance scoring function based on the Hamiache-Navarro (HN) value [15] from cooperative game theory.

Original: Recall that GNNs make predictions via message passing (MP), during which node representations are updatedby aggregating messages fromtheir neighbors.
Revised: Recall that GNNs make predictions via message passing, during which node representations are learned by aggregating messages from neighbors.

Original: MP aggregates bothnode feature information andstructure information, resulting in powerful structure-aware models [7].
Revised: Message passing aggregates both feature and structure information, resulting in powerful structure-aware models [5].

Original: The HN value shares a similaridea to MP by allocating the payoff surplus generated from the cooperation between neighboringplayers (nodes).
Revised: The HN value shares a similar idea to message passing by allocating the payoff surplus generated from the cooperation between neighboring players (nodes).

Original: When used as a scoring function to explain node importance, the HN value capturesboth features and structural interactions between nodes (details in Section 4).
Revised: When used as a scoring function to explain node importance, the HN value captures both features and structural interactions between nodes (details in Section 4).

Original: An example comparingthe Shapley value and the HN value is shown in Figure 1 (a) .
Revised: Figure 1 (a) shows an example comparing the Shapley value and the HN value.

Original: In this example, their difference boilsdown to different aggregation weights of marginal contributions, where the former is uniform and thelatter is structure-aware (details in Section 3.2).
Revised: In this example, their difference boils down to different aggregation weights of marginal contributions, where the former is uniform and the latter is structure-aware (details in Section 3.2).

Original: In summary, we make the following contributions:
Revised: In summary, our contributions are:

Original: • Identify the non-structure-aware limitation of the Shapley value for GNN explanation.
Revised: • Identify the non-structure-aware limitation of the Shapley value for GNN explanation.

Original: • Introduce the structure-aware HN value fromthe cooperative game theory to the graph machinelearning community and connect it to the GNN message passing and GNN Explanation.•
Revised: • Introduce the structure-aware HN value from cooperative game theory to the graph machine learning community and connect it to the GNN message passing and GNN explanation.

Original:  Propose a new HN-value-based GNN explanation method GStarX, and demonstrate the superiorityof GStarX over strong baselines for explaining GNNs on chemical and text graphs.
Revised: • Propose a new HN-value-based GNN explanation method GStarX, and demonstrate the superiority of GStarX over strong baselines for explaining GNNs on chemical and text graphs.

Original:  Preliminaries 
Revised:  Preliminaries 2.1 Graph neural networks

Original: 2.1 Graph neural networks

Original: Consider a graph G with (feature-enriched) nodes V and edges E .
Revised: Consider a graph G with (feature-enriched) nodes V and edges E .

Original: We denote G as G =
Revised: We denote G as G =

Original: ( V , X , A ) ,where X ∈ R n × d denotes d -dimensional features of n nodes in V , and A ∈ { 0 , 1 } n × n denotes theadjacency matrix specifying edges in E .
Revised: ( V , X , A ) , where X ∈ R n × d denotes d -dimensional features of n nodes in V , and A ∈ { 0 , 1 } n × n denotes the adjacency matrix specifying edges in E .

Original: GNNs make predictions on G by learning representationsvia the message-passing mechanism.
Revised: GNNs make predictions on G by learning representations via the message-passing mechanism.

Original: During message passing, the representation of each nodeu ∈ V is updated by aggregating its only representation and representations (messages) from itsneighbors, denoted as N ( u ) .
Revised: During message passing, the representation of each node u ∈ V is updated by aggregating its own representation and representations (messages) from its neighbors. We denote the set of neighbors as N ( u ) .

Original: This aggregation is recursively applied, so u can collect messagesfrom its multi-hop neighbors and produce structure-aware representations [7].
Revised: This aggregation is recursively applied, so u can collect messages from its multi-hop neighbors and produce structure-aware representations [5].

Original: With h ( l ) i denotesthe representation of node i at iteration l , and AGGR ( · , · ) denotes the aggregation operation, e.g. summation, the representation update is shown in Equation 
Revised: With h ( l ) i denotes the representation of node i at iteration l , and AGGR ( · , · ) denotes the aggregation operation, e.g. summation, the representation update is shown in Equation 1.

Original: [Equation]
Revised: [Equation]

Original: 2.2 Cooperative games
Revised: 2.2 Cooperative games

Original: A cooperative game denoted by ( N, v ) , is defined by a set of players N = { 1 , . . . , n } , and acharacteristic function v : 2 N → R .
Revised: A cooperative game denoted by ( N, v ) , is defined by a set of players N = { 1 , . . . , n } , and a characteristic function v : 2 N → R .

Original: v takes a subset of players S ⊆ N , called a coalition , and mapsit to a payoff v ( S ) , where v ( ∅ ) :
Revised: v takes a subset of players S ⊆ N , called a coalition , and maps it to a payoff v ( S ) , where v ( ∅ ) :

Original: = 0 .
Revised: = 0 .

Original: A solution function ϕ is a function maps each given game ( N, v )to ϕ ( N, v )
Revised: A solution function ϕ is a function maps each given game ( N, v ) to ϕ ( N, v )

Original: ∈ R n .
Revised: ∈ R n .

Original: The vector ϕ ( N, v ) , called a solution , represents a certain allocation of the totalpayoff v ( N ) generated by all players to each individual, with the i th coordinate ϕ
Revised: The vector ϕ ( N, v ) , called a solution , represents a certain allocation of the total payoff v ( N ) generated by all players to each individual, with the i th coordinate ϕ

Original: i ( N, v ) being thepayoff attributed to player i .
Revised: i ( N, v ) being the payoff attributed to player i .

Original: ϕ ( N, v ) is also called the “value” of the game when it satisfies certainproperties, and different values were proposed to name solutions with different properties [1, 8].
Revised: ϕ ( N, v ) is also called the “value” of the game when it satisfies certain properties, and different values were proposed to name solutions with different properties [30, 35].

Original: The Shapley value is one popular solution of cooperative games.
Revised: The Shapley value is one popular solution of cooperative games.

Original: The main idea is to assign eachplayer a “fair” share of the total payoff by considering all possible player interactions.
Revised: The main idea is to assign each player a “fair” share of the total payoff by considering all possible player interactions.

Original: For example,when player i cooperates with a coalition S , the total payoff v ( S ∪ { i } ) may be very different fromv ( S ) + v ( { i } ) because of i ’s interaction with S.
Revised: For example, when player i cooperates with a coalition S , the total payoff v ( S ∪ { i } ) may be very different from v ( S ) + v ( { i } ) because of i ’s interaction with S.

Original: Thus the marginal contribution of i to S is defined asby m ( i, S )
Revised: Thus the marginal contribution of i to S is defined as by m ( i, S )

Original: = v ( S ∪ { i } ) − v ( S ) .
Revised: = v ( S ∪ { i } ) − v ( S ) .

Original: Then the formula of the Shapley value for i is shown in Equation 2,where marginal contributions to all possible coalitions S ⊆
Revised: Then the formula of the Shapley value for i is shown in Equation 2, where marginal contributions to all possible coalitions S ⊆

Original: N \{ i } are aggregated.
Revised: N \{ i } are aggregated.

Original: The first identifyin Equation 2 shows that the aggregation weights are first uniformly distributed among coalition sizesk (outer average), then uniformly distributed among all coalitions with the same size (inner average).ϕ [Equation] ( N, v ) = [Table]
Revised: The first identify in Equation 2 shows that the aggregation weights are first uniformly distributed among coalition sizes k (outer average), then uniformly distributed among all coalitions with the same size (inner average).

Original: [Equation]

Original: [Table]
Revised: [Table]

Original: (cid:123)(cid:0) n − 1 k (cid:1) (cid:88) S ⊆ N \{ i } | S | = k m ( i, S ) = (cid:88)
Revised: v

Original: S ⊆ N \{ i } | S | ! ( n − | S | − 1)! n ! m ( i, S ) (2)
Revised: [Table]

Original: Games with communication structures.
Revised: Games with communication structures.

Original: Although the Shapley value is widely used for cooperativegames, its assumption of fully flexible cooperation among all players may not be achievable.
Revised: Although the Shapley value is widely used for cooperative games, its assumption of fully flexible cooperation among all players may not be achievable.

Original: Somecoalitions may be preferred over others and some may even be impossible due to limited communication among players.
Revised: Some coalitions may be preferred over others and some may even be impossible due to limited communication among players.

Original: Thus, [9] uses a graph G as the communication structure of players to representcooperation preference.
Revised: Thus, [26] uses a graph G as the communication structure of players to represent cooperation preference.

Original: A game with a communication structure is defined by a triple ( N, v, G ) , with
Revised: A game with a communication structure is defined by a triple ( N, v, G ) , with

Original: N being  node set of G .
Revised: N being the node set of G .

Original: It is more practical  when cooperation preference is available.
Revised: This game formulation is more practical than fully flexible cooperation when cooperation preference is available.

Original: Several suchvalues with different properties have been proposed [9, 10, 11, 12] including the HN value [6].
Revised: Several values with different properties have been proposed for such games [26, 2, 13, 18] including the HN value [15].

Original:  GNN explanation via feature importance scoring3.1 Problem formalization
Revised:  GNN explanation via feature importance scoring 3.1 Problem formalization

Original: A general approach to formalize a ML explanation problem is through feature importance scoring[3, 13], where features may refer to pixels of images, word of text, or nodes/edges/subgraphs of graphs.
Revised: A general approach to formalize an ML explanation problem is through feature importance scoring [24, 6], where features may refer to pixels of images, words of text, or nodes/edges/subgraphs of graphs.

Original: Let f ( · ) denote a to-be-explained GNN, G =
Revised: Let f ( · ) denote a to-be-explained GNN, G =

Original: ( V , X , A ) denote an input graph, and 0 ≤ γ ≤ 1denote a sparsity constraint to enforce concise explanation.
Revised: ( V , X , A ) denote an input graph, and 0 < γ < 1 denote a sparsity constraint to enforce concise explanation.

Original: GNN explanation via subgraph scoring isaimed to find a subgraph g that maximizes a given evaluation metric E VAL ( · , · , · ) , which measuresthe faithfulness of g to G regarding making predictions with f ( · ) , i.e.
Revised: GNN explanation via subgraph scoring is aimed to find a subgraph g that maximizes a given evaluation metric E VAL ( · , · , · ) , which measures the faithfulness of g to G regarding making predictions with f ( · ) , i.e.

Original: [Equation]
Revised: [Equation]

Original: When the task is graph classification and f ( · ) outputs a one-sum vector f ( G ) ∈ [0 , 1] C containingprobabilities for G belongs to C classes, an example E VAL can be the prediction probability drop forremoving g from G , i.e. E VAL ( f ( · ) , G , g ) =
Revised: When the task is graph classification and f ( · ) outputs a one-sum vector f ( G ) ∈ [0 , 1] C containing probabilities for G belongs to C classes, an example E VAL can be the prediction probability drop for removing g from G , i.e. E VAL ( f ( · ) , G , g ) =

Original: [ f ( G )]
Revised: [ f ( G )]

Original: c ∗ − [ f ( G\ g )]
Revised: c ∗ − [ f ( G\ g )]

Original: c ∗ with c ∗ =
Revised: c ∗ with c ∗ =

Original: arg max c [ f ( G )] c .
Revised: arg max c [ f ( G )] c .

Original: In practice, since the number of subgraphs is combinatorial in the number of nodes, the objectiveis often relaxed to finding a set of important nodes or edges first, and then inducing the subgraph[14, 15, 4].
Revised: In practice, since the number of subgraphs is combinatorial in the number of nodes, the objective is often relaxed to finding a set of important nodes or edges first and then inducing the subgraph

Original: Then a more tractable objective of finding the optimal set of nodes S ∗ ⊆
Revised: A more tractable objective of finding the optimal set of nodes S ∗ ⊆

Original: V 1 is given by
Revised: V 2 is given by

Original: [Equation]
Revised: [Equation]

Original: Existing methods often boil down to Equation 4 with different scoring functions (S CORE ), anddefining a proper S CORE is non-trivial.
Revised: Existing methods often boil down to Equation 4 with different scoring functions (S CORE ), and finding a proper S CORE is non-trivial.

Original: For example, a simple example of S CORE is to evaluate each node i directly as S CORE ( f ( · ) , G , i ) =
Revised: One example of S CORE is to evaluate each node i directly as S CORE ( f ( · ) , G , i ) =

Original: [ f ( { i } )] c ∗ .
Revised: [ f ( { i } )] c ∗ .

Original: However, this choice misses interactionsbetween nodes and corresponds to a trivial case of no message-passing but only a feed-forwardnetwork applied on { i } .
Revised: However, this choice misses interactions between nodes and corresponds to a trivial case in GNNs where no message-passing is performed for { i } .

Original: Another possible choice is to use E VAL as S CORE , like the example above: S CORE ( f ( · ) , G , i ) =
Revised: Another possibility is to use E VAL as S CORE , e.g., S CORE ( f ( · ) , G , i ) =

Original: [ f ( G )]
Revised: [ f ( G )]

Original: c ∗ − [ f ( G\{ i } )] c ∗ .
Revised: c ∗ − [ f ( G\{ i } )] c ∗ .

Original: However, this again fails to capture interactionsbetween nodes; for example, two nodes i and j may be both important but also complimentary, sotheir contribution to G can only be observed when they are missing simultaneously.
Revised: However, this again fails to capture interactions between nodes; for example, two nodes i and j may be both important but also complimentary, so their contribution to G can only be observed when they are missing simultaneously.

Original: 3.2 Scoring functions from cooperative games
Revised: 3.2 Scoring functions from cooperative games

Original: Given the challenges for defining a proper S CORE , solutions to cooperative games, like the Shapley value, have been proposed with f ( · ) as the characteristic function, i.e. S CORE ( f ( · ) , G , i ) =
Revised: Given the challenges for defining a proper S CORE , solutions to cooperative games, like the Shapley value, have been proposed with f ( · ) as the characteristic function, i.e. S CORE ( f ( · ) , G , i ) =

Original: ϕ
Revised: ϕ

Original: i ( |G| , f ( · ))
Revised: i ( |G| , f ( · ))

Original: However, existing works only use the non-structure-aware Shapley value.
Revised: However, existing works only use the non-structure-aware Shapley value.

Original: Incontrast, values defined on games ( N, v, G ) with communication structures G are naturally structureaware but were never considered.
Revised: In contrast, values defined on games ( N, v, G ) with communication structures G are naturally structureaware but were never considered GNN explanation.

Original: Below we discuss the non-structure-aware limitation of the Shapleyvalue in detail and motivating structure-aware values with practical examples in GNN explanation.
Revised: Below we discuss the non-structure-aware limitation of the Shapley value in detail and motivating structure-aware values with practical examples in GNN explanation.

Original: The Shapley value is defined on games ( N, v ) , which by definition take no graph structures.
Revised: The Shapley value is defined on games ( N, v ) , which by definition takes no graph structures.

Original: Itassumes flexible cooperation between players and uniform distribution of coalition importance thatonly depends on | S | (see Equation 2).
Revised: It assumes flexible cooperation between players and uniform distribution of coalition importance that only depends on | S | (see Equation 2).

Original: Even if a G is given and the game is defined as ( N, v, G ) ,the Shapley value will overlook G when aggregating m ( i, S ) .
Revised: Even if a G is given and the game is defined as ( N, v, G ) , the Shapley value will overlook G when aggregating m ( i, S ) .

Original: In contrast, structure-aware values on( N, v, G ) can be interpreted as a weighted aggregation of coalitions with more reasonable weights.
Revised: In contrast, structure-aware values on ( N, v, G ) can be interpreted as a weighted aggregation of coalitions with more reasonable weights.

Original: Although different solutions ϕ ( N, v, G ) have their nuances in weight adjustments [11, 6, 9, 12], theyshare two key properties: (1) the weight is 0 if i and S are disconnected because they are interpretedas players without communication channels [9], and (2) the weight is impacted by the nature ofconnections between i and S because it is easier for better-connected nodes to communicate.
Revised: Although different solutions ϕ ( N, v, G ) have their nuances in weight adjustments [13, 15, 26, 18], they share two key properties: (1) the weight is zero if i and S are disconnected because they are interpreted as players without communication channels [26], and (2) the weight is impacted by the nature of connections between i and S because it is easier for better-connected nodes to communicate.

Original: A synthetic example.
Revised: A synthetic example.

Original: We take the HN value (definition in Section 4.1) as an example structure-awarevalue and compare it to the Shapley value in a simple graph in Figure 1 (a) .
Revised: We take the HN value (definition in Section 4.1) as an example structure-aware value and compare it to the Shapley value in a simple graph in Figure 1 (a) .

Original: To compute ϕ 1 ( N, v, G ) ,both values aggregates m (1 , S ) for S ∈ {∅ , { 2 } , { 3 } , { 2 , 3 }} .
Revised: To compute ϕ 1 ( N, v, G ) , both values aggregates m (1 , S ) for S ∈ {∅ , { 2 } , { 3 } , { 2 , 3 }} .

Original: The Shapley value first assigns auniform weight 13 to three different | S | , and then splits weights uniformly for the | S | = 1 case to be 16 .
Revised: The Shapley value first assigns a uniform weight 13 to three different | S | , and then splits weights uniformly for the | S | = 1 case to be 16 .

Original: However, the HN value assigns weight 0 for S = { 3 } because 1 and 3 are disconnected in coalition{ 1 , 3 } and are assumed to be two independent graphs that shouldn’t interact (property (1) ).
Revised: However, the HN value assigns weight zero for S = { 3 } because 1 and 3 are disconnected in coalition { 1 , 3 } and are assumed to be two independent graphs that shouldn’t interact (property (1) ).

Original: Theirinteraction is rather captured in the S
Revised: Their interaction is rather captured in the S

Original: = { 2 , 3 } case, when 1 and 3 are connected by the bridgingnode 2. This case is also downweighted from 13 to 14 , as 3 is relatively far from 1 (property (2) ).
Revised: = { 2 , 3 } case, when 1 and 3 are connected by the bridging node 2, and this case is also downweighted from 13 to 14 , as 3 is relatively far from(property (2) ).

Original: A practical example.
Revised: A practical example.

Original: The good properties of structure-aware values can help explain graph tasks.
Revised: The good properties of structure-aware values can help explain graph tasks.

Original: The example in Figure 1 (b) is from GraphSST2 (dataset description in Section 5.1), where thegraph for sentiment classification is constructed from the sentence “is still quite good-natured andnot a bad way to spend an hour” with edges generated by the Biaffine parser [16].
Revised: The example in Figure 1 (b) is from GraphSST2 (dataset description in Section 5.1), where the graph for sentiment classification is constructed from the sentence “is still quite good-natured and not a bad way to spend an hour” with edges generated by the Biaffine parser [12].

Original: Assuming amodel can correctly classify it as positive.
Revised: Assuming a model can correctly classify it as positive.

Original: Intuitively, “good” and “not a bad” are central to thehuman explanation.
Revised: Intuitively, “good” and “not a bad” are central to the human explanation.

Original: To compute the Shapley value of the word “good”, the coalition “not good” willcontribute negatively and diminish the positive importance of “good”, despite the two words lackingany direct connection.
Revised: To compute the Shapley value of the word “good”, the coalition “not good” will diminish the positive importance of “good”, despite the two words lacking any direct connection.

Original: A structure-aware value can instead eliminate the {“not”, “good”} coalition,and only consider interactions between “not” and “good” (in fact, “not” and any other word) when thebridging “bad” appears, hence better binding “not” with “bad” and improving the salience of “good”. 
Revised: A structure-aware value can instead eliminate the {“not”, “good”} coalition, and only consider interactions between “not” and “good” (in fact, “not” and any other word) when the bridging “bad” appears, hence better binding “not” with “bad” and improving the salience of “good”. In Section 5.2, we revisit this example to observe impacts of structure-awareness empirically.

Original:  GStarX: Graph Structure-aware eXplanation
Revised:  GStarX: Graph Structure-aware eXplanation

Original: We propose GStarX, which uses a structure-aware HN-value-based S CORE to explain GNNs.
Revised: We propose GStarX, which uses a structure-aware HN-value-based S CORE to explain GNNs.

Original: Wefirst state the definition of the HN value in cooperative game theory (4.1), and then connect it to the
Revised: We first state the definition of the HN value in cooperative game theory (4.1), and then connect it to the

Original: GNN message passing (4.2), and finally give the GStarX algorithm for GNN explanation (4.3).
Revised: GNN message passing (4.2), and finally give the GStarX algorithm for GNN explanation (4.3).

Original: 4.1 The HN value
Revised: 4.1 The HN value

Original: For any game ( N, v, G )  with a communication structure G and S ⊆ N being a coalition, we use
Revised: Let ( N, v, G ) be a game with a communication structure G and S ⊆ N be a coalition.

Original:  S ∗ = ∪ i ∈ S {N ( i ) } ∪ S to denote the union of S and its neighbors in G .
Revised: Let S ∗ = ∪ i ∈ S {N ( i ) }∪ S to be the union of S and its neighbors in G .

Original: We use S/ G to denote thenode partition of S , which is determined by the connected components  of  the induced subgraph ofSin G (denoted as G [ S ] ).
Revised: Let S/ G be the partition of S containing connected components in G , i.e., S/ G = {{ i | i = j or i and j are connected in S by E of G}| j ∈ S } . Let G [ S ] be the induced subgraph of S in G .

Original: For example, in Figure 1 (b) , when S = {“is”, “an”, “hour”}, S ∗ will be {“is”,“good”, “an”, “hour”, “spend”},and S/ G will be {{“is”}, {“an”, “hour”}}.
Revised: For example, in Figure 1 (b) , when S = {“is”, “an”, “hour”}, S ∗ will be {“is”, “good”, “an”, “hour”, “spend”}, S/ G will be {{“is”}, {“an”, “hour”}}, and

Revised: G [ S ] will be the subgraph with a two-node component (cid:3)(cid:2)

Revised: (cid:0)(cid:1) an - (cid:3) (cid:2) (cid:0) (cid:1) hour and a single node component (cid:3) (cid:2) (cid:0) (cid:1) is .

Original: Definition 4.1 ( Surplus ) .
Revised: Definition 4.1 ( Surplus ) .

Original:  p ( j, S )representing the surplus generated by a coalition S cooperatingwith its neighbor j is defined as
Revised: The surplus p ( j, S ) generated by a coalition S cooperating with its neighbor j is defined as

Original: [Equation]
Revised: [Equation]

Original: Intuitively, p ( j, S ) is generated because S is actively cooperating.
Revised: Intuitively, p ( j, S ) is generated because S is actively cooperating.

Original: Thus, when evaluating a fair payoffto S , a portion of p ( j, S ) should be added to its own payoff v ( S ) .
Revised: Thus, when evaluating a fair payoff to S , a portion of p ( j, S ) should be added to its own payoff v ( S ) .

Original: This idea leads to the  definition ofassociated games regarding the original games, where surplus allocation is performed.
Revised: This idea leads to the next definition of associated games regarding the original games, where surplus allocation is performed.

Original: Definition 4.2 ( HN Associated Game ) .
Revised: Definition 4.2 ( HN Associated Game ) .

Original: Given 0 ≤ τ ≤ 1 representing the portion of surplus that will be allocated to a coalition S .
Revised: Given 0 ≤ τ ≤ 1 representing the portion of surplus that will be allocated to a coalition S for its cooperation with other players.

Original: The HN associated game ( N, v ∗ τ , G ) of ( N, v, G ) is defined as
Revised: The HN associated game ( N, v ∗ τ , G ) of ( N, v, G ) is defined as

Original: [Equation]
Revised: [Equation]

Original: otherwise

Original: [Equation]

Original: The HN value is a solution on ( N, v, G ) .
Revised: The HN value is a solution on ( N, v, G ) .

Original: It is computed by iteratively constructing a series of HNassociated games until it converges to a limit game ( N, ˜ v, G ) .
Revised: It is computed by iteratively constructing a series of HN associated games until it converges to a limit game ( N, ˜ v, G ) .

Original: In other words, we first construct v ∗ τfrom v by surplus allocation.
Revised: In other words, we first construct v ∗ τ from v by surplus allocation.

Original: Then we construct v ∗∗ τ from v ∗ τ by allocating the surplus generated fromthe v ∗ τ and so on.
Revised: Then we construct v ∗∗ τ from v ∗ τ by allocating the surplus generated from the v ∗ τ and so on.

Original: The convergence of the limit game is guaranteed and the result ˜ v is independent ofτ under mild conditions as shown in [6].
Revised: The convergence of the limit game is guaranteed and the result ˜ v is independent of τ under mild conditions as shown in [15].

Original: The HN value of each player is uniquely determined byapplying ˜ v to that player, i.e. ϕ i ( N, v, G ) =
Revised: The HN value of each player is uniquely determined by applying ˜ v to that player, i.e. ϕ i ( N, v, G ) =

Original: ˜ v ( { i } ) .
Revised: ˜ v ( { i } ) .

Original: We state the formal definitions of the limit gameand the uniqueness theorem of the HN value in Appendix E.2.
Revised: We state the formal definitions of the limit game and the uniqueness theorem of the HN value in Appendix E.2.

Original: 4.2 Connecting GNNs and the HN surplus allocation through the message passing lens
Revised: 4.2 Connecting GNNs and the HN surplus allocation through the message passing lens

Original: Both the GNN message passing (Equation 1) and the associated game surplus allocation (Equation6) are iterative aggregation algorithms, with considerable alignment.
Revised: Both the GNN message passing (MP) and the associated game surplus allocation (SA) are iterative aggregation algorithms, with considerable alignment.

Original: In fact, the surplus allocationon each singular node set S = { i } is exactly a message passing operation: Equation 6 becomesan instantiation of Equation 1 with AGGR ( a, b ) = a + τ (cid:80) j b j on a scalar node value a and aneighbor set b .
Revised: In fact, SA on each singular node set S = { i } is exactly MP: Equation 6 becomes an instantiation of Equation 1 with AGGR ( a, b ) = a + τ (cid:80) j b j on a scalar node value a and a neighbor set b .

Original: The algorithms differ in that HN surplus allocation applies more broadly tocaseswhere | S |≥ 1 ; it treats S as a supernode when nodes in S form a connected component in G , andhandles disconnected S component-wise via Equation 7.
Revised: These algorithms differ in that SA applies more broadly to | S |≥ 1 cases; it treats S as a supernode when nodes in S form a connected component in G , and handles disconnected S component-wise via Equation 7.

Original: We illustrate the surplus allocation using a real chemical graph example.
Revised: We illustrate SA using a real chemical graph example.

Original: The molecule shown in Figure 1 (c) is taken from MUTAG (dataset description in Section 5.1).
Revised: The molecule shown in Figure 1 (c) is taken from MUTAG (dataset description in Section 5.1).

Original: It is known to be classifiedas mutagenic because of the -NO2 group (nodes 1, 2, and 3) [17].
Revised: It is known to be classified as mutagenic because of the -NO2 group (nodes 1, 2, and 3) [8].

Original: When we compute v ∗ τ ( { 1 } ) ,the surplus p (2 , { 1 } ) , p (3 , { 1 } ) , and p (4 , { 1 } ) are allocated to node 1 (like messages passed to acentral node in GNN).
Revised: When we compute v ∗ τ ( { 1 } ) , the surplus p (2 , { 1 } ) , p (3 , { 1 } ) , and p (4 , { 1 } ) are allocated to node 1 (like messages passed to a central node in GNN).

Original: Then surplus are aggregated together with v ( { 1 } ) following Equation 6 toform v ∗ τ ( { 1 } ) .
Revised: Then surplus are aggregated together with v ( { 1 } ) following Equation 6 to form v ∗ τ ( { 1 } ) .

Original: For graphs, the surplus allocation approach has two advantages over the uniformaggregation used in the Shapley value: (1) : the aggregated payoff in each v ∗ τ is structure-aware likerepresentations learned by GNNs [7], and (2) the iterative computation preserves locality, whichis what GNNs preserve [18].
Revised: For graphs, the SA approach has two advantages over the uniform aggregation approach used in the Shapley value: (1) The aggregated payoff in each v ∗ τ is structure-aware, like representations learned by GNNs [5], and (2) the iterative computation preserves locality, which is preserved by

Original: In other words, close neighbors heavily influence each other due tocooperation in many iterations, while far away nodes have a lesser influence on each other due to littlecooperation. In this example, since the local -NO2 piece generates a high payoff for the mutagenicityclassification, locally allocating the payoff helps us better understand the importance of the nitrogenatom and the oxygen atoms. Whereas aggregating over many unnecessary coalitions with far-awaycarbon atoms can obscure the true contribution of -NO2. We will revisit this example in Section 5.3.
Revised: GNNs [3].

Original: We now state our main algorithm for explaining GNNs with GStarX, which essentially implementsand solves the objective in Equation 4. We first introduce our HN-value-based S CORE for theobjective. To use the HN value as the S CORE , we need to define the players and the characteristicfunction of the game, and then apply the HN value formula in Equation 6 and 7.
Revised: In other words, these two properties mean close neighbors heavily influence each other due to cooperation in many iterations, while far away nodes less influence each other due to little

Original: Suppose the inputsinclude a graph G with nodes V = { u 1 , . . 
Revised: Input: Graph G with nodes V = { u 1 , . . . , u n } , trained

Original: u n } and label y ∈ { 1 , . .

Original: , C } , a GNN f ( · ) outputsa probability vector f ( G ) ∈ [0 , 1] C ,and the predicted class c ∗ = arg max c [ f ( G )] c . We let V beplayers, and let the normalized probability of the predicted class be the characteristic function v : [Equation]
Revised: GNN f ( · ) , empirical expectation f 0 , hyperparameter τ , max sample size m , number of samples J , sparsity γ .

Original: Algorithm

Original: Input: Graph instance G with nodes V = { u 1 , . . . , u n } , trained model f ( · ) , empirical expectation f 0 , hyperparameter τ , maximum sample size m , number of samples J , sparsity γ

Original: Get the predicted class c ∗ = arg max c [ f ( G )]
Revised: Get the predicted class c ∗ = arg max c [ f ( G )]

Original: c
Revised: c

Original: Definethe characteristic function v ( S ) =
Revised: Define characteristic function v ( S ) =

Original: [ f ( g S )]
Revised: [ f ( g S )]

Original: c ∗ − f 0 c ∗ if n ≤ m then ϕ =
Revised: c ∗ − f 0 c ∗ if n ≤ m then ϕ =

Original: Compute-HN-Values ( G , V , v ( · ) , τ ) else ϕ
Revised: Compute-HN ( G , V , v ( · ) , τ ) else ϕ

Original: =
Revised: =

Original: Compute-HN-Values-MC ( G , V , v ( · ) , τ, m, J ) end if
Revised: Compute-HN-MC ( G , V , v ( · ) , τ, m, J ) end if

Original: Sort ϕ in descending order with indices { π 1 , . . .
Revised: Sort ϕ in descending order with indices { π 1 , . . .

Original: , π n } k
Revised: , π n } k

Original: =
Revised: =

Original: ⌊ γ |V|⌋
Revised: ⌊ γ |V|⌋

Original: Return: S ∗ = { u π 1 , . . .
Revised: Return: S ∗ = { u π 1 , . . .

Original: , u π k }
Revised: , u π k }

Revised: Algorithm

Original: Here G [ S ] denotes the subgraph induced by nodesin S ⊆ V  . 
Revised: Input: Graph instance G with nodes V = { u 1 , . . . , u n } , characteristic function v , hyperparameter τ . for S in 2 N do Compute payoff v ( S ) {Eq.(8)} end for

Revised: Construct matrix H { τ,n, G} {Eq.(16)} repeat

Revised: H = HH until H converges

Revised: Get the limit game ˜ v

Revised: = H v {Eq.(17)}

Revised: Assign the first n entries of ˜ v to ϕ

Revised: Return: ϕ cooperation.

Revised: In the MUTAG example, since the local -NO2 generates a high payoff for the mutagenicity classification, locally allocating the payoff helps us better understand the importance of the nitrogen atom and the oxygen atoms.

Revised: Whereas aggregating over many unnecessary coalitions with far-away carbon atoms can obscure the true contribution of -NO2.

Revised: We will revisit this example in Section 5.2.

Revised: 4.3 The GStarX algorithm

Revised: We now state our algorithm for explaining GNNs with GStarX. Notice that GStarX scores nodes in a graph but not each dimension of node features.

Revised: Feature dimension importance explanation is an orthogonal perspective that can be added on top of GStarX. We leave this extension as a future work.

Revised: GStarX formulates the GNN explanation problem as a feature importance scoring problem, where nodes are scored to find the optimal node-induced subgraph as we introduced in Section 3.1.

Revised: It essentially implements and solves the objective in Equation 4, where an HN-value-based S CORE is used.

Revised: To use such S CORE , we need to define the players and the characteristic function of the game, and then apply the formula in Equation 6 and 7.

Revised: Suppose the inputs are a graph G with nodes

Revised: V = { u 1 , . . . , u n } and label y ∈ { 1 , . . .

Revised: , C } , a GNN f ( · ) outputs a probability vector f ( G ) ∈ [0 , 1] C , and the predicted class c ∗ = arg max c [ f ( G )] c .

Revised: Let V be players, and let the normalized probability of the predicted class be the characteristic function v :

Revised: [Equation]

Original: The normalization term f 0 c ∗ =
Revised: Here the normalization term f 0 c ∗ =

Original: 202 E [[ f ( G )] c ∗ ] is the expectation over a random variable G representing the general graph.
Revised: E [[ f ( G )] c ∗ ] is the expectation over a random variable G representing a general graph.

Original: In practice,we approximate it using the empirical expectation over all G in the dataset.
Revised: In practice, we approximate it using the empirical expectation over all G in the dataset.

Original: The function S CORE willthus be the HN value of the game, i.e. S CORE ( f ( · ) , G , i ) =
Revised: S CORE will be the HN value of the game, i.e., S CORE ( f ( · ) , G , i ) =

Original: ϕ
Revised: ϕ

Original: i ( V , v, G ) =
Revised: i ( V , v, G ) =

Original: ˜ v ( { i } ) .
Revised: ˜ v ( { i } ) .

Original: Giventhe S CORE function, we solve the objective by first computing the scores ϕ ∈ R n then selectingthe top ⌊ γ |V|⌋ scores greedily as in Algorithm 1.
Revised: Given S CORE , we solve the objective by first computing the scores ϕ ∈ R n then selecting the top ⌊ γ |V|⌋ scores greedily as in Algorithm 1.

Original: Practically, like other game-theoretic methods, theexact computation of the HN value is infeasible when the number of players n is large (see Appendix E.3).
Revised: Practically, like other game-theoretic methods, the exact computation of the HN value is infeasible when the number of players n is large.

Original: We thus do an exact computation for small graphs (the if-branch) and Monte-Carlo sampling forlarge graphs (the else-branch).
Revised: We thus do an exact computation for small graphs (the if-branch) and Monte-Carlo sampling for large graphs (the else-branch).

Original: See Appendix  E.3 for detailed algorithms for Compute-HN-Values and Compute-HN-Values-MC .
Revised: The Compute-HN function is shown in Algorithm 2, where the H stands for a matrix form of the associated game defined in Definition 4.2.(See Appendix E.2 and E.3 for details of the matrix form and algorithms for Compute-HN-MC ).

Original: Also, even though the algorithm is stated for graph classification, GStarXworks for node classification as well.
Revised: Also, even though the algorithm is stated for graph classification, GStarX works for node classification as well.

Original: This can be easily seen since GNNs classify nodes u i byprocessing an ego-graph centered at u i , so the task can be converted to graph classification, wherethe label of u iis used as the label of the ego-graph.
Revised: This can be easily seen since GNNs classify nodes u i by processing an ego-graph centered at u i , so the task can be converted to graph classification with the label of u i used as the label of the ego-graph.

Original: We focus on graph classification in the main textfor simpler illustration and discuss more about node classification in Appendix B.
Revised: We focus on graph classification in the main text for simpler illustration and discuss more about node classification in Appendix B.

Original:  Experiments 
Revised:  Experiments 5.1 Experiment settings

Original: 5.1 Experiment settings

Original: Datasets.
Revised: Datasets.

Original: We conduct experiments on datasets  including synthetic data, chemical data, and text data.
Revised: We conduct experiments on datasets from different domains including synthetic graphs, chemical graphs, and text graphs.

Original: A brief description of the datasets is shown below with more detailed statistics in Appendix [-A.
-]
Revised: A brief description of the datasets is shown below with more detailed statistics in Appendix [-A.
-]A.

Original: • Chemical graph property prediction.
Revised: • Chemical graph property prediction.

Original: MUTAG [17], BACE and BBBP [ contain  molecule graphs for graph classification, with atoms as nodes, bonds as edges, and chemical properties as  labels.
Revised: MUTAG [8], BACE and BBBP [39] contain chemical molecule graphs for graph classification, with atoms as nodes, bonds as edges, and chemical properties as graph labels.

Original: • Text graph sentiment classification.
Revised: • Text graph sentiment classification.

Original: GraphSST2 and Twitter [20] contain graphs constructed from text.
Revised: GraphSST2 and Twitter [43] contain graphs constructed from text.

Original: Nodes are words with pre-trained BERT embeddings as features.
Revised: Nodes are words with pre-trained BERT embeddings as features.

Original: Edges are generatedby the Biaffine parser [16].
Revised: Edges are generated by the Biaffine parser [12].

Original: Graphs are labeled as positive or negative sentiment.
Revised: Graphs are labeled as positive or negative sentiment.

Original: • Synthetic graph motif detection.
Revised: • Synthetic graph motif detection.

Original: BA2Motifs [15] contains graphs with a Barabasi-Albert (BA)graph of size 20 and a 5-node motifs in each graph.
Revised: BA2Motifs [25] contains graphs with a Barabasi-Albert (BA) base graph of size 20 and a 5-node motif in each graph.

Original: Node features are 10-dimensional all-onevectors.
Revised: Node features are 10-dimensional all-one vectors.

Original: The motif can be  a house-like structure or a cycle, which separates graphs into two classes.
Revised: The motif can be either a house-like structure or a cycle. Graphs are labelled in two classes based on which motif they contain.

Original: GNNs and explanation baselines.
Revised: GNNs and explanation baselines.

Original: We evaluate GStarX by explaining GCNs [21] on all datasetsin our major experiment.
Revised: We evaluate GStarX by explaining GCNs [19] on all datasets in our major experiment in Section 5.2.

Original: In later analysis, we also evaluate on GIN [22] and GAT [23] on certaindatasets following [5].
Revised: In the ablation study in Section 5.3, we further evaluate on GIN [40] and GAT [36] on certain datasets following [44].

Original: All models are trained to convergence with hyperparameters and performanceshown in Appendix A.2.
Revised: All models are trained to convergence with hyperparameters and performance shown in Appendix A.2.

Original: We compare with 5 strong baselines representing the SOTA methods for GNN explanation: GNNExplainer [14], PGExplainer [15], SubgraphX [5], GraphSVX [4], and
Revised: We compare with 5 strong baselines representing the SOTA methods for GNN explanation: GNNExplainer [41], PGExplainer [25],

Original:  OrphicX [24].
Revised: SubgraphX [44], GraphSVX [9], and OrphicX [21].

Original: In particular, SubgraphX and GraphSVX use Shapley-value-based scoring functions.
Revised: In particular, SubgraphX and GraphSVX use Shapley-value-based scoring functions.

Original: Evaluation metrics.
Revised: Evaluation metrics.

Original: Evaluating explanations is non-trivial due to the lack of ground truth.
Revised: Evaluating explanations is non-trivial due to the lack of ground truth.

Original: Wefollow [5, 20] to employ Fidelity , Inverse Fidelity ( Inv-Fidelity ), and Sparsity as our evaluation metrics.
Revised: We follow [44, 43] to employ Fidelity , Inverse Fidelity ( Inv-Fidelity ), and Sparsity as our evaluation metrics.

Original: Fidelity and Inv-Fidelity measure whether the prediction is faithfully important to the model predictionby removing the selected nodes or only keeping the selected nodes respectively.
Revised: Fidelity and Inv-Fidelity measure whether the prediction is faithfully important to the model prediction by removing the selected nodes or only keeping the selected nodes respectively.

Original: Sparsity promotesfair comparison, since including more nodes generally improves Fidelity  , and explanations withdifferent sizes are not directly comparable.
Revised: Sparsity promotes fair comparison by controlling explanations to have similar sizes, since including more nodes generally improves Fidelity and Inv-Fidelity , and explanations with different sizes are not directly comparable.

Original: Ideal explanations are high Fidelity , low Inv-Fidelity , andhigh Sparsity , indicating relevance and conciseness.
Revised: Ideal explanations should have high Fidelity , low Inv-Fidelity , and high Sparsity , indicating relevance and conciseness.

Original: Equations 9-11 show their formulas.
Revised: Equations 9-11 show their formulas.

Original: in always colorful overblown funny very enjoyably .

Original: style traditional almodóvar occasionally , the and in always colorful overblown funny very enjoyably .

Original: style traditional almodóvar occasionally , the and in always colorful overblown funny very enjoyably .

Original: style traditional almodóvar occasionally , the and in always colorful overblown funny very enjoyably .

Original: style traditional almodóvar occasionally , the and in always colorful overblown funny very enjoyably .

Original: style traditional almodóvar occasionally , the and in always colorful overblown funny very enjoyably .

Original: style traditional almodóvar occasionally , the and is the should you lameness on bad in .

Original: how the title ’s movie clue is the should you lameness on bad in .

Original: how the title ’s movie clue is the should you lameness on bad in .

Original: how the title ’s movie clue is the should you lameness on bad in .

Original: how the title ’s movie clue is the should you lameness on bad in .

Original: how the title ’s movie clue is the should you lameness on bad in .

Original: how the title ’s movie clue

Original: GNNExplainer

Original: PGExplainer

Original: SubgraphX

Original: GraphSVX “occasionally funny , always very colorful and enjoyably overblown in the traditional almodóvar style.”

Original: “the title 's lameness should clue you in on how bad the movie is.”

Original: OrphicX

Original: GStarX identifies the sentiment words more accurately compared to baselines.

Original: [Figure]

Original: [Equation]
Revised: [Equation]

Original: Fidelity and Inv-Fidelity are complementary and are both important for a good explanation g .
Revised: Fidelity and Inv-Fidelity are complementary and are both important for a good explanation g .

Original: Fidelityjustifies the necessity for g to be included to predict correctly.
Revised: Fidelity justifies the necessity for g to be included to predict correctly.

Original: Inv-Fidelity justifies the sufficiency of astandalone g to predict correctly.
Revised: Inv-Fidelity justifies the sufficiency of a standalone g to predict correctly.

Original: As they are analogous to precision and recall, we draw an analogyto the F1 score to propose a single-scalar-metric “harmonic fidelity” ( H-Fidelity ), where we normalizethem by Sparsity and take their harmonic mean; see Appendix A.3 for the formula.
Revised: As they are analogous to precision and recall, we draw an analogy to the F1 score to propose a single-scalar-metric “harmonic fidelity” ( H-Fidelity ), where we normalize them by Sparsity and take their harmonic mean; see Appendix A.3 for the formula.

Original: Hyperparameters.
Revised: Hyperparameters.

Original: GStarX includes three hyperparameters: τ for the  associated game,maximumsample size m , and  number of samples J .
Revised: GStarX includes three hyperparameters: τ for the allocated surplus in the associated game, m as the maximum graph size to perform exact HN value calculation, and J as the number of samples for the MC approximation.

Original: In our experiments we choose τ = 0 .
Revised: In our experiments, we choose τ = 0 .

Original:  since we needτ < 2 n for convergence (Appendix E.2) and all graphs  have less than 200 nodes.
Revised: 01 since we need τ < 2 n for convergence (Appendix E.2) and all graphs in the datasets above have less than 200 nodes.

Original: Bigger m and Jshould be better for the MC approximation, and we found m = 10 and J = n work well empirically.
Revised: For m and J , bigger values should be better for the MC approximation, and we found m = 10 and J = n work well empirically.

Original: 5.2 Quantitative studies
Revised: 5.2 Evaluation results

Revised: Quantitative studies.

Original: We report averaged test set H-Fidelity in Table 1.
Revised: We report averaged test set H-Fidelity in Table 1.

Original: We conduct 8 different runs to get results with Sparsity ranging from 0.5-0.85 in 0.05 increments ( Sparsity cannot be precisely guaranteed minor 
Revised: We conduct 8 different runs to get results with Sparsity ranging from 0.5-0.85 in 0.05 increments ( Sparsity cannot be precisely guaranteed, hence it has minor variations across methods) and report the best H-Fidelity for each method.

Original: [Table]
Revised: GStarX outperforms others on 4/6 datasets and has the highest average. We also follow [44] to show the Fidelity vs. Sparsity plots for all 8 sparsity in Appendix A.4. Qualitative studies.

Original: We visualize the explanations of graphs in GraphSST2 in Figure 2 and compare them qualitatively.
Revised: We visualize the explanations of graphs in GraphSST2 in Figure 2 and compare them qualitatively.

Original: We show explanations selected with high and comparable Sparsity on a positive (upper) graph and anegative (lower) graph. GStarX
Revised: We show explanations selected with high and comparable Sparsity on a positive (upper) graph and a negative (lower) graph. GStarX

Original: concisely captures the important words for sentiment classificationwithout including extraneous ones for both sentences.
Revised: concisely captures the important words for sentiment classification without including extraneous ones for both sentences.

Original: Baseline methods generally select some-butnot-all important sentiment words, with extra neutral words as well.
Revised: Baseline methods generally select some-but-not-all important sentiment words, with extra neutral words as well.

Original: Among baselines, SubgraphXgives more reasonable results.
Revised: Among baselines, SubgraphX gives more reasonable results.

Original: However, it cannot cover two groups of important nodes with a limitedbudget because it can only select a connected subgraph as the explanation; e.g. to cover the negativeword “lameness” in  the second sentence, SubgraphX needs at least 3 more nodes along the way,which will significantly decrease Sparsity while including undesirable, neutral words.
Revised: However, it cannot cover two groups of important nodes with a limited budget because it can only select a connected subgraph as the explanation; e.g. is the should you lameness on bad in . how the title ’s movie clue is the should you lameness on bad in . how the title ’s movie clue is the should you lameness on bad in . how the title ’s movie clue is the should you lameness on bad in .

Revised: how the title ’s movie clue is the should you lameness on bad in .

Revised: how the title ’s movie clue is the should you lameness on bad in .

Revised: how the title ’s movie clue

Revised: GNNExplainer

Revised: PGExplainer

Revised: SubgraphX

Revised: GraphSVX “is still quite good – natured and not a bad way to spend an hour” “the title 's lameness should clue you in on how bad the movie is.”

Revised: OrphicX

Revised: GStarX

Revised: [Figure]

Revised: an is quite a

Revised: [Figure]

Revised: spendgood

Revised: [Figure]

Revised: still natured and not hour to an is quite a

Revised: [Figure]

Revised: [Figure]

Revised: still natured

Revised: [Figure]

Revised: not hour to an is quite a

Revised: [Figure]

Revised: to an is quite a

Revised: [Figure]

Revised: not

Revised: [Figure]

Revised: to an is

Revised: [Figure]

Revised: a

Revised: [Figure]

Revised: to an is quite

Revised: [Figure]

Revised: [Figure]

Revised: “Ground

Revised: [Figure]

Revised: to cover the negative word “lameness” in the lower sentence, SubgraphX needs at least three more nodes along the way, which will significantly decrease Sparsity while including undesirable, neutral words.

Revised: Moreover, we discussed in Section 3.2 that the Shapley value will downgrade the positive importance of the word “good” for the upper sentence.

Revised: Comparing the normalized contribution scores of our HN-value-based method GStarX and the Shapley-based method GraphSVX, contribution of “good” is higher in ours: 0.1152 vs. 0.0371.

Original: We visualize explanations selected with high and comparable Sparsity of a mutagenic moleculefrom MUTAG in Figure 3.
Revised: We visualize explanations selected with high and comparable Sparsity of a mutagenic molecule from MUTAG in Figure 3.

Original: Explanations on chemical graphs are harder to evaluate than text graphsas they require domain knowledge.
Revised: Explanations on chemical graphs are harder to evaluate than text graphs as they require domain knowledge.

Original: MUTAG has been widely used as a benchmark for evaluating
Revised: MUTAG has been widely used as a benchmark for evaluating

Original: GNN explanations because human experts recognize -NO2 as mutagenic [17], which makes MUTAGa dataset with “ground truth” 2 .
Revised: GNN explanations because human experts recognize -NO2 as mutagenic [8], which makes MUTAG a dataset with “ground truth” 3 .

Original: Surprisingly, we found that GStarX generates much better HFidelity / Fidelity / Inv-Fidelity than other methods and even the “ground truth” by only selecting the-O in -NO2 as explanations.
Revised: Surprisingly, we found that GStarX generates much better HFidelity / Fidelity / Inv-Fidelity than other methods and even the “ground truth” by only selecting the -O in -NO2 as explanations.

Original: In particular, the -0.234 Inv-Fidelity of GStarX means the selectedsubgraph has an even better prediction result than the original whole graph (0 Inv-Fidelity ) and theground truth (-0.143 Inv-Fidelity ) because nodes not significant to the GNN prediction are removed.
Revised: In particular, the -0.234 Inv-Fidelity of GStarX means the selected subgraph has an even better prediction result than the original whole graph (0 Inv-Fidelity ) and the ground truth (-0.143 Inv-Fidelity ) because nodes not significant to the GNN prediction are removed.

Original: Fidelity metrics of baselines are inferior to GStarX because they include other non-discriminativecarbon atoms despite they capture -NO2 to some extent.
Revised: Fidelity metrics of baselines are inferior to GStarX because they include other non-discriminative carbon atoms despite they capture -NO2 to some extent.

Original: This suggests that even though humanexperts identify -NO2 as the “ground truth” of mutagenicity, the GNN only needs -O to classifymutagenic molecules.
Revised: This suggests that even though human experts identify -NO2 as the “ground truth” of mutagenicity, the GNN only needs -O to classify mutagenic molecules.

Original: With the goal being understand model behavior, GStarX explanation is better.
Revised: With the goal being understand model behavior, GStarX explanation is better.

Original: Moreover, SubgraphX is the only baseline that has better H-Fidelity than the “ground truth”, but itcan only capture one -NO2 because its search algorithm requires the explanation to be connected,so its Inv-Fidelity is not optimal.
Revised: Moreover, SubgraphX is the only baseline that has better H-Fidelity than the “ground truth”, but it [Table] [Table] can only capture one -NO2 because its search algorithm requires the explanation to be connected, so its Inv-Fidelity is not optimal.

Original: In fact, GNNExplainer, PGExplainer, and SubgraphX can nevergenerate explanations including only disconnected -O without -N like GStarX, because the formertwo solve the explanation problem by optimizing edges (as opposed to Equation 4), and the latterrequires connectedness.
Revised: In fact, GNNExplainer, PGExplainer [Table] and SubgraphX can never generate explanations including only disconnected -O without -N like GStarX, because the former two solve the explanation problem by optimizing edges (as opposed to Equation 4), and the latter requires connectedness.

Original: More MUTAG explanation visualizations are in Appendix H.
Revised: More MUTAG explanation visualizations are in Appendix H.

Original: 5.4 Model-agnostic explanation
Revised: 5.3 Ablation study and analysis

Revised: Model-agnostic explanation.

Original: GStarX makes no assumptions about the model architecture and can be applied to  various GNNbackbones.
Revised: GStarX makes no assumptions about the model architecture and can be applied to explain various GNN backbones.

Original: We use GCN for all datasets in our major experiment  for consistency, and  now furtherinvestigate performance on two more popular GNNs: GIN and GAT.
Revised: We use GCN for all datasets in the major experiment above for consistency, and we now further investigate performance on two more popular GNNs: GIN and GAT.

Original: We follow [5] to train GIN on MUTAG and GAT on GraphSST2 3 , and show results in Table 2.
Revised: We follow [44] to train GIN on MUTAG and GAT on GraphSST2 4 , and show results in Table 2.

Original: For both settings, GStarX outperformsthe baselines, which is consistent with results on GCN.
Revised: For both settings, GStarX outperforms the baselines, which is consistent with results on GCN.

Original: 5.5 Efficiency study
Revised: Efficiency study.

Original: The GStarX algorithm scales in O ( J ) with practical J ∝ |V| .
Revised: The GStarX algorithm scales in O ( J ) with practical J ∝ |V| .

Original: Following [5], we study the empiricalefficiency of GStarX by explaining 50 randomly selected graphs from BBBP .
Revised: Following [44], we study the empirical efficiency of GStarX by explaining 50 randomly selected graphs from BBBP .

Original: We report the average run

Original: As some take we analysis.

Original: [Table] [Table] time in Table 3.
Revised: We report the average run time in Table 3.

Original: Our result on the baselines is similar to [5].
Revised: Our results for the baselines are similar to [44].

Original: GStarX is not the fastest method, but it ismore than two times faster than SubgraphX. Since explanation usually doesn’t have strict efficiencyrequirements in real applications, considering GStarX generates higher-quality explanations than thebaselines, we believe the time complexity of GStarX is acceptable. 
Revised: GStarX is not the fastest method, but it is more than two times faster than SubgraphX. Since explanation usually doesn’t have strict efficiency requirements in real applications, considering GStarX generates higherquality explanations than the baselines, we believe the time complexity of GStarX is acceptable. Explanation sparsity study.

Revised: To further study whether the obtained scores by GStarX are sparse, we follow [11] to evaluate an entropy-based sparsity measure on model output scores.

Revised: We show the average GStarX entropy-based sparsity on all datasets, and compare them with three reference score distributions on all n nodes in a graph. 1)

Original: Related work
Revised: An upper bound: Uniform(n), which represents the least sparse output. A practical lower bound: Uniform(0.25*n) which represents very sparse outputs with only top 25% of nodes. 3) Poisson(0.25*n), which is a more realistic version of case 2). Results in Table 4 show the average entropy-based sparsity of GStarX is much lower than Uniform(n) and close to Poisson(0.25*n), which justifies the GStarX outputs are indeed sparse.

Revised: A more detailed discussion of this metric and these three reference distributions is in Appendix A.5.

Original: GNN explanation aims to produce an explanation for  GNN predictions on a given graph, usually asa subgraph induced by important nodes or edges.
Revised: GNN explanation aims to produce an explanation for a GNN prediction on a given graph, usually as a subgraph induced by important nodes or edges.

Original: Many existing methods work by scoring nodes oredges and are thus similar to this work.
Revised: Many existing methods work by scoring nodes or edges and are thus similar to this work.

Original: For example, the scoring function of GNNExplainer [14] isthe mutual information between a masked graph and the prediction on the original graph, where a softmask on edges and node features is generated by direct parameter learning.
Revised: For example, the scoring function of GNNExplainer [41] is the mutual information between a masked graph and the prediction on the original graph, where soft masks on edges and node features are generated by direct parameter learning.

Original: PGExplainer [15] usesthe same scoring function  but generates a discrete mask on edges by training an edge mask predictor.
Revised: PGExplainer [25] uses the same scoring function as [41] but generates a discrete mask on edges by training an edge mask predictor.

Original: SubgraphX [5] uses the Shapley value as its scoring function on subgraphs selected by Monte Carlo
Revised: SubgraphX [44] uses the Shapley value as its scoring function on subgraphs [Table] [Table]

Original:  Tree Search (MCTS), and GraphSVX [4] uses a least-square approximation to the Shapley value toscore nodes and their features.
Revised: selected by Monte Carlo Tree Search (MCTS) [Table] and GraphSVX [9] uses a least-square approximation to the Shapley value to score nodes and their features.

Original: While SubgraphX and GraphSVX were shown to perform better thanprior alternatives, as we show in Section 3, the Shapley value they try to approximate is non-idealas it is non-structure-aware.
Revised: While SubgraphX and GraphSVX were shown to perform better than prior alternatives [Table] as we show in Section 3, the Shapley value they try to approximate is non-ideal as it is non-structure-aware.

Original: Although SubgraphX and GraphSVX use L -hop subgraphs and thustechnically they use the graph structure, we discuss why this approach has limitations in achievingstructure-awareness in Appendix G. While there are many other GNN explanation methods from verydifferent perspectives, i.e. gradient analysis [25], model decomposition [26], surrogate models [27],and causality [28, 24], we defer their details to Appendix C given their lesser relevance.
Revised: Although SubgraphX and GraphSVX use L -hop subgraphs and thus technically they use the graph structure, such structure usage are very limited in achieving structure-awareness as we show in Appendix G. While there are many other GNN explanation methods from very different perspectives, i [Table] e. gradient analysis [28], model decomposition [1], surrogate models [37] [Table]

Revised: and causality [20, 21], we defer their details to Appendix

Revised: C given their lesser relevance.

Original: Cooperative game theory originally studies how to allocate payoffs among a set of players in acooperative game.
Revised: Cooperative game theory originally studies how to allocate payoffs among a set of players in a cooperative game.

Original: Recently, certain ideas from this domain have been successfully used in featureimportance scoring for ML model explanation [2, 29, 3].
Revised: Recently, certain ideas from this domain have been successfully used in feature importance scoring for ML model explanation [22, 32, 24].

Original: When used for  explanation, the players ofthe game are the features, e.g. pixels for images. The value of the game is allocated to the features asimportance scores in the model explanation.
Revised: When used for model explanation, data features becomes players in the game, e.g. pixels for images, and the value of the game gives feature importance scores.

Original: The vast majority of works in this line, like the ones citedabove, deem the Shapley value [1] to be the only choice.
Revised: The vast majority of works in this line, like the ones cited above, deem the Shapley value [30] to be the only choice.

Original: In fact, there are many other values withdifferent properties and used in different situations in cooperative game theory.
Revised: In fact, there are many other values with different properties and used in different situations in cooperative game theory.

Original: However, to the bestof our knowledge, only [30] mentions the Myerson value [9] in the context of proposing a connected Shapley (C-Shapley) value for explaining sequence data, so it is not directly comparable to ours.
Revised: However, to the best of our knowledge, only [4] mentions the Myerson value [26] in the context of proposing a connected Shapley (C-Shapley) value for explaining sequence data, and it is not directly comparable to ours for graph data.

Original: Adetailed discussion of the Myerson value and the C-Shapley value can be found in Appendix F. Inthis work, we follow the cooperative game theory approach to explain models on graph data using anew HN value [6]. We showit is a better choice than the Shapley value as it is structure-aware.
Revised: A detailed discussion of the Myerson value and the C-Shapley value can be found in Appendix F. Our work follows the cooperative game theory approach to explain models on graph data using the HN value [15], which as we show is a better choice than the Shapley value given its structure-awareness.

Original:  Conclusion and future work
Revised:  Conclusion and future work

Original: In summary, we study GNN explanation on graph data via feature importance scoring.
Revised: In summary, we study GNN explanation on graphs via node importance scoring.

Original: In particular,we identify the non-structure-aware challenge of existing Shapley-value-based approaches andpropose GStarX to assign importance scores to each node via a structure-aware HN value.
Revised: We identify the non-structure-aware challenge of existing Shapley-value-based approaches and propose GStarX to assign importance scores to each node via a structure-aware HN value.

Original: Webuild connections between thesurplus allocation in HN value  and  message passing in GNNs anddemonstrate the superior performance of GStarX  over strong baselines on chemicalgraph propertyprediction and text graph sentiment classification.
Revised: We also build connections between the HN value surplus allocation and GNN message passing. GStarX demonstrates its superiority over strong baselines on chemical and text graph classifications. A limitation of GStarX is that the importance of different node feature dimensions is not explained.

Revised: One future work is to add this extension, which could be done by scoring a subset of nodes together with a subset of features each time.

Original: As there is a richliterature of cooperative gametheory beyond the Shapley value, more values are possible for explaining ML models.
Revised: Another future direction is to exploit the rich cooperative game theory literature. Beyond the Shapley value, more values are possible for explaining ML models.

Original: For graph data,edge based values  can potentially be applied to an alternative edge-based objective like Equation 4.
Revised: For graph data, edge-based values like [2] can potentially be applied to an alternative edge-based objective like Equation 4.

Original: Other values may be appropriate to more data types beyond graphs.
Revised: Other values may be appropriate to more data types beyond graphs.

Original: We leave these as future work.
Revised: This work was partially supported by NSF III-1705169, NSF 1937599, NSF 2119643, Okawa Foundation Grant, Amazon Research Awards, Cisco research grant USA000EP280889, Picsart Gifts, and Gifts

Original: Checklist
Revised: Checklist

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] (c) Did you discuss any potential negative societal impacts of your work?
Revised: [Yes] (c) Did you discuss any potential negative societal impacts of your work?

Original: [No](d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
Revised: [No] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A] (b) Did you include complete proofs of all theoretical results?
Revised: [N/A] (b) Did you include complete proofs of all theoretical results?

Original: [N/A] 
Revised: [N/A] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: [Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] (c) Did you report error bars (e g., with respect to the random seed after running experiments multiple times)?[Yes]
Revised: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original:  (d) Did you include the total amount of compute and the type of resources used (e g., type of GPUs, internal cluster, or cloud provider)?
Revised: [Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] 
Revised: [Yes] 4.

Original: If you are using existing assets (e g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] (b) Did you mention the license of the assets?[Yes]
Revised: [Yes] (b) Did you mention the license of the assets?

Original:  (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [Yes] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No]
Revised: [No]

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Original: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
Revised: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Original: [N/A] 
Revised: [N/A] 5.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A]

Original: A Experiment details 
Revised: A Experiment details A.1 Dataset statistics

Original: A.1 Datasets

Original: In Table 4, we provided thebasic statistics of  datasets used in our experiments.
Revised: In Table 5, we provided the statistics of all datasets used in our experiments.

Original: [Table]
Revised: [Table]

Original: A.2 Model architectures and implementation
Revised: A.2 Model architectures and implementation

Original: In Table 5, we provided the hyperparameters and test accuracy for the GCN model used in our majorexperiments.
Revised: In Table 6, we provided the hyperparameters and test accuracy for the GCN model used in our major experiments.

Original: In Table 2, we provided the hyperparameters and test accuracy for the GIN and GATmodel used in our analysis experiment.
Revised: In Table 2, we provided the hyperparameters and test accuracy for the GIN and GAT model used in our analysis experiment.

Original: Most parameters are following [5], with small changes tofurther boost the test accuracy.
Revised: Most parameters are following [44], with small changes to further boost the test accuracy.

Original: We run all experiments on a machine with 80 Intel(R)
Revised: We run all experiments on a machine with 80 Intel(R)

Original: Xeon(R) E5-2698 v4 @ 2.20GHz CPUs, anda single NVIDIA V100 GPU with 16GB RAM.
Revised: Xeon(R) E5-2698 v4 @ 2.20GHz CPUs, and a single NVIDIA V100 GPU with 16GB RAM.

Original: Our implementations are based on Python 3.8.10,
Revised: Our implementations are based on Python 3.8.10,

Original: PyTorch 1.10.0, PyTorch-Geometric 1.7.1 [31], and DIG [32].
Revised: PyTorch 1.10.0, PyTorch-Geometric 1.7.1 [10], and DIG [23].

Original: We adapt the GNN implementationand most baseline explainer implementation from the DIG library, except  GraphSVX  where we usethe official implementation.
Revised: We adapt the GNN implementation and most baseline explainer implementation from the DIG library, except for GraphSVX and OrphicX where we adapt the official implementation.

Original: For the baseline hyperparameters, we closely follow the setting in [5]and [4] for a fair comparison.
Revised: For the baseline hyperparameters, we closely follow the setting in [44] and [9] for a fair comparison.

Original: Please refer to [5] Section 4.1 and [4] Appendix E for details.
Revised: Please refer to [44] Section 4.1 and [9] Appendix E for details.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: A.3 Exact formula for evaluation metrics
Revised: A.3 Exact formula for evaluation metrics

Original: We showed the formula of Fidelity , Inv-Fidelity , and Sparsity  in Equation 9, 10, and 11.
Revised: Formulas for Fidelity , Inv-Fidelity , and Sparsity are shown in Equation 9, 10, and 11.

Original: In Equation12, 13, and 14, we show the formula for normalized fidelity ( N-Fidelity ), normalized inverse fidelity( N-Inv-Fidelity ), and harmonic fidelity ( H-Fidelity ).
Revised: In Equation 12, 13, and 14, we show formulas for normalized fidelity ( N-Fidelity ), normalized inverse fidelity ( N-Inv-Fidelity ), and harmonic fidelity ( H-Fidelity ).

Original: Both the N-Fidelity and N-Inv-Fidelity arein [ − 1 , 1] .
Revised: Both the N-Fidelity and N-Inv-Fidelity are in [ − 1 , 1] .

Original: The H-Fidelity flips N-Inv-Fidelity , rescales both values to be in [0 , 1] , and takes theirharmonic mean.
Revised: The H-Fidelity flips N-Inv-Fidelity , rescales both values to be in [0 , 1] , and takes their harmonic mean.

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Let m 1 = N-Fidelity ( G , g ) , m 2 = N-Inv-Fidelity ( G , g )
Revised: Let m 1 = N-Fidelity ( G , g ) , m 2 = N-Inv-Fidelity ( G , g )

Original: H-Fidelity(
Revised: H-Fidelity

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: =
Revised: =

Original: [Equation]

Original: m

Original: [Equation]
Revised: [Equation]

Original: A.4 Fidelity vs. sparsity plots
Revised: A.4 Fidelity vs. sparsity plots

Original: In Table [Figure] we reported the best H-Fidelity among 8 different sparsities for each method.
Revised: In Table 1, we report the best H-Fidelity among 8 different sparsities for each method on each dataset.

Original: We alsofollow [5] to show the Fidelity vs. Sparsity plots in Figure 4.
Revised: We also follow [44] to show the Fidelity vs. Sparsity plots in Figure 4 row1.

Original: Note that GraphSVX tends to give sparseexplanations on some datasets, we still pick 8 different sparsities for it but mostly on the higher end
Revised: Note that GraphSVX tends to give sparse explanations on some datasets, we still pick 8 different sparsities for it but mostly on the higher end.

Revised: We also show the 1 - Inv-Fidelity vs. sparsity plots and the H-Fidelity vs. sparsity plots.

Revised: Curves in all three plots are the higher the better.

Original: [Figure]
Revised: [Figure]

Original: corresponding to the results shown in Table 1.
Revised: corresponding to the results shown in Table 1.

Original:  three metrics are the higher the better.
Revised: All three metrics are the higher the better.

Original: We see that
Revised: We see that

Original: GStarX outperforms the other methods
Revised: GStarX outperforms the other methods

Revised: A.5 Detailed entropy-based sparsity evaluation

Revised: In Section 5.2 we study whether the obtained scores by GStarX are sparse and follow [11] to apply an entropy-based sparsity measure on scores.

Revised: We now provide a more detailed discussion of this study.

Revised: The entropy-based sparsity, as defined in Definition 2 in [11], is shown in the Equation 15 below.

Revised: Here ϕ is the model output scores for a data instance, and ˜ ϕ

Original: B GStarX for node classification
Revised: [Equation] scores [Equation] The entropy-based sparsity helps us to understand how sparse an explanation is, before the scores are turned into hard explanation by thresholding or selecting top k. In Table 4, we show the average scores for GStarX on all datasets, and compare them with three reference cases.

Revised: The entropy of uniform distribution over all n nodes in a graph, i.e., Uniform(n), which represents the least sparse output and is an upper bound of entropy-based sparsity. 2)

Revised: The entropy of uniform distribution over the top 25% nodes in a graph, i.e., Uniform(0.25*n), where probabilities of the bottom 75% nodes are set to zero.

Revised: This case is very sparse since 75% of nodes are deterministically excluded, which can be treated as a practical lower bound of entropy-based sparsity.

Revised: The entropy of Poisson distribution with mean 0.25*n, i.e. Poisson(0.25*n), which is a more realistic version of the sparse output in case 2).

Revised: Instead of setting all 75% of nodes to have probability zero, we assume the probabilities for tail nodes decrease exponentially as a Poisson distribution while the mean is kept the same as in case 2).

Revised: Results in Table 4 show that the average entropy-based sparsity of GStarX is between

Revised: Uniform(0.25*n) and Uniform(n) and close to Poisson(0.25*n), which justifies the GStarX outputs are indeed sparse.

Revised: B GStarX for node classification

Original: Even  the GStarX algorithm is stated for graph classification, it works for node classification as well.
Revised: Even though the GStarX algorithm is stated for graph classification, it works for node classification as well.

Original: This can be easily seen as the GNN node classification can be covert to classify an ego-graph.
Revised: This can be easily seen as the GNN node classification can be covert to classify an ego-graph.

Original: Givena graph G with V = { u 1 , . . .
Revised: Given a graph G with V = { u 1 , . . .

Original: , u n } .
Revised: , u n } .

Original: Node classification on u i with an L -layer GNN can be convertedto a graph classification.
Revised: Node classification on u i with an L -layer GNN can be converted to a graph classification.

Original: The target graph to classify will be the L -hop ego-graph centered at u i ,because this is the receptive field of the GNN for classifying u
Revised: The target graph to classify will be the L -hop ego-graph centered at u i , because this is the receptive field of the GNN for classifying u

Original: i and nodes further away won’t haveinfluence on the result.
Revised: i and nodes further away won’t influence the result.

Original: The label of the graph will be the label of u i .
Revised: The label of the graph will be the label of u i .

Original: In this case, the final readoutlayer of the GNN will be indexing u
Revised: In this case, the final readout layer of the GNN will be indexing u

Original: i instead of pooling.
Revised: i instead of pooling.

Original: Given this kind of conversion, everythingwe showed in Section 4 follows.
Revised: Given this kind of conversion, everything we showed in Section 4 follows.

Original: We conduct experiments on a standard node classification classification benchmark.

Original: With datasetdescription shown below and results shown in Table 7.

Original: BAShape [14] is a synthetic dataset similar to BA2Motifs but for node classification.

Original: It has one graphcontaining a 300-node BA graph and 80 5-node motifs randomly attached to the BA graph.

Original: Nodefeatures are 10-dimensional all-1 vectors.

Original: Each node in BAShape is labeled with its the membershipof a motif and the location within the motif.

Original: [Table]

Original: C More related work
Revised: C More related work

Original: [Table] GNN explanation continued Besides the perturbation-based method we mentioned in Section6, there are several other type of approaches for GNN explanation.
Revised: GNN explanation continued Besides the perturbation-based method we mentioned in Section 6, there are several other types of approaches for GNN explanation.

Original: Gradient-based methods arewidely used for explaining ML models on images and text.
Revised: Gradient-based methods are widely used for explaining ML models on images and text.

Original: The key idea is to use the gradientsas the approximations of input importance.
Revised: The key idea is to use the gradients as the approximations of input importance.

Original: Such methods like contrastive gradient-based (CG)saliency maps, Class Activation Mapping (CAM), and gradient-weighted CAM (Grad-CAM) havebeen generalized to graph data in [25].
Revised: Such methods as contrastive gradient-based (CG) saliency maps, Class Activation Mapping (CAM), and gradient-weighted CAM (Grad-CAM) have been generalized to graph data in [28].

Original: Decomposition based methods are a popular way to explaindeep NNs for images.
Revised: Decomposition-based methods are a popular way to explain deep NNs for images.

Original: They measure the importance of input features by decomposing the modelpredictions and regard the decomposed terms as importance scores.
Revised: They measure the importance of input features by decomposing the model predictions and regard the decomposed terms as importance scores.

Original: Decomposition methods including 
Revised: Decomposition methods including Layer-wise

Original: Layer-wise Relevance Propagation (LRP) and Excitation Backpropagation (EB) have also beenextended to graphs [25, 26].
Revised: Relevance Propagation (LRP) and Excitation Backpropagation (EB) have also been extended to graphs [28, 1].

Original: Surrogate-based methods work by approximating a complex modelusing an explainable model locally.
Revised: Surrogate-based methods work by approximating a complex model using an explainable model locally.

Original: Possible options to approximate GNNs include linear modelas in GraphLIME [33], additive feature attribution model with the Shapley value as in GraphSVX[4], and Bayesian networks as in [27].
Revised: Possible options to approximate GNNs include linear model as in GraphLIME [17], additive feature attribution model with the Shapley value as in GraphSVX [9], and Bayesian networks as in [37].

Original: GNN explainability has also been studied from the causalperspective.
Revised: GNN explainability has also been studied from the causal perspective.

Original: In [28, 24], generative models were constructed to learn causal factors, and explanationswere produced by analyzing the cause-effect relationship in the causal graph.
Revised: In [20, 21], generative models were constructed to learn causal factors, and explanations were produced by analyzing the cause-effect relationship in the causal graph.

Original: D Properties of the Shapley value
Revised: D Properties of the Shapley value

Original: The Shapley value was proposed as the unique solution of a game ( N, v ) that satisfies three propertiesshown below, i.e. efficiency , symmetry , and additivity [1].
Revised: The Shapley value was proposed as the unique solution of a game ( N, v ) that satisfies three properties shown below, i.e. efficiency , symmetry , and additivity [30].

Original: These three properties together arereferred as an axiomatic characterization of the Shapley value.
Revised: These three properties together are referred as an axiomatic characterization of the Shapley value.

Original: The associated consistency propertiesintroduced in Section 4.1 provides a different axiomatic characterization.
Revised: The associated consistency properties introduced in Section 4.1 provides a different axiomatic characterization.

Original: Property D.1 ( Efficiency ) .
Revised: Property D.1 ( Efficiency ) .

Original: [Equation]
Revised: [Equation]

Original: Property D.2 ( Symmetry ) .
Revised: Property D.2 ( Symmetry ) .

Original: If v ( S ∪ { i } )
Revised: If v ( S ∪ { i } )

Original: = v ( S ∪ { j } ) for all S ∈ N \{ i, j } , thenϕ i ( N, v ) = ϕ j ( N, v )
Revised: = v ( S ∪ { j } ) for all S ∈ N \{ i, j } , then [Equation]

Original: Property D.3 ( Additivity ) .
Revised: Property D.3 ( Additivity ) .

Original: Given two games ( N, v ) and ( N, w ) ,
Revised: Given two games ( N, v ) and ( N, w ) ,

Original: [Equation]
Revised: [Equation]

Original: The efficiency property states that the value should fully distribute the payoff  the game.
Revised: The efficiency property states that the value should fully distribute the payoff of the game.

Original: The symmetryproperty states that if two players make equal contributions to all possible coalitions formed by otherplayers (including the empty coalition), then they should have the same value.
Revised: The symmetry property states that if two players make equal contributions to all possible coalitions formed by other players (including the empty coalition), then they should have the same value.

Original: The additivity propertystates that the value of two independent games should be added player by player.
Revised: The additivity property states that the value of two independent games should be added player by player.

Original: It is the most usefulfor a system of independent games.
Revised: It is the most useful for a system of independent games.

Original: E More about the HN valueE.1 Consistency and associated games
Revised: E Properties and calculation of the HN value E.1 Consistency and associated games

Original: One reason for the Shapley value’s popularity is its axiomatic characterization , indicating that it isthe unique solution which satisfies a set of desirable properties (see Appendix D).
Revised: One reason for the Shapley value’s popularity is its axiomatic characterization , indicating that it is the unique solution that satisfies a set of desirable properties (see Appendix D).

Original: Then [34] proposeda new axiomatic characterization of the Shapley value based on a different associated consistencyproperty.
Revised: Then [14] proposed a new axiomatic characterization of the Shapley value based on a different associated consistency property.

Original: The consistency property is a common analysis tool used in game theory [35, 36, 37, 38].
Revised: The consistency property is a common analysis tool used in game theory [16, 7, 31, 27].

Original: The idea is to analyze a game ( N, v ) by defining other reduced games ( S, v S ) for S ⊆ N , and asolution function ϕ is called consistent when ϕ ( N, v ) yields the same payoff as ϕ ( S, v S ) on each S .
Revised: The idea is to analyze a game ( N, v ) by defining other reduced games ( S, v S ) for S ⊆ N , and a solution function ϕ is called consistent when ϕ ( N, v ) yields the same payoff as ϕ ( S, v S ) on each S .

Original: When ( S, v S ) is defined with desired properties, these good properties can be enforced for a solution by requiring consistency.
Revised: When ( S, v S ) is defined with desired properties, these good properties can be enforced for a solution by requiring consistency.

Original: The associated consistency in [34] is a special case of consistency between( N, v ) and only one other game ( N, v ∗ ) , which is called the associated game .
Revised: The associated consistency in [14] is a special case of consistency between ( N, v ) and only one other game ( N, v ∗ ) , which is called the associated game .

Original: [34] shows that acarefully designed associated game uniquely characterizes the Shapley value.
Revised: [14] shows that a carefully designed associated game uniquely characterizes the Shapley value.

Original: Associated consistencyis also the key idea of the HN value.
Revised: Associated consistency is also the key idea of the HN value.

Original: E.2 Limit game and the axiomatic characterization
Revised: E.2 Limit game and the axiomatic characterization

Original: The HN value is established on an special associated game as we discussed in Section 4.1.
Revised: The HN value is established on a special associated game as we discussed in Section 4.1.

Original: We canactually write this associated game in a more compact matrix form, where we slightly abuse notationand use v and v ∗ τ to represent vectors of payoffs for all S ⊆ N under the original and associatedgame respectively.
Revised: We can actually write this associated game in a more compact matrix form, where we slightly abuse notation and use v and v ∗ τ to represent vectors of payoffs for all S ⊆ N under the original and associated game respectively.

Original: In other words, v ( S ) , which  used to represent evaluating the coalition S using thecharacteristic function v , now can also be interpreted as indexing the vector v with index S .
Revised: In other words, v ( S ) , which is used to represent evaluating the coalition S using the characteristic function v , now can also be interpreted as indexing the vector v with index S .

Original: Lemma E.1.
Revised: Lemma E.1.

Original: A matrix form of the associated game ( N, v ∗ τ , G ) is given by
Revised: A matrix form of the associated game ( N, v ∗ τ , G ) is given by

Original: [Equation]
Revised: [Equation]

Original: The matrix H { τ,n, G} depends on the hyperparameter τ , number of players n , and the graph G .
Revised: The matrix H { τ,n, G} depends on the hyperparameter τ , number of players n , and the graph G .

Original: Whenthese variables are clear from the context, we drop them and write v ∗ τ
Revised: When these variables are clear from the context, we drop them and write v ∗ τ

Original: = H v .
Revised: = H v .

Original: Please refer to [6] forthe proof of Lemma E.1.
Revised: Please refer to [15] for the proof of Lemma E.1.

Original: With the matrix form, we can define the limit game.
Revised: With the matrix form, we can define the limit game.

Original: Definition E.2.
Revised: Definition E.2.

Original: Given a game ( N, v, G ) , its limit game ( N, ˜ v, G ) is defined by
Revised: Given a game ( N, v, G ) , its limit game ( N, ˜ v, G ) is defined by

Original: [Equation]
Revised: [Equation]

Original: Notice that although the matrix H is constructed from the associated game and depends on τ , thepowers of H actually converges to a limit independent from τ , when τ is sufficiently small.
Revised: Notice that although the matrix H is constructed from the associated game and depends on τ , the powers of H actually converge to a limit independent from τ , when τ is sufficiently small.

Original: Thegeneral condition depends on the actual graph, but 0 < τ < 2 n is proven to be sufficient for thecomplete graph case [34].
Revised: The general condition depends on the actual graph, but 0 < τ < 2 n is proven to be sufficient for the complete graph case [14].

Original: As we discussed in Section 4.1, the limit game can be seen as constructingassociated games repeatedly until the characteristic function converges.
Revised: As we discussed in Section 4.1, the limit game can be seen as constructing associated games repeatedly until the characteristic function converges.

Original: An axiomatic characterization of the HN value regarding its uniqueness is given by the followingtheorem based on the limit game.
Revised: An axiomatic characterization of the HN value regarding its uniqueness is given by the following theorem based on the limit game.

Original: The associated consistency is the core property related to this work.
Revised: The associated consistency is the core property related to this work.

Original: We encourage the readers to check [6] for the other two properties.
Revised: We encourage the readers to check [15] for the other two properties.

Original: Theorem E.3.
Revised: Theorem E.3.

Original: There exists a unique solution ϕ that verifies the associated consistency, i.e.ϕ i ( N, v, G ) =
Revised: There exists a unique solution ϕ that verifies the associated consistency, i.e. ϕ i ( N, v, G ) =

Original: ϕ
Revised: ϕ

Original: i ( N, v ∗ τ , G ) , inessential game, and continuity.
Revised: i ( N, v ∗ τ , G ) , inessential game, and continuity.

Original: ϕ is given byϕ i ( N, v, G ) =
Revised: ϕ is given by

Original: ˜ v ( { i } ) (17)
Revised: [Equation]

Original: E.3 The algorithm for computing the HN value
Revised: E.3 The algorithm for computing the HN value

Original: We show the algorithm for Compute-HN-Values (Algorithm 2) and Compute-HN-Values-MC (Algorithm 3) mentioned in Section 4.3.
Revised: We show the algorithm for Compute-HN-MC (Algorithm 3) mentioned in Section 4.3.

Original: The algorithm is a combination of Equation15, 16, and8.
Revised: The algorithm is a combination of Equation 16, 17, and 8.

Revised: F The Myerson value and the C-Shapley value F.1 The Myerson value

Revised: In the study of cooperative games, [26] proposed to characterize the cooperation possibilities between players using a graph structure G , which leads to the communication structure introduced in Section

Original: Input: Graph instance G with nodes V = { u 1 , . . . , u n } , characteristic function v , hyperparameter τ , maximum sample size m , number of samples J
Revised: Input: Graph instance G with nodes V = { u 1 , . . . , u n } , characteristic function v , hyperparameter τ , maximum sample size m , number of samples J

Original: Let ψ 1 , . .
Revised: Let ψ 1 , . .

Original: . , ψ n be n empty lists for j = 1 to J do
Revised: . , ψ n be n empty lists for j = 1 to J do

Original: Sample g S j from G s.t.
Revised: Sample g S j from G s.t.

Original: S
Revised: S

Original: j = { u j 1 , . . . , u j l } and l < m ϕ j = Compute-HN-Values ( g S j , S j , v ( · ) , τ ) for k = 1 to l do
Revised: j = { u j 1 , . . . , u j l } and l < m ϕ j = Compute-HN ( g S j , S j , v ( · ) , τ ) for k = 1 to l do

Original: Append ϕ j k to ψ j k end for end for
Revised: Append ϕ jk to ψ j k end for end for

Original: Set ϕ
Revised: Set ϕ

Original: i to be the mean of ψ
Revised: i to be the mean of ψ

Original: i
Revised: i

Original: Return: ϕ Input: Graph instance G with nodes V = { u 1 , . . . , u n } , characteristic function v , hyperparameter τ for S in 2 N do Compute payoff v ( S ) {Equation 8} end for Construct matrix H from τ , n , and G {Equation 15} repeat
Revised: Return: ϕ 2.2 and the Myerson value as a solution for this special type of games ( N, v, G ) .

Original: H = HH until H converges

Original: Get the limit game ˜ v

Original: = H v e {Equation 16}

Original: Assign the first n entries of ˜ v to ϕ

Original: Return: ϕ

Original: F The myerson value and the C-Shapley valueF.1 The Myerson value

Original: In the study of cooperative games, [9] proposed to characterize the cooperation possibilities betweenplayers using a graph structure G , which leads to the communication structure introduced in Section2.2 and the Myerson value as a solution for this special type of games ( N, v, G ) .

Original: The Myerson valueis closely related to the Shapley value.
Revised: The Myerson value is closely related to the Shapley value.

Original: In fact, it is the Shapley value on a transformed game whereplayers are partitioned by the graph.
Revised: In fact, it is the Shapley value on a transformed game where players are partitioned by the graph.

Original: We now formally introduce the partition and the transformedgame.
Revised: We now formally introduce the partition and the transformed game.

Original: Definition F.1 ( Partition ) .
Revised: Definition F.1 ( Partition ) .

Original: Given a set of players N and a graph G .
Revised: Given a set of players N and a graph G .

Original: For any coalition S ⊆ N , thepartition of S is denoted by S/ G and defined by
Revised: For any coalition S ⊆ N , the partition of S is denoted by S/ G and defined by

Original: S/ G = {{ i | i and j are connected in S by G}| j ∈ S } and a member of the set S/ G is called a component of S .
Revised: S/ G = {{ i | i and j are connected in S by G}| j ∈ S } and a member of the set S/ G is called a component of S .

Original: Definition F.2 ( Transformed Game ) .
Revised: Definition F.2 ( Transformed Game ) .

Original: Given a game ( N, v, G ) , we can transform it to a new gamev/ G such that for all S ⊆ N
Revised: Given a game ( N, v, G ) , we can transform it to a new game v/ G such that for all S ⊆ N

Original: [Equation]
Revised: [Equation]

Original: Intuitively, given a coalition S , the transformed game treats each connected component of S asindependent, evaluates them separately, and sums their payoff as the payoff of S .
Revised: Intuitively, given a coalition S , the transformed game treats each connected component of S as independent, evaluates them separately, and sums their payoff as the payoff of S .

Original: The Shapley value has a axiomatic characterization that uniquely determines it as we introduced in Appendix D. Likewise, the Myerson value was proposed to be a unique solution that satisfies thecomponent efficiency and the fairness property defined below.
Revised: The Shapley value has an axiomatic characterization that uniquely determines it as we introduced in Appendix D. Likewise, the Myerson value was proposed to be a unique solution that satisfies the component efficiency and the fairness property defined below.

Original: Property F.3 ( Component Efficiency ) .
Revised: Property F.3 ( Component Efficiency ) .

Original: For a game ( N, v, G ) and any connected component S ∈
Revised: For a game ( N, v, G ) and any connected component S ∈

Original: N/ G , a solution is component efficient if
Revised: N/ G , a solution is component efficient if

Original: [Equation]
Revised: [Equation]

Original: Property F.4 ( Fairness ) .
Revised: Property F.4 ( Fairness ) .

Original: For a game ( N, v, G ) and any edge ( i, j ) in G , let ˜ G be G with the edge( i, j ) removed, a solution is fair if
Revised: For a game ( N, v, G ) and any edge ( i, j ) in G , let ˜ G be G with the edge ( i, j ) removed, a solution is fair if

Original: [Equation]
Revised: [Equation]

Original: The component efficiency property is an extension of the regular efficiency property to games with acommunication structure.
Revised: The component efficiency property is an extension of the regular efficiency property to games with a communication structure.

Original: It requires efficiency to hold for each disconnected piece because thesepieces are assumed as independent from each other.
Revised: It requires efficiency to hold for each disconnected piece because these pieces are assumed as independent from each other.

Original: The fairness property states that if breaking anedge ( i, j ) changes the value of player i , then the value of player j should be changed by the sameamount.
Revised: The fairness property states that if breaking an edge ( i, j ) changes the value of player i , then the value of player j should be changed by the same amount.

Original: Theorem F.5 ( Myerson Value ) .
Revised: Theorem F.5 ( Myerson Value ) .

Original: There exists a unique solution ϕ of game ( N, v, G ) satisfyingcomponent efficiency and fairness.
Revised: There exists a unique solution ϕ of game ( N, v, G ) satisfying component efficiency and fairness.

Original: With ˜ ϕ represents the Shapley value, the solution is given by theformulaϕ ( N, v, [Equation] ( N [Equation]
Revised: With ˜ ϕ represents the Shapley value, the solution is given by the formula

Original: ( v/ G

Original: [Equation]
Revised: [Equation]

Original: For games with a communication structure, the Myerson value is a better choice than the Shapleyvalue as it uses the graph structure.
Revised: For games with a communication structure, the Myerson value is a better choice than the Shapley value as it uses the graph structure.

Original: However, it also suffer from some criticisms.
Revised: However, it also suffers from some criticisms.

Original: For example, thefairness assumption may not be realistic.
Revised: For example, the fairness assumption may not be realistic.

Original: When an existing edge is broken, the value changes forplayers on the two edge ends can be asymmetric.
Revised: When an existing edge is broken, the value changes for players on the two edge ends can be asymmetric.

Original: Intuitively, if the edge connects a popular hubplayer i to a leaf player j , then the change of i can be less significant than j since j becomes isolatedwhen ( i, j ) is removed.
Revised: Intuitively, if the edge connects a popular hub player i to a leaf player j , then the change of i can be less significant than j since j becomes isolated when ( i, j ) is removed.

Original: This is also the case when the game value is used for model explanation.
Revised: This is also the case when the game value is used for model explanation.

Original: Forexample in Figure 1 (b), when the edge ("good", "quite") is broken, the value of "quite" should changea lot.
Revised: For example in Figure 1 (b), when the edge ("good", "quite") is broken, the value of "quite" should change a lot.

Original: It used to contribute positively together with "good", and thus gets some payoff allocation, butit now becomes an isolated node, which is neutral by itself.
Revised: It used to contribute positively together with "good", and thus gets some payoff allocation, but it now becomes an isolated node, which is neutral by itself.

Original: On the other hand, the word "good" canstill contribute positively by itself and interact with other nodes through its other edges, and thus itsvalue shouldn’t change too much.
Revised: On the other hand, the word "good" can still contribute positively by itself and interact with other nodes through its other edges, and thus its value shouldn’t change too much.

Original: Because of such criticisms, we choose to use the HN value as ourscoring function, which characterize the value by associated consistency rather than fairness.
Revised: Because of such criticisms, we choose to use the HN value as our scoring function, which characterizes the value by associated consistency rather than fairness.

Original: F.2 The C_Shapley value
Revised: F.2 The C-Shapley value

Original: The Myerson value was also mentioned in [30] for  model explanation on text, where the C_Shapleyvalue was proposed  an approximation of the Shapley value, and it was claimed to be equal tothe Myerson value.
Revised: The Myerson value was also mentioned in [4] for the model explanation on text, where the C-Shapley value was proposed as an approximation of the Shapley value, and it was claimed to be equal to the Myerson value.

Original: We have discussed why Shapley value and Myerson are not ideal choice forexplaining graph data in Section 3 and Appendix F.1.
Revised: We have discussed why Shapley value and Myerson are not-ideal choices for explaining graph data in Section 3 and Appendix F.1.

Original: These are partially the reason why our HNvalue based method is better than the C_Shapley value.
Revised: These are partially the reason why our HNvalue-based method is better than the C-Shapley value.

Original: However, the major reason why we don’t dodirect comparison to the C_Shapley value as a baseline is because its formula only works for linegraphs like sequence data, and not even all nodes in line graphs.
Revised: However, the major reason why we don’t do a direct comparison to the C-Shapley value as a baseline is that its formula only works for line graphs like sequence data, and not even all nodes in line graphs.

Original: In contrast, our target task is generalgraph prediction for graph with possibly complicated topological structures.
Revised: In contrast, our target task is general graph prediction for graphs with possibly complicated topological structures.

Original: We now clarify a mistake of the C_Shapley value formula and explain why it won’t work for generalgraphs.
Revised: We now clarify a mistake of the C-Shapley value formula and explain why it won’t work for general graphs.

Original: The notations are following the [30], where d is the number of players corresponding to n inour notation, and [ d ] corresponding to N .
Revised: The notations are following the [4], where d is the number of players corresponding to n in our notation, and [ d ] corresponding to N .

Original: The formula for the C_Shapley value is given in Equation 6 in Definition 2 in the paper, and it isstated for "a graph G" without mentioning any assumptions of the graph.
Revised: The formula for the C-Shapley value is given in Equation 6 in Definition 2 in the paper, and it is stated for "a graph G" without mentioning any assumptions of the graph.

Original: However, from the proof ofthis formula in Appendix B.2 in the paper, the line graph assumption can be seen in two places.
Revised: However, from the proof of this formula in Appendix B.2 in the paper, the line graph assumption can be seen in two places.

Original: Thefirst place is Equation 20, where the set C is explicitly defined only for subsequences.
Revised: The first place is Equation 20, where the set C is explicitly defined only for subsequences.

Original: The secondplace is Equation 22, the first line converts (cid:80) A : U S ( A )
Revised: The second place is Equation 22, the first line converts (cid:80) A : U S ( A )

Original: =
Revised: =

Original: U to (cid:80) d
Revised: U to (cid:80) d

Original: −| U |− 2 i = , which is implicitly saying
Revised: −| U |− 2 i = , which is implicitly saying

Original: V S ( A ) can be picked from all d but | U | − 2 nodes.
Revised: V S ( A ) can be picked from all d but | U | − 2 nodes.

Original: However, this conversion is only possible whenthere are exactly 2 edges between U and [ d ] \ U , i.e. the middle part of a line graph.
Revised: However, this conversion is only possible when there are exactly 2 edges between U and [ d ] \ U , i.e. the middle part of a line graph.

Original: If there are ledges between U and [ d ] \ U , then the summation should go up to d − | U | − l .
Revised: If there are l edges between U and [ d ] \ U , then the summation should go up to d − | U | − l .

Original: When l = 0 , i.e. Uequals [ d ] or a connected component of [ d ] , no partition is needed and the coefficient simply evaluatesto 1.
Revised: When l = 0 , i.e. U equals [ d ] or a connected component of [ d ] , no partition is needed and the coefficient simply evaluates to 1.

Original: By correcting all these cases, the final formula for the C_Shapley value coefficients of marginalcontributions thus becomes
Revised: By correcting all these cases, the final formula for the C-Shapley value coefficients of marginal contributions thus becomes

Original: [Equation]
Revised: [Equation]

Original: for l > 0 , and 1 for l = 0 .
Revised: for l > 0 , and 1 for l = 0 .

Original: The correct formula for the C_Shapley value of general graphs will be
Revised: The correct formula for the C-Shapley value of general graphs will be

Original: [Equation]
Revised: [Equation]

Original: with l represents the edges between U and [ d ] \ U and C represents all connected subgraphs in [ d ]containing i .
Revised: with l represents the edges between U and [ d ] \ U and C represents all connected subgraphs in [ d ] containing i .

Original: Shapley
Revised: To verify this formula with the 3-node toy graph in Figure 5. When computing the value of node(left), the three connected components containing 0 are C

Original: C_Shapley Myerson HN 1/ 1/ 1/ 1/ 1/ 1/ 1/  1/ 1/ 1/  1/ 1/ 1/ 1/
Revised: = {{ 0 } , { 0 , 1 } , { 0 , 1 , 2 }} . Since 0 is an

Original: Shapley
Revised: Shapley

Original: C_Shapley Myerson HN 1/ 1/ 1/ 1/ 1/ 1/ 1/  1/ 1/ 1/  1/ 1/ 1/ 
Revised: C_Shapley Myerson HN 1/ 1/ 1/ 1/ 1/ 1/ 1/  1/ 1/ 1/  1/ 1/ 1/ 1/

Original: To verify this formula with the 3-node toy graph in Figure 5. When computing the value of node 0(left), the three connected components containing 0 are C
Revised: Shapley

Original: = {{ 0 } , { 0 , 1 } , { 0 , 1 , 2 }} . Since 0 is anend node and has no leaf nodes to its left, l for these three components will be 1, 1, and 0 respectively.
Revised: C_Shapley Myerson HN 1/ 1/ 1/ 1/ 1/ 1/ 1/ 1/ 1/    1/ 1/ 1/ 1/ end node and has no leaf nodes to its left, l for these three components will be 1, 1, and 0 respectively.

Original: According to our new formula in Equation 21, the coefficients will be 12 , 16 , and 13 respectively, withthe disconnected { 0 , 2 } case removed.
Revised: According to our new formula in Equation 22, the coefficients will be 12 , 16 , and 13 respectively, with the disconnected { 0 , 2 } case removed.

Original: This matches the original idea of Myerson value, wherethe { 0 , 2 } − { 2 } case is reduced to the { 0 } − ∅ case, which turns the Shapley coefficients from[ 13 , 16 , 16 , 13 ] to [ 13 + 16 , 16 , 16 − 16 , 13 ] , which is [ 12 , 16 , 0 , 13 ] .
Revised: This matches the original idea of Myerson value, where the { 0 , 2 } − { 2 } case is reduced to the { 0 } − ∅ case, which turns the Shapley coefficients from [ 13 , 16 , 16 , 13 ] to [ 13 + 16 , 16 , 16 − 16 , 13 ] , which is [ 12 , 16 , 0 , 13 ] .

Original: However, the original CShapley formulafrom Equation 6 in the [30] evaluates to [ 13 , 112 , 0 , 130 ] , which doesn’t match the Myerson value andnot even sum up to 1.
Revised: However, the original C-Shapley formula from Equation 6 in the [4] evaluates to [ 13 , 112 , 0 , 130 ] , which doesn’t match the Myerson value and not even sum up to 1.

Original: Another example of computing the value of node 1 is shown in Figure 5 right.
Revised: Another example of computing the value of node 1 is shown in Figure 5 right.

Original: The C_Shapley, even with the correct formula, eventually boils down to an approximation of the
Revised: The C-Shapley, even with the correct formula, eventually boils down to an approximation of the

Original: Shapley value or the Myerson value, which as we discussed are less idea than the HN value.
Revised: Shapley value or the Myerson value, which as we discussed are less ideal than the HN value.

Original: Also,the correct formula in Equation 21 requires generating all possible subgraphs U containing the node iand specify the edges between U and [ d ] \ U .
Revised: Also, the correct formula in Equation 22 requires generating all possible subgraphs U containing the node i and specify the edges between U and [ d ] \ U .

Original: This makes the computation very complicated, we thusskip the comparison to the C_Shapley value.
Revised: This makes the computation very complicated, we thus skip the comparison to the C-Shapley value.

Original: G Use the graph structure via an L-hop cutoff
Revised: G Use the graph structure via an L-hop cutoff

Original: Although the Shapley value itself is not structure-aware, we do note the existing Shapley-valuebased GNN explanation methods uses a L-hop cutoff to help approximate the Shapley value [5, 4].
Revised: Although the Shapley value itself is not structure-aware, we do note the existing Shapley-valuebased GNN explanation methods use an L-hop cutoff to help approximate the Shapley value [44, 9].

Original: Technically, this operation uses the graph structure, so we can’t strictly refer  these explanationmethods as not structure aware.
Revised: Technically, this operation uses the graph structure, so we can’t strictly refer to these explanation methods as not structure-aware.

Original: However, we argue that the L-hop cutoff is a naive way of utilizingthe graph structure.
Revised: However, we argue that the L-hop cutoff is a naive way of utilizing the graph structure.

Original: It has several concerns, and it is not the same structure-aware as the HN value.
Revised: It has several concerns, and it is not the same structure-aware as the HN value.

Original: The L-hop cutoff approximates the Shapley value of node i by considering only the L-hop neighborsof i when explaining a L-layer GNN.
Revised: The L-hop cutoff approximates the Shapley value of node i by considering only the L-hop neighbors of i when explaining an L-layer GNN.

Original: The rationale of this operation is that an L-layer GNNs onlypropagate messages within L-hops so a node more than L-hop away from i has never passed anymessages to i which means no interactions are possible.
Revised: The rationale of this operation is that an L-layer GNNs only propagate messages within L-hops so a node more than L-hop away from i has never passed any messages to i which means no interactions are possible.

Original: In existing Shapley-value-based GNNexplanation methods, this L-hop cutoff operation was meant for reducing the exponentially growingcomputations of the Shapley value, and the ultimate goal is still  compute the Shapley value.
Revised: In existing Shapley-value-based GNN explanation methods, this L-hop cutoff operation was meant for reducing the exponentially growing computations of the Shapley value, and the ultimate goal is still to compute the Shapley value.

Original: The
Revised: The

Original: L-hop cutoff operation has several issues making it a less desirable choice.
Revised: L-hop cutoff operation has several issues making it a less desirable choice.

Original: 1) Even meant to savecomputation, there are still many nodes involved in the computation after applying the L-hop cutoff,since the number of nodes grow exponentially as L grow.
Revised: 1) Even meant to save computation, there are still many nodes involved in the computation after applying the L-hop cutoff since the number of nodes grows exponentially as L grows.

Original: For advanced GNNs, the L can be large.
Revised: For advanced GNNs, the L can be large.

Original: When L is larger than the diameter of the graph, which is actually the case for many recent deep
Revised: When L is larger than the diameter of the graph, which is actually the case for many recent deep

Original: GNNs, the L-hop cutoff is not effective anymore.
Revised: GNNs, the L-hop cutoff is not effective anymore.

Original: When constructing coalitions of nodes within thelocal graph of L-hops, the computation still follows the Shapley value formula.
Revised: When constructing coalitions of nodes within the local graph of L-hops, the computation still follows the Shapley value formula.

Original: This means the usefulgraph structure information among these nodes is forfeited which causes the structure-awarenessconcern of Shapley value as we discussed in Section 3,
Revised: This means the useful graph structure information among these nodes is forfeited which causes the structure-awareness concern of Shapley value as we discussed in Section 3,

Original: H More explanation visualizations
Revised: H More explanation visualizations

Original: Under the same setting as Figure 3, we visualize more explanations in 6. 
Revised: Under the same setting as Figure 3, we visualize more explanations in 6. [Figure] [Figure]

Revised: “occasionally funny , always very colorful and enjoyably overblown in the traditional almodóvar style

Original: [Figure]
Revised: [Figure]

Revised: [Figure]