Original: Explaining Graph Neural Networks with Structure-Aware Cooperative Games
Revised: Explaining Graph Neural Networks with Structure-Aware Cooperative Games

Original: s
Revised: Anonymous Author(s)

Original: Abstract
Revised: Abstract

Original: Explaining  machine learning models is important and have attracted an increasedinterest.
Revised: Explaining predictions made by machine learning models is important and haveattracted an increased interest.

Original: The Shapley value from  game theory has been proposed as a primeapproach to compute feature importance towards predictions, for images, text,tabular data, and recently graph neural networks (GNNs) on graphs.
Revised: The Shapley value from cooperative game theoryhas been proposed as a prime approach to compute feature importance towardspredictions, especially for images, text, tabular data, and recently graph neuralnetworks (GNNs) on graphs.

Original: In this work,we revisit the appropriateness of the Shapley value for GNN explanation, wherethe task is to identify the most important subgraph and constituent nodes for GNNpredictions.
Revised: In this work, we revisit the appropriateness of the Shapley value for GNN explanation, where the task is to identify the most important subgraph and constituent nodes for GNN predictions.

Original: We purport that the Shapley value is a non-ideal choice for graph databecause it is by definition not structure-aware.
Revised: We purport thatthe Shapley value is a non-ideal choice for graph data because it is by definitionnot structure-aware.

Original: We propose a Graph Structure-awareeXplanation (GStarX) method to leverage the critical graph structure information toimprove the explanation.
Revised: We propose a Graph Structure-aware eXplanation (GStarX)method to leverage the critical graph structure information to improve the explanation.

Original: Specifically, we define a scoring function based on a newstructure-aware value from  game theory called the HN value.
Revised: Specifically, we define a scoring function based on a new structure-awarevalue from the cooperative game theory called the HN value.

Original: When used to scorenode importance, the HN value utilizes graph structures to attribute cooperationsurplus between neighbor nodes, resembling message passing in GNNs, so thatnode importance scores reflect not only the node feature importance, but also thestructural roles.
Revised: When used to scorenode importance, the HN value utilizes graph structures to attribute cooperationsurplus between neighbor nodes, resembling message passing in GNNs, so thatnode importance scores reflect not only the node feature importance, but also thestructural roles.

Original: We demonstrate that GStarX produces qualitatively more intuitiveexplanations, and quantitatively improves explanation fidelity over strong baselineson chemical graph property prediction and text graph sentiment classification.
Revised: We demonstrate that GStarX produces qualitatively more intuitiveexplanations, and quantitatively improves explanation fidelity over strong baselineson chemical graph property prediction and text graph sentiment classification.

Original:  Introduction
Revised:  Introduction

Original: Explainability is crucial for complex machine learning (ML) models in sensitive applications, helpingestablish user trust and providing insights for potentialmodel improvements.
Revised: Explainability is crucial for complex machine learning (ML) models in sensitive applications, helpingestablish user trust and providing insights for potential improvements.

Original: Many efforts focuson explaining models on images, text and tabular data.
Revised: Many efforts focus onexplaining models on images, text and tabular data.

Original: In contrast, the explainability of models ongraph data is yet underexplored.
Revised: In contrast, the explainability of models ongraph data is yet underexplored.

Original: Since explainability can be especially critical for many graph taskslike drug discovery and interest in deep graph models is growing rapidly, further investigation ofgraph explainability is warranted.
Revised: Since explainability can be especially critical for many graph taskslike drug discovery and interest in deep graph models is growing rapidly, further investigation ofgraph explainability is warranted.

Original: In this work, we study graph ML explanation with graph neuralnetworks (GNNs) as the target models, given their popularity and widespread use.
Revised: In this work, we study graph ML explanation with graph neuralnetworks (GNNs) as the target models, given their popularity and widespread use.

Original: In ML explainability the Shapley value [1] has been deemed as a “fair” scoring function for computingfeature importance.
Revised: In ML explainability the Shapley value [1] has been deemed as a “fair” scoring function for computingfeature importance.

Original: Originally from  cooperative game theory, many values, including the Shapleyvalue, have been proposed for allocating a total payoff to players in a game.
Revised: Originally from the cooperative game theory, many values, including the Shapleyvalue, have been proposed for allocating a total payoff to players in a game.

Original: When used for scoringthe feature importance of a data instance, the model prediction is treated as the total payoff andthe features are considered as players.
Revised: When used for scoringthe feature importance of a data instance, the model prediction is treated as the total payoff andthe features are considered as players.

Original: In particular, for an instance with n features { x 1 , . . .
Revised: In particular, for an instance with n features { x 1 , . . .

Original: x n } ,the Shapley value of its i th feature x i is computed via aggregating m ( i, S ) , which are the marginalcontributions of x i to sets of other features x S ⊆ { x 1 , . . .
Revised: x n } ,the Shapley value of its i th feature x i is computed via aggregating m ( i, S ) , which are the marginalcontributions of x i to sets of other features x S ⊆ { x 1 , . . .

Original: , x n } \ { x i } .
Revised: , x n } \ { x i } .

Original: Each x S is called a coalition .m
Revised: Each x S is called a coalition .m

Original: ( i, S ) is computed as the difference between model outputs for x S ∪ { x i } and x S , which is meantto capture the interaction between x i and x S .
Revised: ( i, S ) is computed as the difference between model outputs for x S ∪ { x i } and x S , which is meantto capture the interaction between x i and x S .

Original: The Shapley value is widely used for explaining MLmodels on images, text, and tabular data, when the features are pixels, words, and attributes [2, 3].
Revised: The Shapley value is widely used for explaining MLmodels on images, text, and tabular data, when the features are pixels, words, and attributes [2, 3].

Original: “is still quite good – natured and not a bad way to spend an hour” spendgood bad still natured and not hour to an is quite a way 3 1   3  31/ 1/ 1/ 1/ 1/ 1/  1/
Revised: “is still quite good – natured and not a bad way to spend an hour” spendgood bad still natured and not hour to an is quite a way 3 1   3  31/ 1/ 1/ 1/ 1/ 1/  1/

Original: HNweights
Revised: HNweights

Original: [Figure]
Revised: [Figure]

Revised: weights

Original: Marginal Contributions
Revised: Marginal Contributions

Original: The Shapley value has recently been extended to explain GNNs on graphs by feature importancescoring as above, where features are nodes [4] or supernodes [5].
Revised: The Shapley value has recently been extended to explain GNNs on graphs by feature importancescoring as above, where features are nodes [4] or supernodes [5].

Original: We argue that the Shapley valueis a non-ideal choice for (super)node importance scoring because its contribution aggregation isnon-structure-aware .
Revised: We argue that the Shapley valueis a non-ideal choice for (super)node importance scoring because its contribution aggregation isnon-structure-aware .

Original: The Shapley value aggregation assumes no structural relationship between x iand x S even though they are both parts of the input graph (a review of the Shapley value in Section2.2).
Revised: The Shapley value aggregation assumes no structural relationship between x iand x S even though they are both parts of the input graph (a review of the Shapley value in Section2.2).

Original: Since the graph structure generally contains critical information and is crucial to the success of
Revised: Since the graph structure generally contains critical information and is crucial to the success of

Original: GNNs, we consider properly leveraging the structure with a better structure-aware scoring function.
Revised: GNNs, we consider properly leveraging the structure with a better structure-aware scoring function.

Original: We propose Graph Structure-aware eXplanation (GStarX), where we construct a structure-awarenode importance scoring function based on the Hamiache-Navarro (HN) value [6] from cooperativegame theory.
Revised: We propose Graph Structure-aware eXplanation (GStarX), where we construct a structure-aware nodeimportance scoring function based on the HN value [6] from the cooperative game theory.

Original: Recall that GNNs make predictions via message passing (MP), during which noderepresentations are learned by aggregating messages from  neighbors.
Revised: Recall that GNNs make predictions via message passing (MP), during which node representations are updatedby aggregating messages from their neighbors.

Original: MP aggregates both nodefeature information and structure information, resulting in powerful structure-aware models [7].
Revised: MP aggregates both node feature information andstructure information, resulting in powerful structure-aware models [7].

Original: The HN value shares a similar idea to MP by allocating the payoff surplus generated from the cooperationbetween neighboring players (nodes).
Revised: The HN value shares a similaridea to MP by allocating the payoff surplus generated from the cooperation between neighboringplayers (nodes).

Original: When used as a scoring function to explain node importance,the HN value captures both features and structural interactions between nodes (details in Section 4).
Revised: When used as a scoring function to explain node importance, the HN value capturesboth features and structural interactions between nodes (details in Section 4).

Original: Figure 1 (a) shows an example comparing the Shapley value and the HN value.
Revised: An example comparingthe Shapley value and the HN value is shown in Figure 1 (a) .

Original: In this example, theirdifference boils down to different aggregation weights of marginal contributions, where the former isuniform and the latter is structure-aware (details in Section 3.2).
Revised: In this example, their difference boilsdown to different aggregation weights of marginal contributions, where the former is uniform and thelatter is structure-aware (details in Section 3.2).

Original: In summary, our contributions are:
Revised: In summary, we make the following contributions:

Original: • Identify the non-structure-aware limitation of the Shapley value for GNN explanation.
Revised: • Identify the non-structure-aware limitation of the Shapley value for GNN explanation.

Original: • Introduce the structure-aware HN value from  cooperative game theory to the graph machinelearning community and connect it to the GNN message passing and GNN Explanation.•
Revised: • Introduce the structure-aware HN value from the cooperative game theory to the graph machinelearning community and connect it to the GNN message passing and GNN Explanation.•

Original: Propose a new HN-value-based GNN explanation method GStarX, and demonstrate the superiorityof GStarX over strong baselines for explaining GNNs on chemical and text graphs.
Revised: Propose a new HN-value-based GNN explanation method GStarX, and demonstrate the superiorityof GStarX over strong baselines for explaining GNNs on chemical and text graphs.

Original:  Preliminaries2.1 Graph neural networks
Revised:  Preliminaries

Revised: 2.1 Graph neural networks

Original: Consider a graph G with (feature-enriched) nodes V and edges E .
Revised: Consider a graph G with (feature-enriched) nodes V and edges E .

Original: We denote G as G =
Revised: We denote G as G =

Original: ( V , X , A ) ,where X ∈ R n × d denotes d -dimensional features of n nodes in V , and A ∈ { 0 , 1 } n × n denotes theadjacency matrix specifying edges in E .
Revised: ( V , X , A ) ,where X ∈ R n × d denotes d -dimensional features of n nodes in V , and A ∈ { 0 , 1 } n × n denotes theadjacency matrix specifying edges in E .

Original: GNNs make predictions on G by learning representationsvia the message-passing mechanism.
Revised: GNNs make predictions on G by learning representationsvia the message-passing mechanism.

Original: During message passing, the representation of each nodeu ∈ V is updated by aggregating its only representation and representations (messages) from itsneighbors, denoted as N ( u ) .
Revised: During message passing, the representation of each nodeu ∈ V is updated by aggregating its only representation and representations (messages) from itsneighbors, denoted as N ( u ) .

Original: This aggregation is recursively applied, so u can collect messagesfrom its multi-hop neighbors and produce structure-aware representations [7].
Revised: This aggregation is recursively applied, so u can collect messagesfrom its multi-hop neighbors and produce structure-aware representations [7].

Original: With h ( l ) i denotesthe representation of node i at iteration l , and AGGR ( · , · ) 
Revised: With h ( l ) i denotesthe representation of node i at iteration l , and AGGR ( · , · ) denotes the aggregation operation, e.g. summation, the representation update is shown in Equation

Original: [Equation]
Revised: [Equation]

Original: 2.2 Cooperative games
Revised: 2.2 Cooperative games

Original: A cooperative game denoted by ( N, v ) , is defined by a set of players N = { 1 , . . . , n } , and acharacteristic function v : 2 N → R .
Revised: A cooperative game denoted by ( N, v ) , is defined by a set of players N = { 1 , . . . , n } , and acharacteristic function v : 2 N → R .

Original: v takes a subset of players S ⊆ N , called a coalition , and mapsit to a payoff v ( S ) , where v ( ∅ ) :
Revised: v takes a subset of players S ⊆ N , called a coalition , and mapsit to a payoff v ( S ) , where v ( ∅ ) :

Original: = 0 .
Revised: = 0 .

Original: A solution function ϕ is a function maps each given game ( N, v )to ϕ ( N, v )
Revised: A solution function ϕ is a function maps each given game ( N, v )to ϕ ( N, v )

Original: ∈ R n .
Revised: ∈ R n .

Original: The vector ϕ ( N, v ) , called a solution , represents a certain allocation of the totalpayoff v ( N ) generated by all players to each individual, with the i th coordinate ϕ
Revised: The vector ϕ ( N, v ) , called a solution , represents a certain allocation of the totalpayoff v ( N ) generated by all players to each individual, with the i th coordinate ϕ

Original: i ( N, v ) being thepayoff attributed to player i .
Revised: i ( N, v ) being thepayoff attributed to player i .

Original: ϕ ( N, v ) is also called the “value” of the game when it satisfies certainproperties, and different values were proposed to name solutions with different properties [1, 8].
Revised: ϕ ( N, v ) is also called the “value” of the game when it satisfies certainproperties, and different values were proposed to name solutions with different properties [1, 8].

Original: The Shapley value is one popular solution of cooperative games.
Revised: The Shapley value is one popular solution of cooperative games.

Original: The main idea is to assign eachplayer a “fair” share of the total payoff by considering all possible player interactions.
Revised: The main idea is to assign eachplayer a “fair” share of the total payoff by considering all possible player interactions.

Original: For example,when player i cooperates with a coalition S , the total payoff v ( S ∪ { i } ) may be very different fromv ( S )
Revised: For example,when player i cooperates with a coalition S , the total payoff v ( S ∪ { i } ) may be very different fromv ( S )

Original: + v ( { i } ) because of i ’s interaction with S.
Revised: + v ( { i } ) because of i ’s interaction with S.

Original: Thus the marginal contribution of i to S is defined asby m ( i, S )
Revised: Thus the marginal contribution of i to S is defined asby m ( i, S )

Original: = v ( S ∪ { i } ) − v ( S ) .
Revised: = v ( S ∪ { i } ) − v ( S ) .

Original: Then the formula of the Shapley value for i is shown in Equation 2,where marginal contributions to all possible coalitions
Revised: Then the formula of the Shapley value for i is shown in Equation 2,where marginal contributions to all possible coalitions

Original: S ⊆
Revised: S ⊆

Original: N \{ i } are aggregated.
Revised: N \{ i } are aggregated.

Original: The first identifyin Equation 2 shows that the aggregation weights are first uniformly distributed among coalition sizesk (outer average), then uniformly distributed among all coalitions with the same size (inner average).ϕ
Revised: The first identifyin Equation 2 shows that the aggregation weights are first uniformly distributed among coalition sizesk (outer average), then uniformly distributed among all coalitions with the same size (inner average).ϕ

Original: [Equation]
Revised: [Equation]

Original:  N, v ) =
Revised: ( N, v ) =

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: (cid:123)(cid:0) n − 1 k (cid:1) (cid:88)
Revised: (cid:123)(cid:0) n − 1 k (cid:1) (cid:88)

Original: S ⊆ N \{ i } | S | =
Revised: S ⊆ N \{ i } | S | =

Original: k m ( i, S )
Revised: k m ( i, S )

Original: = (cid:88)
Revised: = (cid:88)

Original: S ⊆ N \{ i } | S | !
Revised: S ⊆ N \{ i } | S | !

Original: ( n − | S | − 1)!
Revised: ( n − | S | − 1)!

Original: n ! m
Revised: n ! m

Original: ( i, S ) (2)
Revised: ( i, S ) (2)

Original: Games with communication structures.
Revised: Games with communication structures.

Original: Although the Shapley value is widely used for cooperativegames, its assumption of fully flexible cooperation among all players may not be achievable.
Revised: Although the Shapley value is widely used for cooperativegames, its assumption of fully flexible cooperation among all players may not be achievable.

Original: Somecoalitions may be preferred over others and some may even be impossible due to limited communication among players.
Revised: Somecoalitions may be preferred over others and some may even be impossible due to limited communication among players.

Original: Thus, [9] uses a graph G as the communication structure of players to representcooperation preference.
Revised: Thus, [9] uses a graph G as the communication structure of players to representcooperation preference.

Original: A game with a communication structure is defined by a triple ( N, v, G ) , with
Revised: A game with a communication structure is defined by a triple ( N, v, G ) , with

Original: N being node set of G .
Revised: N being node set of G .

Original: It is more practical when cooperation preference is available.
Revised: It is more practical when cooperation preference is available.

Original: Several suchvalues with different properties have been proposed [9, 10, 11, 12] including the HN value [6].
Revised: Several suchvalues with different properties have been proposed [9, 10, 11, 12] including the HN value [6].

Original:  GNN explanation via feature importance scoring3.1 Problem formalization
Revised:  GNN explanation via feature importance scoring3.1 Problem formalization

Original: A general approach to formalize a ML explanation problem is through feature importance scoring[3, 13], where features may refer to pixels of images, word of text, or nodes/edges/subgraphs of graphs.
Revised: A general approach to formalize a ML explanation problem is through feature importance scoring[3, 13], where features may refer to pixels of images, word of text, or nodes/edges/subgraphs of graphs.

Original: Let f ( · ) denote a to-be-explained GNN, G =
Revised: Let f ( · ) denote a to-be-explained GNN, G =

Original: ( V , X , A ) denote an input graph, and 0 ≤ γ ≤ 1denote a sparsity constraint to enforce concise explanation.
Revised: ( V , X , A ) denote an input graph, and 0 ≤ γ ≤ 1denote a sparsity constraint to enforce concise explanation.

Original: GNN explanation via subgraph scoring isaimed to find a subgraph g that maximizes a given evaluation metric E VAL ( · , · , · ) , which measuresthe faithfulness of g to G regarding making predictions with f ( · ) , i.e.
Revised: GNN explanation via subgraph scoring isaimed to find a subgraph g that maximizes a given evaluation metric E VAL ( · , · , · ) , which measuresthe faithfulness of g to G regarding making predictions with f ( · ) , i.e.

Original: [Equation]
Revised: [Equation]

Original: When the task is graph classification and f ( · ) outputs a one-sum vector f ( G ) ∈ [0 , 1] C containingprobabilities for G belongs to C classes, an example E VAL can be the prediction probability drop forremoving g from G , i.e. E VAL ( f ( · ) , G , g ) =
Revised: When the task is graph classification and f ( · ) outputs a one-sum vector f ( G ) ∈ [0 , 1] C containingprobabilities for G belongs to C classes, an example E VAL can be the prediction probability drop forremoving g from G , i.e. E VAL ( f ( · ) , G , g ) =

Original: [ f ( G )]
Revised: [ f ( G )]

Original: c ∗ − [ f ( G\ g )]
Revised: c ∗ − [ f ( G\ g )]

Original: c ∗ with c ∗ =
Revised: c ∗ with c ∗ =

Original: arg max c [ f ( G )] c .
Revised: arg max c [ f ( G )] c .

Original: In practice, since the number of subgraphs is combinatorial in the number of nodes, the objectiveis often relaxed to finding a set of important nodes or edges first, and then inducing the subgraph[14, 15, 4].
Revised: In practice, since the number of subgraphs is combinatorial in the number of nodes, the objectiveis often relaxed to finding a set of important nodes or edges first, and then inducing the subgraph[14, 15, 4].

Original: Then a more tractable objective of finding the optimal set of nodes S ∗ ⊆
Revised: Then a more tractable objective of finding the optimal set of nodes S ∗ ⊆

Original: V 1 is given by
Revised: V 1 is given by

Original: [Equation]
Revised: [Equation]

Original: Existing methods often boil down to Equation 4 with different scoring functions (S CORE ), anddefining a proper S CORE is non-trivial.
Revised: Existing methods often boil down to Equation 4 with different scoring functions (S CORE ), anddefining a proper S CORE is non-trivial.

Original: For example, one example of S CORE is to evaluate each node i directly as S CORE ( f ( · ) , G , i ) =
Revised: For example, a simple example of S CORE is to evaluate each node i directly as S CORE ( f ( · ) , G , i ) =

Original: [ f ( { i } )] c ∗ .
Revised: [ f ( { i } )] c ∗ .

Original: However, this choice misses interactions betweennodes and corresponds to a trivial case of no message-passing for { i } .
Revised: However, this choice misses interactionsbetween nodes and corresponds to a trivial case of no message-passing but only a feed-forwardnetwork applied on { i } .

Original: Another possibility is to use E VAL as S CORE , e.g., S CORE ( f ( · ) , G , i ) =
Revised: Another possible choice is to use E VAL as S CORE , like the example above: S CORE ( f ( · ) , G , i ) =

Original: [ f ( G )]
Revised: [ f ( G )]

Original: c ∗ − [ f ( G\{ i } )] c ∗ .
Revised: c ∗ − [ f ( G\{ i } )] c ∗ .

Original: However, this again fails tocapture interactions between nodes; for example, two nodes i and j may be both important but alsocomplimentary, so their contribution to G can only be observed when they are missing simultaneously.
Revised: However, this again fails to capture interactionsbetween nodes; for example, two nodes i and j may be both important but also complimentary, sotheir contribution to G can only be observed when they are missing simultaneously.

Original: 3.2 Scoring functions from cooperative games
Revised: 3.2 Scoring functions from cooperative games

Original: Given the challenges for defining a proper S CORE , solutions to cooperative games, like the Shapley value, have been proposed with f ( · ) as the characteristic function, i.e. S CORE ( f ( · ) , G , i ) =
Revised: Given the challenges for defining a proper S CORE , solutions to cooperative games, like the Shapley value, have been proposed with f ( · ) as the characteristic function, i.e. S CORE ( f ( · ) , G , i ) =

Original: ϕ
Revised: ϕ

Original: i ( |G| , f ( · ))
Revised: i ( |G| , f ( · ))

Original: However, existing works only use the non-structure-aware Shapley value.
Revised: However, existing works only use the non-structure-aware Shapley value.

Original: Incontrast, values defined on games ( N, v, G ) with communication structures G are naturally structureaware but were never considered.
Revised: Incontrast, values defined on games ( N, v, G ) with communication structures G are naturally structureaware but were never considered.

Original: Below we discuss the non-structure-aware limitation of the Shapleyvalue in detail and motivating structure-aware values with practical examples in GNN explanation.
Revised: Below we discuss the non-structure-aware limitation of the Shapleyvalue in detail and motivating structure-aware values with practical examples in GNN explanation.

Original: The Shapley value is defined on games ( N, v ) , which by definition take no graph structures.
Revised: The Shapley value is defined on games ( N, v ) , which by definition take no graph structures.

Original: Itassumes flexible cooperation between players and uniform distribution of coalition importance thatonly depends on | S | (see Equation 2).
Revised: Itassumes flexible cooperation between players and uniform distribution of coalition importance thatonly depends on | S | (see Equation 2).

Original: Even if a G is given and the game is defined as ( N, v, G ) ,the Shapley value will overlook G when aggregating m ( i, S ) .
Revised: Even if a G is given and the game is defined as ( N, v, G ) ,the Shapley value will overlook G when aggregating m ( i, S ) .

Original: In contrast, structure-aware values on( N, v, G ) can be interpreted as a weighted aggregation of coalitions with more reasonable weights.
Revised: In contrast, structure-aware values on( N, v, G ) can be interpreted as a weighted aggregation of coalitions with more reasonable weights.

Original: Although different solutions ϕ ( N, v, G ) have their nuances in weight adjustments [11, 6, 9, 12], theyshare two key properties: (1) the weight is 0 if i and S are disconnected because they are interpretedas players without communication channels [9], and (2) the weight is impacted by the nature ofconnections between i and S because it is easier for better-connected nodes to communicate.
Revised: Although different solutions ϕ ( N, v, G ) have their nuances in weight adjustments [11, 6, 9, 12], theyshare two key properties: (1) the weight is 0 if i and S are disconnected because they are interpretedas players without communication channels [9], and (2) the weight is impacted by the nature ofconnections between i and S because it is easier for better-connected nodes to communicate.

Original: A synthetic example.
Revised: A synthetic example.

Original: We take the HN value (definition in Section 4.1) as an example structure-awarevalue and compare it to the Shapley value in a simple graph in Figure 1 (a) .
Revised: We take the HN value (definition in Section 4.1) as an example structure-awarevalue and compare it to the Shapley value in a simple graph in Figure 1 (a) .

Original: To compute ϕ 1 ( N, v, G ) ,both values aggregates m (1 , S ) for S ∈ {∅ , { 2 } , { 3 } , { 2 , 3 }} .
Revised: To compute ϕ 1 ( N, v, G ) ,both values aggregates m (1 , S ) for S ∈ {∅ , { 2 } , { 3 } , { 2 , 3 }} .

Original: The Shapley value first assigns auniform weight 13 to three different | S | , and then splits weights uniformly for the | S | = 1 case to be 16 .
Revised: The Shapley value first assigns auniform weight 13 to three different | S | , and then splits weights uniformly for the | S | = 1 case to be 16 .

Original: However, the HN value assigns weight 0 for S
Revised: However, the HN value assigns weight 0 for S

Original: = { 3 } because 1 and 3 are disconnected in coalition{ 1 , 3 } and are assumed to be two independent graphs that shouldn’t interact (property (1) ).
Revised: = { 3 } because 1 and 3 are disconnected in coalition{ 1 , 3 } and are assumed to be two independent graphs that shouldn’t interact (property (1) ).

Original: Theirinteraction is rather captured in the S
Revised: Theirinteraction is rather captured in the S

Original: = { 2 , 3 } case, when 1 and 3 are connected by the bridgingnode 2.
Revised: = { 2 , 3 } case, when 1 and 3 are connected by the bridgingnode 2.

Original: This case is also downweighted from 13 to 14 , as 3 is relatively far from 1 (property (2) ).
Revised: This case is also downweighted from 13 to 14 , as 3 is relatively far from 1 (property (2) ).

Original: A practical example.
Revised: A practical example.

Original: The good properties of structure-aware values can help explain graph tasks.
Revised: The good properties of structure-aware values can help explain graph tasks.

Original: The example in Figure 1 (b) is from GraphSST2 (dataset description in Section 5.1), where thegraph for sentiment classification is constructed from the sentence “is still quite good-natured andnot a bad way to spend an hour” with edges generated by the Biaffine parser [16].
Revised: The example in Figure 1 (b) is from GraphSST2 (dataset description in Section 5.1), where thegraph for sentiment classification is constructed from the sentence “is still quite good-natured andnot a bad way to spend an hour” with edges generated by the Biaffine parser [16].

Original: Assuming amodel can correctly classify it as positive.
Revised: Assuming amodel can correctly classify it as positive.

Original: Intuitively, “good” and “not a bad” are central to thehuman explanation.
Revised: Intuitively, “good” and “not a bad” are central to thehuman explanation.

Original: To compute the Shapley value of the word “good”, the coalition “not good” willcontribute negatively and diminish the positive importance of “good”, despite the two words lackingany direct connection.
Revised: To compute the Shapley value of the word “good”, the coalition “not good” willcontribute negatively and diminish the positive importance of “good”, despite the two words lackingany direct connection.

Original: A structure-aware value can instead eliminate the {“not”, “good”} coalition,and only consider interactions between “not” and “good” (in fact, “not” and any other word) when thebridging “bad” appears, hence better binding “not” with “bad” and improving the salience of “good”.
Revised: A structure-aware value can instead eliminate the {“not”, “good”} coalition,and only consider interactions between “not” and “good” (in fact, “not” and any other word) when thebridging “bad” appears, hence better binding “not” with “bad” and improving the salience of “good”.

Original: In Section 5.2, we revisit this example to observe impacts of structure-awareness empirically.

Original:  GStarX: Graph Structure-aware eXplanation
Revised:  GStarX: Graph Structure-aware eXplanation

Original: We propose GStarX, which uses a structure-aware HN-value-based S CORE to explain GNNs.
Revised: We propose GStarX, which uses a structure-aware HN-value-based S CORE to explain GNNs.

Original: Wefirst state the definition of the HN value in cooperative game theory (4.1), and then connect it to the
Revised: Wefirst state the definition of the HN value in cooperative game theory (4.1), and then connect it to the

Original: GNN message passing (4.2), and finally give the GStarX algorithm for GNN explanation (4.3).
Revised: GNN message passing (4.2), and finally give the GStarX algorithm for GNN explanation (4.3).

Original: 4.1 The HN value
Revised: 4.1 The HN value

Original: For any game ( N, v, G ) with a communication structure G and S ⊆ N being a coalition, let S ∗ = ∪ i ∈ S {N ( i ) } ∪ S be the union of S and its neighbors in G .let S/ G =
Revised: For any game ( N, v, G ) with a communication structure G and S ⊆ N being a coalition, we use S ∗ = ∪ i ∈ S {N ( i ) } ∪ S to denote the union of S and its neighbors in G .

Original: { i | i and j are connected in S by E of G}| j ∈ S } be a partition of S to nodes forming connected components in G . Let G [ S ] be the induced subgraph of S in G .
Revised: We use S/ G to denote thenode partition of S , which is determined by the connected components of the induced subgraph of Sin G (denoted as G [ S ] ).

Original: For example, in Figure 1 (b) ,when S = {“is”, “an”, “hour”}, S ∗ will be {“is”, “good”, “an”, “hour”, “spend”},S/ G will be {{“is”},{“an”, “hour”}} , and  G[ S ] will be the subgraph with a two-node (cid:3)(cid:2) (cid:0)(cid:1) an - (cid:3) (cid:2) (cid:0) (cid:1) hour and a single node (cid:3) (cid:2) (cid:0) (cid:1) is .
Revised: For example, in Figure 1 (b) , when S = {“is”, “an”, “hour”}, S ∗ will be {“is”,“good”, “an”, “hour”, “spend”}, and S/ G will be {{“is”}, {“an”, “hour”}}.

Original: Definition 4.1 ( Surplus ) .
Revised: Definition 4.1 ( Surplus ) .

Original: p ( j, S ) representing the surplus generated by a coalition S cooperatingwith its neighbor j is defined as
Revised: p ( j, S ) representing the surplus generated by a coalition S cooperatingwith its neighbor j is defined as

Original: [Equation]
Revised: [Equation]

Original: Intuitively, p ( j, S ) is generated because S is actively cooperating.
Revised: Intuitively, p ( j, S ) is generated because S is actively cooperating.

Original: Thus, when evaluating a fair payoffto S , a portion of p ( j, S ) should be added to its own payoff v ( S ) .
Revised: Thus, when evaluating a fair payoffto S , a portion of p ( j, S ) should be added to its own payoff v ( S ) .

Original: This idea leads to the definition ofassociated games regarding the original games, where surplus allocation is performed.
Revised: This idea leads to the definition ofassociated games regarding the original games, where surplus allocation is performed.

Original: Definition 4.2 ( HN Associated Game ) .
Revised: Definition 4.2 ( HN Associated Game ) .

Original: Given 0 ≤ τ ≤ 1 representing the portion of surplus that will be allocated to a coalition S .
Revised: Given 0 ≤ τ ≤ 1 representing the portion of surplus that will be allocated to a coalition S .

Original: The HN associated game ( N, v ∗ τ , G ) of ( N, v, G ) is defined as
Revised: The HN associated game ( N, v ∗ τ , G ) of ( N, v, G ) is defined as

Original: [Equation]
Revised: [Equation]

Revised: otherwise

Revised: [Equation]

Original: The HN value is a solution on ( N, v, G ) .
Revised: The HN value is a solution on ( N, v, G ) .

Original: It is computed by iteratively constructing a series of HNassociated games until it converges to a limit game ( N, ˜ v, G ) .
Revised: It is computed by iteratively constructing a series of HNassociated games until it converges to a limit game ( N, ˜ v, G ) .

Original: In other words, we first construct v ∗ τfrom v by surplus allocation.
Revised: In other words, we first construct v ∗ τfrom v by surplus allocation.

Original: Then we construct v ∗∗ τ from v ∗ τ by allocating the surplus generated fromthe v ∗ τ and so on.
Revised: Then we construct v ∗∗ τ from v ∗ τ by allocating the surplus generated fromthe v ∗ τ and so on.

Original: The convergence of the limit game is guaranteed and the result ˜ v is independent ofτ under mild conditions as shown in [6].
Revised: The convergence of the limit game is guaranteed and the result ˜ v is independent ofτ under mild conditions as shown in [6].

Original: The HN value of each player is uniquely determined byapplying ˜ v to that player, i.e. ϕ i ( N, v, G ) =
Revised: The HN value of each player is uniquely determined byapplying ˜ v to that player, i.e. ϕ i ( N, v, G ) =

Original: ˜ v ( { i } ) .
Revised: ˜ v ( { i } ) .

Original: We state the formal definitions of the limit gameand the uniqueness theorem of the HN value in Appendix E.2.
Revised: We state the formal definitions of the limit gameand the uniqueness theorem of the HN value in Appendix E.2.

Original: 4.2 Connecting GNNs and the HN surplus allocation through the message passing lens
Revised: 4.2 Connecting GNNs and the HN surplus allocation through the message passing lens

Original: Both the GNN message passing (MP) and the associated game surplus allocation (SA) are iterativeaggregation algorithms, with considerable alignment.
Revised: Both the GNN message passing (Equation 1) and the associated game surplus allocation (Equation6) are iterative aggregation algorithms, with considerable alignment.

Original: In fact, SA on each singular node set S = { i }is exactly MP: Equation 6 becomes an instantiation of Equation 1 with AGGR ( a, b ) = a + τ (cid:80) j b jon a scalar node value a and a neighbor set b .
Revised: In fact, the surplus allocationon each singular node set S = { i } is exactly a message passing operation: Equation 6 becomesan instantiation of Equation 1 with AGGR ( a, b ) = a + τ (cid:80) j b j on a scalar node value a and aneighbor set b .

Original: These algorithms differ in that SA applies more broadlyto | S |≥ 1 cases; it treats S as a supernode when nodes in S form a connected component in G , andhandles disconnected S component-wise via Equation 7.
Revised: The algorithms differ in that HN surplus allocation applies more broadly to caseswhere | S |≥ 1 ; it treats S as a supernode when nodes in S form a connected component in G , andhandles disconnected S component-wise via Equation 7.

Original: We illustrate the surplus allocation using a real chemical graph example.
Revised: We illustrate the surplus allocation using a real chemical graph example.

Original: The molecule shown in
Revised: The molecule shown in

Original: Figure 1 (c) is taken from MUTAG (dataset description in Section 5.1).
Revised: Figure 1 (c) is taken from MUTAG (dataset description in Section 5.1).

Original: It is known to be classifiedas mutagenic because of the -NO2 group (nodes 1, 2, and 3) [17].
Revised: It is known to be classifiedas mutagenic because of the -NO2 group (nodes 1, 2, and 3) [17].

Original: When we compute v ∗ τ ( { 1 } ) ,the surplus p (2 , { 1 } ) , p (3 , { 1 } ) , and p (4 , { 1 } ) are allocated to node 1 (like messages passed to acentral node in GNN).
Revised: When we compute v ∗ τ ( { 1 } ) ,the surplus p (2 , { 1 } ) , p (3 , { 1 } ) , and p (4 , { 1 } ) are allocated to node 1 (like messages passed to acentral node in GNN).

Original: Then surplus are aggregated together with v ( { 1 } ) following Equation 6 toform v ∗ τ ( { 1 } ) .
Revised: Then surplus are aggregated together with v ( { 1 } ) following Equation 6 toform v ∗ τ ( { 1 } ) .

Original: For graphs, the surplus allocation approach has two advantages over the uniformaggregation used in the Shapley value: (1) : the aggregated payoff in each v ∗ τ is structure-aware likerepresentations learned by GNNs [7], and (2) the iterative computation preserves locality, whichis what GNNs preserve [18].
Revised: For graphs, the surplus allocation approach has two advantages over the uniformaggregation used in the Shapley value: (1) : the aggregated payoff in each v ∗ τ is structure-aware likerepresentations learned by GNNs [7], and (2) the iterative computation preserves locality, whichis what GNNs preserve [18].

Original: In other words, close neighbors heavily influence each other due tocooperation in many iterations, while far away nodes have a lesser influence on each other due to littlecooperation.
Revised: In other words, close neighbors heavily influence each other due tocooperation in many iterations, while far away nodes have a lesser influence on each other due to littlecooperation.

Original: In this example, since the local -NO2 piece generates a high payoff for the mutagenicityclassification, locally allocating the payoff helps us better understand the importance of the nitrogenatom and the oxygen atoms.
Revised: In this example, since the local -NO2 piece generates a high payoff for the mutagenicityclassification, locally allocating the payoff helps us better understand the importance of the nitrogenatom and the oxygen atoms.

Original: Whereas aggregating over many unnecessary coalitions with far-awaycarbon atoms can obscure the true contribution of -NO2.
Revised: Whereas aggregating over many unnecessary coalitions with far-awaycarbon atoms can obscure the true contribution of -NO2.

Original: We will revisit this example in Section 5.2.
Revised: We will revisit this example in Section 5.3.

Original: 4.3 The GStarX algorithm
Revised: 4.3 The GStarX algorithm

Original: We now state our  algorithm for explaining GNNs with GStarX. Notice that GStarX scores nodes ina graph but not each dimension of node features. Feature dimension importance explanationis an important and more general problem that can be added on top of GStarX .
Revised: We now state our main algorithm for explaining GNNs with GStarX, which essentially implementsand solves the objective in Equation 4.

Original: We leave thisextension as a future work. GStarX essentially implements and solves the objective in Equation 4,where an HN-value-based S CORE is used.
Revised: We first introduce our HN-value-based S CORE for theobjective.

Original: To use such S CORE , we need to define the players andthe characteristic function of the game, and then apply the  formula in Equation 6 and 7.
Revised: To use the HN value as the S CORE , we need to define the players and the characteristicfunction of the game, and then apply the HN value formula in Equation 6 and 7.

Original: Supposethe inputs include a graph G with nodes V = { u 1 , . .
Revised: Suppose the inputsinclude a graph G with nodes V = { u 1 , . .

Original: u n } and label y ∈ { 1 , . .
Revised: u n } and label y ∈ { 1 , . .

Original: , C } , a GNN f ( · )outputs a probability vector f ( G ) ∈ [0 , 1] C , and the predicted class c ∗ = arg max c [ f ( G )] c .
Revised: , C } , a GNN f ( · ) outputsa probability vector f ( G ) ∈ [0 , 1] C , and the predicted class c ∗ = arg max c [ f ( G )] c .

Original: Let V beplayers, and let the normalized probability of the predicted class be the characteristic function v :
Revised: We let V beplayers, and let the normalized probability of the predicted class be the characteristic function v :

Original: [Equation]
Revised: [Equation]

Original: Algorithm
Revised: Algorithm

Original: Input: Graph  G with nodes V = { u 1 , . . . , u n } , trained GNN f ( · ) , empirical expectation f 0 , hyperparameter τ , max sample size m , number of samples J , sparsity γ.
Revised: Input: Graph instance G with nodes V = { u 1 , . . . , u n } , trained model f ( · ) , empirical expectation f 0 , hyperparameter τ , maximum sample size m , number of samples J , sparsity γ

Original: Get the predicted class c ∗ = arg max c [ f ( G )]
Revised: Get the predicted class c ∗ = arg max c [ f ( G )]

Original: c
Revised: c

Original: Define  characteristic function v ( S ) =
Revised: Define the characteristic function v ( S ) =

Original: [ f ( g S )]
Revised: [ f ( g S )]

Original: c ∗ − f 0 c ∗ if n ≤ m then ϕ =
Revised: c ∗ − f 0 c ∗ if n ≤ m then ϕ =

Original: Compute-HN ( G , V , v ( · ) , τ ) else ϕ
Revised: Compute-HN-Values ( G , V , v ( · ) , τ ) else ϕ

Original: =
Revised: =

Original: Compute-HN-MC ( G , V , v ( · ) , τ, m, J ) end if
Revised: Compute-HN-Values-MC ( G , V , v ( · ) , τ, m, J ) end if

Original: Sort ϕ in descending order with indices { π 1 , . . .
Revised: Sort ϕ in descending order with indices { π 1 , . . .

Original: , π n } k
Revised: , π n } k

Original: =
Revised: =

Original: ⌊ γ |V|⌋
Revised: ⌊ γ |V|⌋

Original: Return: S ∗ = { u π 1 , . . .
Revised: Return: S ∗ = { u π 1 , . . .

Original: , u π k }
Revised: , u π k }

Original: Algorithm

Original: Input: Graph instance G with nodes V = { u 1 , . . . , u n } , characteristic function v , hyperparameter τ . for S  in2 N do Compute payoff v ( S ) {Eq.(8)} end for
Revised: Here G [ S ] denotes the subgraph induced by nodes in S ⊆ V .

Original: Construct matrix H { τ,n, G} {Eq.(16)} repeat

Original: H = HH until H converges

Original: Get the limit game ˜ v

Original: = H v {Eq.(17)}

Original: Assign the first n entries of ˜ v to ϕ

Original: Return: ϕ

Original: Here the normalization term f 0 c ∗ =
Revised: The normalization term f 0 c ∗ =

Original:  E [[ f ( G )] c ∗ ] is the expectation over a random variable G representing a general graph.
Revised: 202 E [[ f ( G )] c ∗ ] is the expectation over a random variable G representing the general graph.

Original: In practice, we approximate it using the empirical expectation over all G in thedataset.
Revised: In practice,we approximate it using the empirical expectation over all G in the dataset.

Original:  S CORE will be the HN value of the game, i.e., S CORE ( f ( · ) , G , i ) =
Revised: The function S CORE willthus be the HN value of the game, i.e. S CORE ( f ( · ) , G , i ) =

Original: ϕ
Revised: ϕ

Original: i ( V , v, G ) =
Revised: i ( V , v, G ) =

Original: ˜ v ( { i } ) .
Revised: ˜ v ( { i } ) .

Original: Given  S CORE , we solve the objective by first computing the scores ϕ ∈ R n then selecting the top⌊ γ |V|⌋ scores greedily as in Algorithm 1.
Revised: Given the S CORE function, we solve the objective by first computing the scores ϕ ∈ R n then selectingthe top ⌊ γ |V|⌋ scores greedily as in Algorithm 1.

Original: Practically, like other game-theoretic methods, the exactcomputation of the HN value is infeasible when the number of players n is large.
Revised: Practically, like other game-theoretic methods, theexact computation of the HN value is infeasible when the number of players n is large (see Appendix E.3).

Original: We thus do anexact computation for small graphs (the if-branch) and Monte-Carlo sampling for large graphs (theelse-branch).
Revised: We thus do an exact computation for small graphs (the if-branch) and Monte-Carlo sampling forlarge graphs (the else-branch).

Original: The Compute-HN function is shown in Algorithm 2, where the H stands for a matrixform of the associated game defined in Definition 4.2.

Original: See AppendixE.2 and E.3 for details ofthe matrix form and algorithms for Compute-HN-MC .
Revised: See Appendix E.3 for detailed algorithms for Compute-HN-Values and Compute-HN-Values-MC .

Original: Also, even though the algorithm is stated forgraph classification, GStarX works for node classification as well.
Revised: Also, even though the algorithm is stated for graph classification, GStarXworks for node classification as well.

Original: This can be easily seen since 
Revised: This can be easily seen since GNNs classify nodes

Original: GNNs classify nodes u i by processing an ego-graph centered at u i , so the task can be converted tograph classification, where the label of u i is used as the label of the ego-graph.
Revised: u i byprocessing an ego-graph centered at u i , so the task can be converted to graph classification, wherethe label of u i is used as the label of the ego-graph.

Original: We focus on graphclassification in the main text for simpler illustration and discuss  node classification in Appendix B.
Revised: We focus on graph classification in the main textfor simpler illustration and discuss more about node classification in Appendix B.

Original:  Experiments
Revised:  Experiments

Original: 5.1 Experiment settings
Revised: 5.1 Experiment settings

Original: Datasets.
Revised: Datasets.

Original: We conduct experiments on datasets including synthetic data, chemical data, and text data.
Revised: We conduct experiments on datasets including synthetic data, chemical data, and text data.

Original: A brief description of the datasets is shown below with more detailed statistics in Appendix A.

Revised: A brief description of the datasets is shown below with more detailed statistics in Appendix A.

Original: • Chemical graph property prediction.
Revised: • Chemical graph property prediction.

Original: MUTAG [17], BACE and BBBP [19] contain molecule graphsfor graph classification, with atoms as nodes, bonds as edges, and chemical properties as labels.
Revised: MUTAG [17], BACE and BBBP [ contain molecule graphs for graph classification, with atoms as nodes, bonds as edges, and chemical properties as labels.

Original: • Text graph sentiment classification.
Revised: • Text graph sentiment classification.

Original: GraphSST2 and Twitter [20] contain graphs constructedfrom text.
Revised: GraphSST2 and Twitter [20] contain graphs constructed from text.

Original: Nodes are words with pre-trained BERT embeddings as features.
Revised: Nodes are words with pre-trained BERT embeddings as features.

Original: Edges are generatedby the Biaffine parser [16].
Revised: Edges are generatedby the Biaffine parser [16].

Original: Graphs are labeled as positive or negative sentiment.•
Revised: Graphs are labeled as positive or negative sentiment.

Original:  Synthetic graph motif detection.
Revised: • Synthetic graph motif detection.

Original: BA2Motifs [15] contains graphs with a Barabasi-Albert (BA) graph of size 20 and a 5-node motifs in each graph.
Revised: BA2Motifs [15] contains graphs with a Barabasi-Albert (BA)graph of size 20 and a 5-node motifs in each graph.

Original: Node features are 10-dimensional all-onevectors.
Revised: Node features are 10-dimensional all-onevectors.

Original: The motif can be a house-like structure or a cycle, which separates graphs into two classes.
Revised: The motif can be a house-like structure or a cycle, which separates graphs into two classes.

Original: GNNs and explanation baselines.
Revised: GNNs and explanation baselines.

Original: We evaluate GStarX by explaining GCNs [21] on all datasetsin our major experiment.
Revised: We evaluate GStarX by explaining GCNs [21] on all datasetsin our major experiment.

Original: In later analysis, we also evaluate on GIN [22] and GAT [23] on certaindatasets following [5].
Revised: In later analysis, we also evaluate on GIN [22] and GAT [23] on certaindatasets following [5].

Original: All models are trained to convergence with hyperparameters and performanceshown in Appendix A.2.
Revised: All models are trained to convergence with hyperparameters and performanceshown in Appendix A.2.

Original: We compare with 5 strong baselines representing the SOTA methods for
Revised: We compare with 5 strong baselines representing the SOTA methods for

Original: GNN explanation: GNNExplainer [14], PGExplainer [15], SubgraphX [5], GraphSVX [4], and
Revised: GNN explanation: GNNExplainer [14], PGExplainer [15], SubgraphX [5], GraphSVX [4], and

Original: OrphicX [24].
Revised: OrphicX [24].

Original: In particular, SubgraphX and GraphSVX use Shapley-value-based scoring functions.
Revised: In particular, SubgraphX and GraphSVX use Shapley-value-based scoring functions.

Original: Evaluation metrics.
Revised: Evaluation metrics.

Original: Evaluating explanations is non-trivial due to the lack of ground truth.
Revised: Evaluating explanations is non-trivial due to the lack of ground truth.

Original: Wefollow [5, 20] to employ Fidelity , Inverse Fidelity ( Inv-Fidelity ), and Sparsity as our evaluation metrics.
Revised: Wefollow [5, 20] to employ Fidelity , Inverse Fidelity ( Inv-Fidelity ), and Sparsity as our evaluation metrics.

Original: Fidelity and Inv-Fidelity measure whether the prediction is faithfully important to the model predictionby removing the selected nodes or only keeping the selected nodes respectively.
Revised: Fidelity and Inv-Fidelity measure whether the prediction is faithfully important to the model predictionby removing the selected nodes or only keeping the selected nodes respectively.

Original: Sparsity promotesfair comparison, since including more nodes generally improves Fidelity , and explanations with is the should you lameness on bad in .
Revised: Sparsity promotesfair comparison, since including more nodes generally improves Fidelity , and explanations withdifferent sizes are not directly comparable.

Original: how the title ’s movie clue is the should you lameness on bad in  .
Revised: Ideal explanations are high Fidelity , low Inv-Fidelity , andhigh Sparsity , indicating relevance and conciseness. Equations 9-11 show their formulas. in always colorful overblown funny very enjoyably . style traditional almodóvar occasionally , the and in always colorful overblown funny very enjoyably .

Original: how the title ’s movie clue is the should you lameness on bad in  .
Revised: style traditional almodóvar occasionally , the and in always colorful overblown funny very enjoyably . style traditional almodóvar occasionally , the and in always colorful overblown funny very enjoyably . style traditional almodóvar occasionally , the and in always colorful overblown funny very enjoyably . style traditional almodóvar occasionally , the and in always colorful overblown funny very enjoyably .

Original:  how the title ’s movie clue is the should you lameness on bad in .
Revised: style traditional almodóvar occasionally , the and is the should you lameness on bad in . how the title ’s movie clue is the should you lameness on bad in . how the title ’s movie clue is the should you lameness on bad in . how the title ’s movie clue is the should you lameness on bad in .

Original: how the title ’s movie clue is the should you lameness on bad in . how the title ’s movie clue is the should you lameness on bad in .
Revised: how the title ’s movie clue is the should you lameness on bad in . how the title ’s movie clue is the should you lameness on bad in .

Original: how the title ’s movie clue
Revised: how the title ’s movie clue

Original: GNNExplainer
Revised: GNNExplainer

Original: PGExplainer
Revised: PGExplainer

Original: SubgraphX
Revised: SubgraphX

Original: GraphSVX “is still quite good – natured and not a bad way to spend an hour” “the title 's lameness should clue you in on how bad the movie is.”
Revised: GraphSVX “occasionally funny , always very colorful and enjoyably overblown in the traditional almodóvar style.” “the title 's lameness should clue you in on how bad the movie is.”

Original: OrphicX
Revised: OrphicX

Original: GStarX [Figure] to an is quite a way spendgood bad still natured and not hour to an is quite a way spend - [Figure]
Revised: GStarX identifies the sentiment words more accurately compared to baselines.

Original: still natured and not hour to an is quite a

Original: [Figure]
Revised: [Figure]

Original: to an is quite a

Original: [Figure]

Original: [Figure]

Original: not hour to an is quite a

Original: [Figure]

Original: [Figure]

Original: still

Original: [Figure]

Original: not hour to an is quite a

Original: [Figure]

Original: [Figure]

Original: “Ground

Original: [Figure]

Original: different sizes are not directly comparable.

Original: Ideal explanations are high Fidelity , low Inv-Fidelity , andhigh Sparsity , indicating relevance and conciseness.

Original: Equations 9-11 show their formulas.

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Fidelity and Inv-Fidelity are complementary and are both important for a good explanation g .
Revised: Fidelity and Inv-Fidelity are complementary and are both important for a good explanation g .

Original: Fidelityjustifies the necessity for g to be included to predict correctly.
Revised: Fidelityjustifies the necessity for g to be included to predict correctly.

Original: Inv-Fidelity justifies the sufficiency of astandalone g to predict correctly.
Revised: Inv-Fidelity justifies the sufficiency of astandalone g to predict correctly.

Original: As they are analogous to precision and recall, we draw an analogyto the F1 score to propose a single-scalar-metric “harmonic fidelity” ( H-Fidelity ), where we normalizethem by Sparsity and take their harmonic mean; see Appendix A.3 for the formula.
Revised: As they are analogous to precision and recall, we draw an analogyto the F1 score to propose a single-scalar-metric “harmonic fidelity” ( H-Fidelity ), where we normalizethem by Sparsity and take their harmonic mean; see Appendix A.3 for the formula.

Original: Hyperparameters.
Revised: Hyperparameters.

Original: GStarX includes three hyperparameters: τ for the associated game, maximumsample size m , and number of samples J .
Revised: GStarX includes three hyperparameters: τ for the associated game, maximumsample size m , and number of samples J .

Original: In our experiments we choose τ = 0 .
Revised: In our experiments we choose τ = 0 .

Original: since we needτ < 2 n for convergence (Appendix E.2) and all graphs have less than 200 nodes.
Revised: since we needτ < 2 n for convergence (Appendix E.2) and all graphs have less than 200 nodes.

Original: Bigger m and Jshould be better for the MC approximation, and we found m = 10 and J = n work well empirically.
Revised: Bigger m and Jshould be better for the MC approximation, and we found m = 10 and J = n work well empirically.

Original: 5.2 Evaluation results
Revised: 5.2 Quantitative studies

Original: Quantitative studies.

Original: We report averaged test set H-Fidelity in Table 1.
Revised: We report averaged test set H-Fidelity in Table 1.

Original: We conduct 8 different runsto get results with Sparsity ranging from 0.5-0.85 in 0.05 increments ( Sparsity cannot be preciselyguaranteed, hence it has minorvariations across methods) and report the best H-Fidelity for each
Revised: We conduct 8 different runs to get results with Sparsity ranging from 0.5-0.85 in 0.05 increments ( Sparsity cannot be precisely guaranteed minor

Original: [Table]
Revised: [Table]

Original: Qualitative studies. We visualize the explanations of graphs in GraphSST2 in Figure 2 and comparethem qualitatively.
Revised: We visualize the explanations of graphs in GraphSST2 in Figure 2 and compare them qualitatively.

Original: We show explanations selected with high and comparable Sparsity on a positive(upper) graph and a negative (lower) graph. GStarX
Revised: We show explanations selected with high and comparable Sparsity on a positive (upper) graph and anegative (lower) graph. GStarX

Original: concisely captures the important words forsentiment classification without including extraneous ones for both sentences.
Revised: concisely captures the important words for sentiment classificationwithout including extraneous ones for both sentences.

Original: Baseline methodsgenerally select some-but-not-all important sentiment words, with extra neutral words as well.
Revised: Baseline methods generally select some-butnot-all important sentiment words, with extra neutral words as well.

Original: Among baselines, SubgraphX gives more reasonable results.
Revised: Among baselines, SubgraphXgives more reasonable results.

Original: However, it cannot cover two groupsof important nodes with a limited budget because it can only select a connected subgraph as theexplanation; e.g. to cover the negative word “lameness” in the lower sentence, SubgraphX needsat least 3 more nodes along the way, which will significantly decrease Sparsity while includingundesirable, neutral words.
Revised: However, it cannot cover two groups of important nodes with a limitedbudget because it can only select a connected subgraph as the explanation; e.g. to cover the negativeword “lameness” in the second sentence, SubgraphX needs at least 3 more nodes along the way,which will significantly decrease Sparsity while including undesirable, neutral words.

Original: Moreover, we discussed in Section 3.2 that the Shapley value willdowngrade the positive importance of the word “good” for the upper sentence.

Original: Comparing thenormalized contribution scores of our HN-value-based method GStarX and the Shapley-basedmethod GraphSVX, contribution of “good” is higher in ours: 0.1152 vs. 0.0371.

Original: We visualize explanations selected with high and comparable Sparsity of a mutagenic moleculefrom MUTAG in Figure 3.
Revised: We visualize explanations selected with high and comparable Sparsity of a mutagenic moleculefrom MUTAG in Figure 3.

Original: Explanations on chemical graphs are harder to evaluate than text graphsas they require domain knowledge.
Revised: Explanations on chemical graphs are harder to evaluate than text graphsas they require domain knowledge.

Original: MUTAG has been widely used as a benchmark for evaluating
Revised: MUTAG has been widely used as a benchmark for evaluating

Original: GNN explanations because human experts recognize -NO2 as mutagenic [17], which makes MUTAGa dataset with “ground truth” 2 .
Revised: GNN explanations because human experts recognize -NO2 as mutagenic [17], which makes MUTAGa dataset with “ground truth” 2 .

Original: Surprisingly, we found that GStarX generates much better HFidelity / Fidelity / Inv-Fidelity than other methods and even the “ground truth” by only selecting the-O in -NO2 as explanations.
Revised: Surprisingly, we found that GStarX generates much better HFidelity / Fidelity / Inv-Fidelity than other methods and even the “ground truth” by only selecting the-O in -NO2 as explanations.

Original: In particular, the -0.234 Inv-Fidelity of GStarX means the selectedsubgraph has an even better prediction result than the original whole graph (0 Inv-Fidelity ) and theground truth (-0.143 Inv-Fidelity ) because nodes not significant to the GNN prediction are removed.
Revised: In particular, the -0.234 Inv-Fidelity of GStarX means the selectedsubgraph has an even better prediction result than the original whole graph (0 Inv-Fidelity ) and theground truth (-0.143 Inv-Fidelity ) because nodes not significant to the GNN prediction are removed.

Original: Fidelity metrics of baselines are inferior to GStarX because they include other non-discriminativecarbon atoms despite they capture -NO2 to some extent.
Revised: Fidelity metrics of baselines are inferior to GStarX because they include other non-discriminativecarbon atoms despite they capture -NO2 to some extent.

Original: This suggests that even though humanexperts identify -NO2 as the “ground truth” of mutagenicity, the GNN only needs -O to classifymutagenic molecules.
Revised: This suggests that even though humanexperts identify -NO2 as the “ground truth” of mutagenicity, the GNN only needs -O to classifymutagenic molecules.

Original: With the goal being understand model behavior, GStarX explanation is better.
Revised: With the goal being understand model behavior, GStarX explanation is better.

Original: Moreover, SubgraphX is the only baseline that has better H-Fidelity than the “ground truth”, but itcan only capture one -NO2 because its search algorithm requires the explanation to be connected,so its Inv-Fidelity is not optimal.
Revised: Moreover, SubgraphX is the only baseline that has better H-Fidelity than the “ground truth”, but itcan only capture one -NO2 because its search algorithm requires the explanation to be connected,so its Inv-Fidelity is not optimal.

Original: In fact, GNNExplainer, PGExplainer, and SubgraphX can nevergenerate explanations including only disconnected -O without -N like GStarX, because the formertwo solve the explanation problem by optimizing edges (as opposed to Equation 4), and the latterrequires connectedness.
Revised: In fact, GNNExplainer, PGExplainer, and SubgraphX can nevergenerate explanations including only disconnected -O without -N like GStarX, because the formertwo solve the explanation problem by optimizing edges (as opposed to Equation 4), and the latterrequires connectedness.

Original: More MUTAG explanation visualizations are in Appendix H.
Revised: More MUTAG explanation visualizations are in Appendix H.

Original: Explanation sparsity study.
Revised: 5.4 Model-agnostic explanation

Original: To further study whether the obtained scores by GStarX are sparse,we follow [25] to evaluate an entropy-based sparsity measure on model output scores. Weshow the average GStarX entropy-based sparsity on all datasets, and compare them with threereference score distributions on all n nodes in a graph.
Revised: GStarX makes no assumptions about the model architecture and can be applied to various GNNbackbones. We use GCN for all datasets in our major experiment for consistency, and now furtherinvestigate performance on two more popular GNNs: GIN and GAT.

Original: An upper bound: Uniform(n),which represents the least sparse output. A practical lower bound: Uniform(0.25*n) whichrepresents very sparse outputs with only top 25% of nodes. 3) Poisson(0.25*n), which is amore realistic version of case 2).
Revised: We follow [5] to train GIN on

Original: Results in Table 2 show the average entropy-based sparsity of
Revised: MUTAG and GAT on GraphSST2 3 , and show results in Table 2.

Original:  GStarX is close to Poisson(0.25*n), which justifies the GStarX outputs are indeed sparse.
Revised: For both settings, GStarX outperformsthe baselines, which is consistent with results on GCN.

Original: A moredetailed discussion of this metric and these three reference distributions is in Appendix A.5.
Revised: The GStarX algorithm scales in O ( J ) with practical J ∝ |V| . Following [5], we study the empiricalefficiency of GStarX by explaining 50 randomly selected graphs from BBBP . We report the average run

Revised: As some take we analysis.

Original: [Table]
Revised: [Table]

Original: We conduct more studies to further evaluate GStarX in Appendix.
Revised: [Table] time in Table 3.

Original: In Appendix A.6, we show GStarXis model-agnostic and can be used to explain different GNN architectures. In Appendix A.7, we show GStarX is efficient enough for practical usage.
Revised: Our result on the baselines is similar to [5]. GStarX is not the fastest method, but it ismore than two times faster than SubgraphX. Since explanation usually doesn’t have strict efficiencyrequirements in real applications, considering GStarX generates higher-quality explanations than thebaselines, we believe the time complexity of GStarX is acceptable.

Original:  Related work
Revised:  Related work

Original: GNN explanation aims to produce an explanation for GNN predictions on a given graph, usually asa subgraph induced by important nodes or edges.
Revised: GNN explanation aims to produce an explanation for GNN predictions on a given graph, usually asa subgraph induced by important nodes or edges.

Original: Many existing methods work by scoring nodes oredges and are thus similar to this work.
Revised: Many existing methods work by scoring nodes oredges and are thus similar to this work.

Original: For example, the scoring function of GNNExplainer [14] isthe mutual information between a masked graph and the prediction on the original graph, where a softmask on edges and node features is generated by direct parameter learning.
Revised: For example, the scoring function of GNNExplainer [14] isthe mutual information between a masked graph and the prediction on the original graph, where a softmask on edges and node features is generated by direct parameter learning.

Original: PGExplainer [15] usesthe same scoring function but generates a discrete mask on edges by training an edge mask predictor.
Revised: PGExplainer [15] usesthe same scoring function but generates a discrete mask on edges by training an edge mask predictor.

Original: SubgraphX [5] uses the Shapley value as its scoring function on subgraphs selected by Monte Carlo
Revised: SubgraphX [5] uses the Shapley value as its scoring function on subgraphs selected by Monte Carlo

Original: Tree Search (MCTS), and GraphSVX [4] uses a least-square approximation to the Shapley value toscore nodes and their features.
Revised: Tree Search (MCTS), and GraphSVX [4] uses a least-square approximation to the Shapley value toscore nodes and their features.

Original: While SubgraphX and GraphSVX were shown to perform better thanprior alternatives, as we show in Section 3, the Shapley value they try to approximate is non-idealas it is non-structure-aware.
Revised: While SubgraphX and GraphSVX were shown to perform better thanprior alternatives, as we show in Section 3, the Shapley value they try to approximate is non-idealas it is non-structure-aware.

Original: Although SubgraphX and GraphSVX use L -hop subgraphs and thustechnically they use the graph structure, we discuss why this approach has limitations in achievingstructure-awareness in Appendix G. While there are many other GNN explanation methods from verydifferent perspectives, i.e. gradient analysis [26], model decomposition [27], surrogate models [28],and causality [29, 24], we defer their details to Appendix C given their lesser relevance.
Revised: Although SubgraphX and GraphSVX use L -hop subgraphs and thustechnically they use the graph structure, we discuss why this approach has limitations in achievingstructure-awareness in Appendix G. While there are many other GNN explanation methods from verydifferent perspectives, i.e. gradient analysis [25], model decomposition [26], surrogate models [27],and causality [28, 24], we defer their details to Appendix C given their lesser relevance.

Original: Cooperative game theory originally studies how to allocate payoffs among a set of players in acooperative game.
Revised: Cooperative game theory originally studies how to allocate payoffs among a set of players in acooperative game.

Original: Recently, certain ideas from this domain have been successfully used in featureimportance scoring for ML model explanation [2, 30, 3].
Revised: Recently, certain ideas from this domain have been successfully used in featureimportance scoring for ML model explanation [2, 29, 3].

Original: When used for explanation, the players ofthe game are the features, e.g. pixels for images.
Revised: When used for explanation, the players ofthe game are the features, e.g. pixels for images.

Original: The value of the game is allocated to the features asimportance scores in the model explanation.
Revised: The value of the game is allocated to the features asimportance scores in the model explanation.

Original: The vast majority of works in this line, like the ones citedabove, deem the Shapley value [1] to be the only choice.
Revised: The vast majority of works in this line, like the ones citedabove, deem the Shapley value [1] to be the only choice.

Original: In fact, there are many other values withdifferent properties and used in different situations in cooperative game theory.
Revised: In fact, there are many other values withdifferent properties and used in different situations in cooperative game theory.

Original: However, to the bestof our knowledge, only [31] mentions the Myerson value [9] in the context of proposing a connected
Revised: However, to the bestof our knowledge, only [30] mentions the Myerson value [9] in the context of proposing a connected

Original: Shapley (C-Shapley) value for explaining sequence data, so it is not directly comparable to ours.
Revised: Shapley (C-Shapley) value for explaining sequence data, so it is not directly comparable to ours.

Original: Adetailed discussion of the Myerson value and the C-Shapley value can be found in Appendix F. Inthis work, we follow the cooperative game theory approach to explain models on graph data using anew HN value [6].
Revised: Adetailed discussion of the Myerson value and the C-Shapley value can be found in Appendix F. Inthis work, we follow the cooperative game theory approach to explain models on graph data using anew HN value [6].

Original: We show it is a better choice than the Shapley value as it is structure-aware.
Revised: We show it is a better choice than the Shapley value as it is structure-aware.

Original:  Conclusion and future work
Revised:  Conclusion and future work

Original: In summary, we study GNN explanation on graphs via node importance scoring.
Revised: In summary, we study GNN explanation on graph data via feature importance scoring.

Original: We identify thenon-structure-aware challenge of existing Shapley-value-based approaches and propose GStarX toassign importance scores to each node via a structure-aware HN value.
Revised: In particular,we identify the non-structure-aware challenge of existing Shapley-value-based approaches andpropose GStarX to assign importance scores to each node via a structure-aware HN value.

Original: We also build connectionsbetween theHN value surplus allocation  andGNN message passing and demonstrate the superiorperformance of GStarX over strong baselines on chemical  and text graph classifications. A limitationof GStarX is that the importance of different node feature dimensions is not explained.
Revised: Webuild connections between the surplus allocation in HN value and message passing in GNNs anddemonstrate the superior performance of GStarX over strong baselines on chemical graph propertyprediction and text graph sentiment classification.

Original: Onefuture work is to add this extension, which may be done by scoring a subset of nodes togetherwith a subset of features each time.

Original: Another future direction is to exploit the rich  cooperative gametheory literature. Beyond the Shapley value, more values are possible for explaining ML models.
Revised: As there is a rich literature of cooperative gametheory beyond the Shapley value, more values are possible for explaining ML models.

Original: Forgraph data, edge based values can potentially be applied to an alternative edge-based objective like Equation 4.
Revised: For graph data,edge based values can potentially be applied to an alternative edge-based objective like Equation 4.

Original: Other values may be appropriate to more data types beyond graphs.
Revised: Other values may be appropriate to more data types beyond graphs.

Revised: We leave these as future work.

Original: Checklist
Revised: Checklist

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
Revised: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes]
Revised: [Yes]

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [No] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: [No](d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?

Original: [Yes]
Revised: [Yes]

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [N/A]
Revised: [N/A]

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?
Revised: mental results (either in the supplemental material or as a URL)?

Original: [Yes](b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
Revised: [Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes]
Revised: [Yes]

Original: (c) Did you report error bars (e
Revised: (c) Did you report error bars (e

Original: g., with respect to the random seed after running experiments multiple times)?
Revised: g., with respect to the random seed after running experiments multiple times)?

Original: [Yes]
Revised: [Yes]

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes]
Revised: [Yes]

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
Revised: If you are using existing assets (e g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: Yes
Revised: [Yes]

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No]
Revised: [No]

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?
Revised: using/curating?

Original: [N/A]
Revised: [N/A]

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable
Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable

Original: information or offensive content?
Revised: information or offensive content?

Original: [N/A]
Revised: [N/A]

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, if
Revised: (a) Did you include the full text of instructions given to participants and screenshots, if

Original: applicable?
Revised: applicable?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?
Revised: Board (IRB) approvals, if applicable?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount
Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount

Original: spent on participant compensation?
Revised: spent on participant compensation?

Original: [N/A]
Revised: [N/A]

Original: A Experiment detailsA.1 Datasets
Revised: A Experiment details

Revised: A.1 Datasets

Original: In Table 3, we provided the basic statistics of datasets used in our experiments.
Revised: In Table 4, we provided the basic statistics of datasets used in our experiments.

Original: [Table]
Revised: [Table]

Original: A.2 Model architectures and implementation
Revised: A.2 Model architectures and implementation

Original: In Table 4, we provided the hyperparameters and test accuracy for the GCN model used in our majorexperiments.
Revised: In Table 5, we provided the hyperparameters and test accuracy for the GCN model used in our majorexperiments.

Original: In Table 6, we provided the hyperparameters and test accuracy for the GIN and GATmodel used in our analysis experiment.
Revised: In Table 2, we provided the hyperparameters and test accuracy for the GIN and GATmodel used in our analysis experiment.

Original: Most parameters are following [5], with small changes tofurther boost the test accuracy.
Revised: Most parameters are following [5], with small changes tofurther boost the test accuracy.

Original: We run all experiments on a machine with 80 Intel(R)
Revised: We run all experiments on a machine with 80 Intel(R)

Original: Xeon(R) E5-2698 v4 @
Revised: Xeon(R) E5-2698 v4 @

Original: 2.20GHz CPUs, anda single NVIDIA
Revised: 2.20GHz CPUs, anda single NVIDIA

Original: V100 GPU with 16GB RAM.
Revised: V100 GPU with 16GB RAM.

Original: Our implementations are based on Python 3.8.10,
Revised: Our implementations are based on Python 3.8.10,

Original: PyTorch 1.10.0, PyTorch-Geometric 1.7.1 [32], and DIG [33].
Revised: PyTorch 1.10.0, PyTorch-Geometric 1.7.1 [31], and DIG [32].

Original: We adapt the GNN implementationand most baseline explainer implementation from the DIG library, except GraphSVX where we usethe official implementation.
Revised: We adapt the GNN implementationand most baseline explainer implementation from the DIG library, except GraphSVX where we usethe official implementation.

Original: For the baseline hyperparameters, we closely follow the setting in [5]and [4] for a fair comparison.
Revised: For the baseline hyperparameters, we closely follow the setting in [5]and [4] for a fair comparison.

Original: Please refer to [5] Section 4.1 and [4] Appendix E for details.
Revised: Please refer to [5] Section 4.1 and [4] Appendix E for details.

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: A.3 Exact formula for evaluation metrics
Revised: A.3 Exact formula for evaluation metrics

Original: We showed the formula of Fidelity , Inv-Fidelity , and Sparsity in Equation 9, 10, and 11.
Revised: We showed the formula of Fidelity , Inv-Fidelity , and Sparsity in Equation 9, 10, and 11.

Original: In Equation12, 13, and 14, we show the formula for normalized fidelity ( N-Fidelity ), normalized inverse fidelity( N-Inv-Fidelity ), and harmonic fidelity ( H-Fidelity ).
Revised: In Equation12, 13, and 14, we show the formula for normalized fidelity ( N-Fidelity ), normalized inverse fidelity( N-Inv-Fidelity ), and harmonic fidelity ( H-Fidelity ).

Original: Both the N-Fidelity and N-Inv-Fidelity arein [ − 1 , 1] .
Revised: Both the N-Fidelity and N-Inv-Fidelity arein [ − 1 , 1] .

Original: The H-Fidelity flips N-Inv-Fidelity , rescales both values to be in [0 , 1] , and  theirharmonic mean.
Revised: The H-Fidelity flips N-Inv-Fidelity , rescales both values to be in [0 , 1] , and takes theirharmonic mean.

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Let m 1 = N-Fidelity ( G , g ) , m 2 = N-Inv-Fidelity ( G , g ) 
Revised: Let m 1 = N-Fidelity ( G , g ) , m 2 = N-Inv-Fidelity ( G , g ) H-Fidelity ( [Equation] [Equation]

Revised: =

Original: [Equation]
Revised: [Equation]

Revised: m

Original: [Equation]
Revised: [Equation]

Original: A.4 Fidelity vs. sparsity plots
Revised: A.4 Fidelity vs. sparsity plots

Original: In Table 1, we reported the best H-Fidelity among 8 different sparsities for each method.
Revised: In Table [Figure] we reported the best H-Fidelity among 8 different sparsities for each method.

Original: We alsofollow [5] to show the Fidelity vs. Sparsity plots in Figure 4.
Revised: We alsofollow [5] to show the Fidelity vs. Sparsity plots in Figure 4.

Original: Note that GraphSVX tends to give sparseexplanations on some datasets, we still pick 8 different sparsities for it but mostly on the higher end.
Revised: Note that GraphSVX tends to give sparseexplanations on some datasets, we still pick 8 different sparsities for it but mostly on the higher end

Original: [Figure]
Revised: [Figure]

Original: corresponding to the results shown in Table 1.
Revised: corresponding to the results shown in Table 1.

Original: All three metrics are the higher the better.
Revised: three metrics are the higher the better.

Original: We see that
Revised: We see that

Original: GStarX outperforms the other methods
Revised: GStarX outperforms the other methods

Original: A.5 Detailed entropy-based sparsity evaluation

Original: In Section 5.2 we study whether the obtained scores by GStarX are sparse and follow [25] to apply anentropy-based sparsity measure on scores.

Original: We now provide are more detailed discussion of this study.

Original: The entropy-based sparsity, as defined in Definition 2 in [25], is shown in the Equation 15 below.

Original: Here ϕ is the model output scores for an instance, and ˜ ϕ i

Original: = ϕ

Original: i (cid:80) i ϕ

Original: i represent normalized scores.

Original: [Equation]

Original: The entropy-based sparsity helps us understand how sparse an explanation is, before the scores areturned into hard explanation by thresholding or selecting top k. In Table 2, we show the averagescores for GStarX on all datasets, and compare them with three reference cases.

Original: The entropy ofuniform distribution over all n nodes in a graph, i.e., Uniform(n), which represents the least sparse

Original: [Table]

Original: [Table]

Original: output and is an upper bound of entropy-based sparsity. 2)

Original: The entropy of uniform distribution overthe top 25% nodes in a graph, i.e., Uniform(0.25*n), where probabilities of the bottom 75% nodesare set to zero.

Original: This case is very sparse since 75% of nodes are deterministically excluded, which canbe treated as a practical lower bound of entropy-based sparsity.

Original: The entropy of Poisson distributionwith mean 0.25*n, i.e. Poisson(0.25*n), which is a more realistic version of the sparse output incase 2).

Original: Instead of setting all 75% of nodes to have probability zero, we assume the probabilitiesfor tail nodes decrease exponentially as a Poisson distribution while the mean is kept the same asin case 2).

Original: Results in Table 2 show that the average entropy-based sparsity of GStarX is between

Original: Uniform(0.25*n) and Uniform(n) and close to Poisson(0.25*n), which justifies the GStarX outputsare indeed sparse.

Original: A.6 Model-agnostic explanation

Original: GStarX makes no assumptions about the model architecture and can be applied to various GNNbackbones.

Original: We use GCN for all datasets in our major experiment for consistency, and now furtherinvestigate performance on two more popular GNNs: GIN and GAT.

Original: We follow [5] to train GIN on

Original: MUTAG and GAT on GraphSST2 3 , and show results in Table 6.

Original: For both settings, GStarX outperformsthe baselines, which is consistent with results on GCN.

Original: A.7 Efficiency study

Original: The GStarX algorithm scales in O ( J ) with practical J ∝ |V| .

Original: Following [5], we study the empiricalefficiency of GStarX by explaining 50 randomly selected graphs from BBBP .

Original: We report the average runtime in Table 7.

Original: Our result on the baselines is similar to [5].

Original: GStarX is not the fastest method, but it ismore than two times faster than SubgraphX. Since explanation usually doesn’t have strict efficiencyrequirements in real applications, considering GStarX generates higher-quality explanations than thebaselines, we believe the time complexity of GStarX is acceptable.

Original: B GStarX for node classification
Revised: B GStarX for node classification

Original: Eventhough the GStarX algorithm is stated for graph classification, it works for node classificationas well.
Revised: Even the GStarX algorithm is stated for graph classification, it works for node classification as well.

Original: This can be easily seen as the GNN node classification can be covert to classify an ego-graph.
Revised: This can be easily seen as the GNN node classification can be covert to classify an ego-graph.

Original: Given a graph G with V = { u 1 , . . .
Revised: Givena graph G with V = { u 1 , . . .

Original: , u n } .
Revised: , u n } .

Original: Node classification on u i with an L -layer GNN can beconverted to a graph classification.
Revised: Node classification on u i with an L -layer GNN can be convertedto a graph classification.

Original: The target graph to classify will be the L -hop ego-graph centeredat u i , because this is the receptive field of the GNN for classifying u
Revised: The target graph to classify will be the L -hop ego-graph centered at u i ,because this is the receptive field of the GNN for classifying u

Original: i and nodes further away won’tinfluence the result.
Revised: i and nodes further away won’t haveinfluence on the result.

Original: The label of the graph will be the label of u i .
Revised: The label of the graph will be the label of u i .

Original: In this case, the final readout layerof the GNN will be indexing u i instead of pooling.
Revised: In this case, the final readoutlayer of the GNN will be indexing u i instead of pooling.

Original: Given this kind of conversion, everything weshowed in Section 4 follows.
Revised: Given this kind of conversion, everythingwe showed in Section 4 follows.

Revised: We conduct experiments on a standard node classification classification benchmark.

Original: C More related work
Revised: With datasetdescription shown below and results shown in Table 7. BAShape [14] is a synthetic dataset similar to BA2Motifs but for node classification. It has one graphcontaining a 300-node BA graph and 80 5-node motifs randomly attached to the BA graph. Nodefeatures are 10-dimensional all-1 vectors.

Revised: Each node in BAShape is labeled with its the membershipof a motif and the location within the motif.

Original:  GNN explanation continued Besides the perturbation-based method we mentioned in Section6, there are several other types of approaches for GNN explanation.
Revised: [Table] C More related work [Table] GNN explanation continued Besides the perturbation-based method we mentioned in Section6, there are several other type of approaches for GNN explanation.

Original: Gradient-based methods arewidely used for explaining ML models on images and text.
Revised: Gradient-based methods arewidely used for explaining ML models on images and text.

Original: The key idea is to use the gradients as the approximations of input importance.
Revised: The key idea is to use the gradientsas the approximations of input importance.

Original: Such methods as contrastive gradient-based (CG) saliency maps, Class Activation Mapping (CAM), and gradient-weighted CAM (Grad-CAM) have been generalizedto graph data in [26].
Revised: Such methods like contrastive gradient-based (CG)saliency maps, Class Activation Mapping (CAM), and gradient-weighted CAM (Grad-CAM) havebeen generalized to graph data in [25].

Original: Decomposition-based methods are a popular way to explain deep NNs forimages.
Revised: Decomposition based methods are a popular way to explaindeep NNs for images.

Original: They measure the importance of input features by decomposing the model predictions andregard the decomposed terms as importance scores.
Revised: They measure the importance of input features by decomposing the modelpredictions and regard the decomposed terms as importance scores.

Original: Decomposition methods includingLayer-wise
Revised: Decomposition methods including

Original:  Relevance Propagation (LRP) and Excitation Backpropagation (EB) have also been extended to graphs[26, 27].
Revised: Layer-wise Relevance Propagation (LRP) and Excitation Backpropagation (EB) have also beenextended to graphs [25, 26].

Original: Surrogate-based methods work by approximating a complex model using an explainablemodel locally.
Revised: Surrogate-based methods work by approximating a complex modelusing an explainable model locally.

Original: Possible options to approximate GNNs include linear model as in GraphLIME [34],additive feature attribution model with the Shapley value as in GraphSVX [4], and Bayesian networksas in [28].
Revised: Possible options to approximate GNNs include linear modelas in GraphLIME [33], additive feature attribution model with the Shapley value as in GraphSVX[4], and Bayesian networks as in [27].

Original: GNN explainability has also been studied from the causal perspective.
Revised: GNN explainability has also been studied from the causalperspective.

Original: In [29, 24],generative models were constructed to learn causal factors, and explanations were produced byanalyzing the cause-effect relationship in the causal graph.
Revised: In [28, 24], generative models were constructed to learn causal factors, and explanationswere produced by analyzing the cause-effect relationship in the causal graph.

Original: D Properties of the Shapley value
Revised: D Properties of the Shapley value

Original: The Shapley value was proposed as the unique solution of a game ( N, v ) that satisfies three propertiesshown below, i.e. efficiency , symmetry , and additivity [1].
Revised: The Shapley value was proposed as the unique solution of a game ( N, v ) that satisfies three propertiesshown below, i.e. efficiency , symmetry , and additivity [1].

Original: These three properties together arereferred as an axiomatic characterization of the Shapley value.
Revised: These three properties together arereferred as an axiomatic characterization of the Shapley value.

Original: The associated consistency propertiesintroduced in Section 4.1 provides a different axiomatic characterization.
Revised: The associated consistency propertiesintroduced in Section 4.1 provides a different axiomatic characterization.

Original: Property D.1 ( Efficiency ) .
Revised: Property D.1 ( Efficiency ) .

Original: [Equation]
Revised: [Equation]

Original: Property D.2 ( Symmetry ) .
Revised: Property D.2 ( Symmetry ) .

Original: If v ( S ∪ { i } )
Revised: If v ( S ∪ { i } )

Original: = v ( S ∪ { j } ) for all S ∈ N \{ i, j } , thenϕ i ( N, v ) =
Revised: = v ( S ∪ { j } ) for all S ∈ N \{ i, j } , thenϕ i ( N, v ) =

Original: ϕ j ( N, v )
Revised: ϕ j ( N, v )

Original: Property D.3 ( Additivity ) .
Revised: Property D.3 ( Additivity ) .

Original: Given two games ( N, v ) and ( N, w ) ,
Revised: Given two games ( N, v ) and ( N, w ) ,

Original: [Equation]
Revised: [Equation]

Original: The efficiency property states that the value should fully distribute the payoffof the game.
Revised: The efficiency property states that the value should fully distribute the payoff the game.

Original: Thesymmetry property states that if two players make equal contributions to all possible coalitionsformed by other players (including the empty coalition), then they should have the same value.
Revised: The symmetryproperty states that if two players make equal contributions to all possible coalitions formed by otherplayers (including the empty coalition), then they should have the same value.

Original: Theadditivity property states that the value of two independent games should be added player by player.
Revised: The additivity propertystates that the value of two independent games should be added player by player.

Original: It is the most useful for a system of independent games.
Revised: It is the most usefulfor a system of independent games.

Original: E More about the HN valueE.1 Consistency and associated games
Revised: E More about the HN valueE.1 Consistency and associated games

Original: One reason for the Shapley value’s popularity is its axiomatic characterization , indicating that it isthe unique solution that satisfies a set of desirable properties (see Appendix D).
Revised: One reason for the Shapley value’s popularity is its axiomatic characterization , indicating that it isthe unique solution which satisfies a set of desirable properties (see Appendix D).

Original: Then [35] proposeda new axiomatic characterization of the Shapley value based on a different associated consistencyproperty.
Revised: Then [34] proposeda new axiomatic characterization of the Shapley value based on a different associated consistencyproperty.

Original: The consistency property is a common analysis tool used in game theory [36, 37, 38, 39].
Revised: The consistency property is a common analysis tool used in game theory [35, 36, 37, 38].

Original: The idea is to analyze a game ( N, v ) by defining other reduced games ( S, v S ) for S ⊆ N , and asolution function ϕ is called consistent when ϕ ( N, v ) yields the same payoff as ϕ ( S, v S ) on each S .
Revised: The idea is to analyze a game ( N, v ) by defining other reduced games ( S, v S ) for S ⊆ N , and asolution function ϕ is called consistent when ϕ ( N, v ) yields the same payoff as ϕ ( S, v S ) on each S .

Original: When ( S, v S ) is defined with desired properties, these good properties can be enforced for a solutionby requiring consistency.
Revised: When ( S, v S ) is defined with desired properties, these good properties can be enforced for a solution by requiring consistency.

Original: The associated consistency in [35] is a special case of consistency between( N, v ) and only one other game ( N, v ∗ ) , which is called the associated game .
Revised: The associated consistency in [34] is a special case of consistency between( N, v ) and only one other game ( N, v ∗ ) , which is called the associated game .

Original: [35] shows that acarefully designed associated game uniquely characterizes the Shapley value.
Revised: [34] shows that acarefully designed associated game uniquely characterizes the Shapley value.

Original: Associated consistencyis also the key idea of the HN value.
Revised: Associated consistencyis also the key idea of the HN value.

Original: E.2 Limit game and the axiomatic characterization
Revised: E.2 Limit game and the axiomatic characterization

Original: The HN value is established on a special associated game as we discussed in Section 4.1.
Revised: The HN value is established on an special associated game as we discussed in Section 4.1.

Original: We canactually write this associated game in a more compact matrix form, where we slightly abuse notationand use v and v ∗ τ to represent vectors of payoffs for all S ⊆ N under the original and associatedgame respectively.
Revised: We canactually write this associated game in a more compact matrix form, where we slightly abuse notationand use v and v ∗ τ to represent vectors of payoffs for all S ⊆ N under the original and associatedgame respectively.

Original: In other words, v ( S ) , whichis used to represent evaluating the coalition S the characteristic indexing the vector v  index 
Revised: In other words, v ( S ) , which used to represent evaluating the coalition S using thecharacteristic function v , now can also be interpreted as indexing the vector v with index S .

Original: Lemma E.1.
Revised: Lemma E.1.

Original: A matrix form of the associated game ( N, v ∗ τ , G ) is given by
Revised: A matrix form of the associated game ( N, v ∗ τ , G ) is given by

Original: [Equation]
Revised: [Equation]

Original: The matrix H { τ,n, G} depends on the hyperparameter τ , number of players n , and the graph G .
Revised: The matrix H { τ,n, G} depends on the hyperparameter τ , number of players n , and the graph G .

Original: Whenthese variables are clear from the context, we drop them and write v ∗ τ
Revised: Whenthese variables are clear from the context, we drop them and write v ∗ τ

Original: = H v .
Revised: = H v .

Original: Please refer to [6] forthe proof of Lemma E.1.
Revised: Please refer to [6] forthe proof of Lemma E.1.

Original: With the matrix form, we can define the limit game.
Revised: With the matrix form, we can define the limit game.

Original: Definition E.2.
Revised: Definition E.2.

Original: Given a game ( N, v, G ) , its limit game ( N, ˜ v, G ) is defined by
Revised: Given a game ( N, v, G ) , its limit game ( N, ˜ v, G ) is defined by

Original: [Equation]
Revised: [Equation]

Original: Notice that although the matrix H is constructed from the associated game and depends on τ , thepowers of H actually converge to a limit independent from τ , when τ is sufficiently small.
Revised: Notice that although the matrix H is constructed from the associated game and depends on τ , thepowers of H actually converges to a limit independent from τ , when τ is sufficiently small.

Original: Thegeneral condition depends on the actual graph, but 0 < τ < 2 n is proven to be sufficient for thecomplete graph case [35].
Revised: Thegeneral condition depends on the actual graph, but 0 < τ < 2 n is proven to be sufficient for thecomplete graph case [34].

Original: As we discussed in Section 4.1, the limit game can be seen as constructingassociated games repeatedly until the characteristic function converges.
Revised: As we discussed in Section 4.1, the limit game can be seen as constructingassociated games repeatedly until the characteristic function converges.

Original: An axiomatic characterization of the HN value regarding its uniqueness is given by the followingtheorem based on the limit game.
Revised: An axiomatic characterization of the HN value regarding its uniqueness is given by the followingtheorem based on the limit game.

Original: The associated consistency is the core property related to this work.
Revised: The associated consistency is the core property related to this work.

Original: We encourage the readers to check [6] for the other two properties.
Revised: We encourage the readers to check [6] for the other two properties.

Original: Theorem E.3.
Revised: Theorem E.3.

Original: There exists a unique solution ϕ that verifies the associated consistency, i.e.ϕ i ( N, v, G ) =
Revised: There exists a unique solution ϕ that verifies the associated consistency, i.e.ϕ i ( N, v, G ) =

Original: ϕ
Revised: ϕ

Original: i ( N, v ∗ τ , G ) , inessential game, and continuity.
Revised: i ( N, v ∗ τ , G ) , inessential game, and continuity.

Original: ϕ is given byϕ i ( N, v, G ) =
Revised: ϕ is given byϕ i ( N, v, G ) =

Original: ˜ v ( { i } ) (18)
Revised: ˜ v ( { i } ) (17)

Original: E.3 The algorithm for computing the HN value
Revised: E.3 The algorithm for computing the HN value

Original: We show the algorithm for Compute-HN-MC (Algorithm 3) mentioned in Section 4.3.
Revised: We show the algorithm for Compute-HN-Values (Algorithm 2) and Compute-HN-Values-MC (Algorithm 3) mentioned in Section 4.3.

Original: The algorithmis a combination of Equation  16, 17, and 8.
Revised: The algorithm is a combination of Equation 15, 16, and8.

Original: Input: Graph instance G with nodes V = { u 1 , . . . , u n } , characteristic function v , hyperparameter τ , maximum sample size m , number of samples J
Revised: Input: Graph instance G with nodes V = { u 1 , . . . , u n } , characteristic function v , hyperparameter τ , maximum sample size m , number of samples J

Original: Let ψ 1 , . .
Revised: Let ψ 1 , . .

Original: . , ψ n be n empty lists for j = 1 to J do
Revised: . , ψ n be n empty lists for j = 1 to J do

Original: Sample g S j from G s.t.
Revised: Sample g S j from G s.t.

Original: S
Revised: S

Original: j = { u j 1 , . . . , u j l } and l < m ϕ j = Compute-HN ( g S j , S j , v ( · ) , τ ) for k = 1 to l do
Revised: j = { u j 1 , . . . , u j l } and l < m ϕ j = Compute-HN-Values ( g S j , S j , v ( · ) , τ ) for k = 1 to l do

Original: Append ϕ
Revised: Append ϕ

Original: j k to ψ j k end for end for
Revised: j k to ψ j k end for end for

Original: Set ϕ
Revised: Set ϕ

Original: i to be the mean of ψ
Revised: i to be the mean of ψ

Original: i
Revised: i

Original: Return: ϕ
Revised: Return: ϕ

Original: F The Myerson value and the C-Shapley value
Revised: Input: Graph instance G with nodes V = { u 1 , . . . , u n } , characteristic function v , hyperparameter τ for S in 2 N do Compute payoff v ( S ) {Equation 8} end for Construct matrix H from τ , n , and G {Equation 15} repeat H = HH until H converges

Revised: Get the limit game ˜ v

Original: F.1 The Myerson value
Revised: = H v e {Equation 16} Assign the first n entries of ˜ v to ϕ Return: ϕ F The myerson value and the C-Shapley valueF.1 The Myerson value

Original: In the study of cooperative games, [9] proposed to characterize the cooperation possibilities betweenplayers using a graph structure G , which leads to the communication structure introduced in Section2.2 and the Myerson value as a solution for this special type of games ( N, v, G ) .
Revised: In the study of cooperative games, [9] proposed to characterize the cooperation possibilities betweenplayers using a graph structure G , which leads to the communication structure introduced in Section2.2 and the Myerson value as a solution for this special type of games ( N, v, G ) .

Original: The Myerson valueis closely related to the Shapley value.
Revised: The Myerson valueis closely related to the Shapley value.

Original: In fact, it is the Shapley value on a transformed game whereplayers are partitioned by the graph.
Revised: In fact, it is the Shapley value on a transformed game whereplayers are partitioned by the graph.

Original: We now formally introduce the partition and the transformedgame.
Revised: We now formally introduce the partition and the transformedgame.

Original: Definition F.1 ( Partition ) .
Revised: Definition F.1 ( Partition ) .

Original: Given a set of players N and a graph G .
Revised: Given a set of players N and a graph G .

Original: For any coalition S ⊆ N , thepartition of S is denoted by S/ G and defined by
Revised: For any coalition S ⊆ N , thepartition of S is denoted by S/ G and defined by

Original: S/ G = {{ i | i and j are connected in S by G}| j ∈ S } and a member of the set S/ G is called a component of S .
Revised: S/ G = {{ i | i and j are connected in S by G}| j ∈ S } and a member of the set S/ G is called a component of S .

Original: Definition F.2 ( Transformed Game ) .
Revised: Definition F.2 ( Transformed Game ) .

Original: Given a game ( N, v, G ) , we can transform it to a new gamev/ G such that for all S ⊆ N
Revised: Given a game ( N, v, G ) , we can transform it to a new gamev/ G such that for all S ⊆ N

Original: [Equation]
Revised: [Equation]

Original: Intuitively, given a coalition S , the transformed game treats each connected component of S asindependent, evaluates them separately, and sums their payoff as the payoff of S .
Revised: Intuitively, given a coalition S , the transformed game treats each connected component of S asindependent, evaluates them separately, and sums their payoff as the payoff of S .

Original: The Shapley value has an axiomatic characterization that uniquely determines it as we introducedin Appendix D. Likewise, the Myerson value was proposed to be a unique solution that satisfies thecomponent efficiency and the fairness property defined below.
Revised: The Shapley value has a axiomatic characterization that uniquely determines it as we introduced in Appendix D. Likewise, the Myerson value was proposed to be a unique solution that satisfies thecomponent efficiency and the fairness property defined below.

Original: Property F.3 ( Component Efficiency ) .
Revised: Property F.3 ( Component Efficiency ) .

Original: For a game ( N, v, G ) and any connected component S ∈
Revised: For a game ( N, v, G ) and any connected component S ∈

Original: N/ G , a solution is component efficient if
Revised: N/ G , a solution is component efficient if

Original: [Equation]
Revised: [Equation]

Original: Property F.4 ( Fairness ) .
Revised: Property F.4 ( Fairness ) .

Original: For a game ( N, v, G ) and any edge ( i, j ) in G , let ˜ G be G with the edge( i, j ) removed, a solution is fair if
Revised: For a game ( N, v, G ) and any edge ( i, j ) in G , let ˜ G be G with the edge( i, j ) removed, a solution is fair if

Original: [Equation]
Revised: [Equation]

Original: The component efficiency property is an extension of the regular efficiency property to games with acommunication structure.
Revised: The component efficiency property is an extension of the regular efficiency property to games with acommunication structure.

Original: It requires efficiency to hold for each disconnected piece because thesepieces are assumed as independent from each other.
Revised: It requires efficiency to hold for each disconnected piece because thesepieces are assumed as independent from each other.

Original: The fairness property states that if breaking anedge ( i, j ) changes the value of player i , then the value of player j should be changed by the sameamount.
Revised: The fairness property states that if breaking anedge ( i, j ) changes the value of player i , then the value of player j should be changed by the sameamount.

Original: Theorem F.5 ( Myerson Value ) .
Revised: Theorem F.5 ( Myerson Value ) .

Original: There exists a unique solution ϕ of game ( N, v, G ) satisfyingcomponent efficiency and fairness.
Revised: There exists a unique solution ϕ of game ( N, v, G ) satisfyingcomponent efficiency and fairness.

Original: With ˜ ϕ represents the Shapley value, the solution is given by theformula
Revised: With ˜ ϕ represents the Shapley value, the solution is given by theformulaϕ ( N, v, [Equation] ( N [Equation]

Revised: ( v/ G

Original: [Equation]
Revised: [Equation]

Original: For games with a communication structure, the Myerson value is a better choice than the Shapleyvalue as it uses the graph structure.
Revised: For games with a communication structure, the Myerson value is a better choice than the Shapleyvalue as it uses the graph structure.

Original: However, it also suffers from some criticisms.
Revised: However, it also suffer from some criticisms.

Original: For example, thefairness assumption may not be realistic.
Revised: For example, thefairness assumption may not be realistic.

Original: When an existing edge is broken, the value changes forplayers on the two edge ends can be asymmetric.
Revised: When an existing edge is broken, the value changes forplayers on the two edge ends can be asymmetric.

Original: Intuitively, if the edge connects a popular hubplayer i to a leaf player j , then the change of i can be less significant than j since j becomes isolatedwhen ( i, j ) is removed.
Revised: Intuitively, if the edge connects a popular hubplayer i to a leaf player j , then the change of i can be less significant than j since j becomes isolatedwhen ( i, j ) is removed.

Original: This is also the case when the game value is used for model explanation.
Revised: This is also the case when the game value is used for model explanation.

Original: Forexample in Figure 1 (b), when the edge ("good", "quite") is broken, the value of "quite" should changea lot.
Revised: Forexample in Figure 1 (b), when the edge ("good", "quite") is broken, the value of "quite" should changea lot.

Original: It used to contribute positively together with "good", and thus gets some payoff allocation, butit now becomes an isolated node, which is neutral by itself.
Revised: It used to contribute positively together with "good", and thus gets some payoff allocation, butit now becomes an isolated node, which is neutral by itself.

Original: On the other hand, the word "good" canstill contribute positively by itself and interact with other nodes through its other edges, and thus itsvalue shouldn’t change too much.
Revised: On the other hand, the word "good" canstill contribute positively by itself and interact with other nodes through its other edges, and thus itsvalue shouldn’t change too much.

Original: Because of such criticisms, we choose to use the HN value as ourscoring function, which characterizes the value by associated consistency rather than fairness.
Revised: Because of such criticisms, we choose to use the HN value as ourscoring function, which characterize the value by associated consistency rather than fairness.

Original: F.2 The C-Shapley value
Revised: F.2 The C_Shapley value

Original: The Myerson value was also mentioned in [31] forthe model explanation on text, where the CShapley value was proposedas an approximation of the Shapley value, and it was claimed to be equalto the Myerson value.
Revised: The Myerson value was also mentioned in [30] for model explanation on text, where the C_Shapleyvalue was proposed an approximation of the Shapley value, and it was claimed to be equal tothe Myerson value.

Original: We have discussed why Shapley value and Myerson are not-ideal choicesfor explaining graph data in Section 3 and Appendix F.1.
Revised: We have discussed why Shapley value and Myerson are not ideal choice forexplaining graph data in Section 3 and Appendix F.1.

Original: These are partially the reason why our HN-value-based method is better than the C-Shapley value.
Revised: These are partially the reason why our HNvalue based method is better than the C_Shapley value.

Original: However, the major reason why we don’tdo a direct comparison to the C-Shapley value as a baseline is that its formula only works for line graphs like sequence data, and not even all nodes in line graphs.
Revised: However, the major reason why we don’t dodirect comparison to the C_Shapley value as a baseline is because its formula only works for linegraphs like sequence data, and not even all nodes in line graphs.

Original: In contrast, our target task is generalgraph prediction for graphs with possibly complicated topological structures.
Revised: In contrast, our target task is generalgraph prediction for graph with possibly complicated topological structures.

Original: We now clarify a mistake of the C-Shapley value formula and explain why it won’t work for generalgraphs.
Revised: We now clarify a mistake of the C_Shapley value formula and explain why it won’t work for generalgraphs.

Original: The notations are following the [31], where d is the number of players corresponding to n inour notation, and [ d ] corresponding to N .
Revised: The notations are following the [30], where d is the number of players corresponding to n inour notation, and [ d ] corresponding to N .

Original: The formula for the C-Shapley value is given in Equation 6 in Definition 2 in the paper, and it isstated for "a graph G" without mentioning any assumptions of the graph.
Revised: The formula for the C_Shapley value is given in Equation 6 in Definition 2 in the paper, and it isstated for "a graph G" without mentioning any assumptions of the graph.

Original: However, from the proof ofthis formula in Appendix B.2 in the paper, the line graph assumption can be seen in two places.
Revised: However, from the proof ofthis formula in Appendix B.2 in the paper, the line graph assumption can be seen in two places.

Original: Thefirst place is Equation 20, where the set C is explicitly defined only for subsequences.
Revised: Thefirst place is Equation 20, where the set C is explicitly defined only for subsequences.

Original: The secondplace is Equation 22, the first line converts (cid:80) A : U S ( A )
Revised: The secondplace is Equation 22, the first line converts (cid:80) A : U S ( A )

Original: =
Revised: =

Original: U to (cid:80) d
Revised: U to (cid:80) d

Original: −| U |− 2 i = , which is implicitly saying
Revised: −| U |− 2 i = , which is implicitly saying

Original: V S ( A ) can be picked from all d but | U | − 2 nodes.
Revised: V S ( A ) can be picked from all d but | U | − 2 nodes.

Original: However, this conversion is only possible whenthere are exactly 2 edges between U and [ d ] \ U , i.e. the middle part of a line graph.
Revised: However, this conversion is only possible whenthere are exactly 2 edges between U and [ d ] \ U , i.e. the middle part of a line graph.

Original: If there are ledges between U and [ d ] \ U , then the summation should go up to d − | U | − l .
Revised: If there are ledges between U and [ d ] \ U , then the summation should go up to d − | U | − l .

Original: When l = 0 , i.e. Uequals [ d ] or a connected component of [ d ] , no partition is needed and the coefficient simply evaluatesto 1.
Revised: When l = 0 , i.e. Uequals [ d ] or a connected component of [ d ] , no partition is needed and the coefficient simply evaluatesto 1.

Original: By correcting all these cases, the final formula for the C-Shapley value coefficients of marginalcontributions thus becomes[Equation]
Revised: By correcting all these cases, the final formula for the C_Shapley value coefficients of marginalcontributions thus becomes

Original: [Equation]
Revised: [Equation]

Original: 21) for l > 0 , and 1 for l = 0 .
Revised: for l > 0 , and 1 for l = 0 .

Original: The correct formula for the C-Shapley value of general graphs will be
Revised: The correct formula for the C_Shapley value of general graphs will be

Original: [Equation]
Revised: [Equation]

Original: with l represents the edges between U and [ d ] \ U and C represents all connected subgraphs in [ d ]containing i .
Revised: with l represents the edges between U and [ d ] \ U and C represents all connected subgraphs in [ d ]containing i .

Original: Shapley
Revised: Shapley

Original: C_Shapley Myerson HN 1/ 1/ 1/ 1/ 1/ 1/ 1/  1/ 1/ 1/  1/ 1/ 1/ 1/
Revised: C_Shapley Myerson HN 1/ 1/ 1/ 1/ 1/ 1/ 1/  1/ 1/ 1/  1/ 1/ 1/ 1/

Original: Shapley
Revised: Shapley

Original: C_Shapley Myerson HN 1/ 1/ 1/ 1/ 1/ 1/ 1/ 1/ 1/    1/ 1/ 1/ 1/
Revised: C_Shapley Myerson HN 1/ 1/ 1/ 1/ 1/ 1/ 1/ 1/ 1/    1/ 1/ 1/ 1/

Original: To verify this formula with the 3-node toy graph in Figure 5.
Revised: To verify this formula with the 3-node toy graph in Figure 5.

Original: When computing the value of node 0(left), the three connected components containing 0 are C
Revised: When computing the value of node 0(left), the three connected components containing 0 are C

Original: = {{ 0 } , { 0 , 1 } , { 0 , 1 , 2 }} .
Revised: = {{ 0 } , { 0 , 1 } , { 0 , 1 , 2 }} .

Original: Since 0 is anend node and has no leaf nodes to its left, l for these three components will be 1, 1, and 0 respectively.
Revised: Since 0 is anend node and has no leaf nodes to its left, l for these three components will be 1, 1, and 0 respectively.

Original: According to our new formula in Equation 22, the coefficients will be 12 , 16 , and 13 respectively, withthe disconnected { 0 , 2 } case removed.
Revised: According to our new formula in Equation 21, the coefficients will be 12 , 16 , and 13 respectively, withthe disconnected { 0 , 2 } case removed.

Original: This matches the original idea of Myerson value, wherethe { 0 , 2 } − { 2 } case is reduced to the { 0 } − ∅ case, which turns the Shapley coefficients from [ 13 , 16 , 16 , 13 ] to [ 13 + 16 , 16 , 16 − 16 , 13 ] , which is [ 12 , 16 , 0 , 13 ] .
Revised: This matches the original idea of Myerson value, wherethe { 0 , 2 } − { 2 } case is reduced to the { 0 } − ∅ case, which turns the Shapley coefficients from[ 13 , 16 , 16 , 13 ] to [ 13 + 16 , 16 , 16 − 16 , 13 ] , which is [ 12 , 16 , 0 , 13 ] .

Original: However, the original C-Shapley formulafrom Equation 6 in the [31] evaluates to [ 13 , 112 , 0 , 130 ] , which doesn’t match the Myerson value andnot even sum up to 1.
Revised: However, the original CShapley formulafrom Equation 6 in the [30] evaluates to [ 13 , 112 , 0 , 130 ] , which doesn’t match the Myerson value andnot even sum up to 1.

Original: Another example of computing the value of node 1 is shown in Figure 5 right.
Revised: Another example of computing the value of node 1 is shown in Figure 5 right.

Original: The C-Shapley, even with the correct formula, eventually boils down to an approximation of the
Revised: The C_Shapley, even with the correct formula, eventually boils down to an approximation of the

Original: Shapley value or the Myerson value, which as we discussed are less ideal than the HN value.
Revised: Shapley value or the Myerson value, which as we discussed are less idea than the HN value.

Original: Also,the correct formula in Equation 22 requires generating all possible subgraphs U containing the node iand specify the edges between U and [ d ] \ U .
Revised: Also,the correct formula in Equation 21 requires generating all possible subgraphs U containing the node iand specify the edges between U and [ d ] \ U .

Original: This makes the computation very complicated, we thusskip the comparison to the C-Shapley value.
Revised: This makes the computation very complicated, we thusskip the comparison to the C_Shapley value.

Original: G Use the graph structure via an L-hop cutoff
Revised: G Use the graph structure via an L-hop cutoff

Original: Although the Shapley value itself is not structure-aware, we do note the existing Shapley-valuebased GNN explanation methods use an L-hop cutoff to help approximate the Shapley value [5, 4].
Revised: Although the Shapley value itself is not structure-aware, we do note the existing Shapley-valuebased GNN explanation methods uses a L-hop cutoff to help approximate the Shapley value [5, 4].

Original: Technically, this operation uses the graph structure, so we can’t strictly referto these explanationmethods as not structure-aware.
Revised: Technically, this operation uses the graph structure, so we can’t strictly refer these explanationmethods as not structure aware.

Original: However, we argue that the L-hop cutoff is a naive way of utilizingthe graph structure.
Revised: However, we argue that the L-hop cutoff is a naive way of utilizingthe graph structure.

Original: It has several concerns, and it is not the same structure-aware as the HN value.
Revised: It has several concerns, and it is not the same structure-aware as the HN value.

Original: The L-hop cutoff approximates the Shapley value of node i by considering only the L-hop neighborsof i when explaining an L-layer GNN.
Revised: The L-hop cutoff approximates the Shapley value of node i by considering only the L-hop neighborsof i when explaining a L-layer GNN.

Original: The rationale of this operation is that an L-layer GNNs onlypropagate messages within L-hops so a node more than L-hop away from i has never passed anymessages to i which means no interactions are possible.
Revised: The rationale of this operation is that an L-layer GNNs onlypropagate messages within L-hops so a node more than L-hop away from i has never passed anymessages to i which means no interactions are possible.

Original: In existing Shapley-value-based GNNexplanation methods, this L-hop cutoff operation was meant for reducing the exponentially growingcomputations of the Shapley value, and the ultimate goal is stillto compute the Shapley value.
Revised: In existing Shapley-value-based GNNexplanation methods, this L-hop cutoff operation was meant for reducing the exponentially growingcomputations of the Shapley value, and the ultimate goal is still compute the Shapley value.

Original: The
Revised: The

Original: L-hop cutoff operation has several issues making it a less desirable choice.
Revised: L-hop cutoff operation has several issues making it a less desirable choice.

Original: 1) Even meant to savecomputation, there are still many nodes involved in the computation after applying the L-hop cutoffsince the number of nodes grows exponentially as L grows.
Revised: 1) Even meant to savecomputation, there are still many nodes involved in the computation after applying the L-hop cutoff,since the number of nodes grow exponentially as L grow.

Original: For advanced GNNs, the L can be large.
Revised: For advanced GNNs, the L can be large.

Original: When L is larger than the diameter of the graph, which is actually the case for many recent deep
Revised: When L is larger than the diameter of the graph, which is actually the case for many recent deep

Original: GNNs, the L-hop cutoff is not effective anymore.
Revised: GNNs, the L-hop cutoff is not effective anymore.

Original: When constructing coalitions of nodes within thelocal graph of L-hops, the computation still follows the Shapley value formula.
Revised: When constructing coalitions of nodes within thelocal graph of L-hops, the computation still follows the Shapley value formula.

Original: This means the usefulgraph structure information among these nodes is forfeited which causes the structure-awarenessconcern of Shapley value as we discussed in Section 3,
Revised: This means the usefulgraph structure information among these nodes is forfeited which causes the structure-awarenessconcern of Shapley value as we discussed in Section 3,

Original: H More explanation visualizations
Revised: H More explanation visualizations

Original: Under the same setting as Figure 3, we visualize more explanations in 6.[Figure] [Figure]
Revised: Under the same setting as Figure 3, we visualize more explanations in 6.

Original: “occasionally funny , always very colorful and enjoyably overblown in the traditional almodóvar style

Original: [Figure]
Revised: [Figure]

Original: [Figure]