Original:  Explaining Graph Neural Networks with Structure-Aware Cooperative Games
Revised: GStarX: Explaining Graph Neural Networks with Structure-Aware Cooperative Games

Original: Abstract
Revised: Abstract

Original: Explaining machine learning models is an important and increasingly popular area of research interest.
Revised: Explaining machine learning models is an important and increasingly popular area of research interest.

Original: The Shapley value from game theory has been proposed as a prime approach to compute feature importance towards model predictions on images, text, tabular data, and recently graph neural networks (GNNs) on graphs.
Revised: The Shapley value from game theory has been proposed as a prime approach to compute feature importance towards model predictions on images, text, tabular data, and recently graph neural networks (GNNs) on graphs.

Original: In this work, we revisit the appropriateness of the Shapley value for
Revised: In this work, we revisit the appropriateness of the Shapley value for

Original: GNN explanation, where the task is to identify the most important subgraph and constituent nodes for GNN predictions.
Revised: GNN explanation, where the task is to identify the most important subgraph and constituent nodes for GNN predictions.

Original: We claim that the Shapley value is a non-ideal choice for graph data because it is by definition not structure-aware.
Revised: We claim that the Shapley value is a non-ideal choice for graph data because it is by definition not structure-aware.

Original: We propose a Graph Structure-aware eXplanation (GStarX) method to leverage the critical graph structure information to improve the explanation.
Revised: We propose a Graph Structure-aware eXplanation (GStarX) method to leverage the critical graph structure information to improve the explanation.

Original: Specifically, we define a scoring function based on a new structure-aware value from cooperative game theory proposed by Hamiache and Navarro (HN).
Revised: Specifically, we define a scoring function based on a new structure-aware value from cooperative game theory proposed by Hamiache and Navarro (HN).

Original: When used to score node importance, the HN value utilizes graph structures to attribute cooperation surplus between neighbor nodes, resembling message passing in GNNs, so that node importance scores reflect not only the node feature importance, but also the node structural roles.
Revised: When used to score node importance, the HN value utilizes graph structures to attribute cooperation surplus between neighbor nodes, resembling message passing in GNNs, so that node importance scores reflect not only the node feature importance, but also the node structural roles.

Original: We demonstrate that GStarX produces qualitatively more intuitive explanations, and quantitatively improves explanation fidelity over strong baselines on chemical graph property prediction and text graph sentiment classification.
Revised: We demonstrate that GStarX produces qualitatively more intuitive explanations, and quantitatively improves explanation fidelity over strong baselines on chemical graph property prediction and text graph sentiment classification.

Original:  Introduction
Revised:  Introduction

Original: Explainability is crucial for complex machine learning (ML) models in sensitive applications, helping establish user trust and providing insights for potential model improvements.
Revised: Explainability is crucial for complex machine learning (ML) models in sensitive applications, helping establish user trust and providing insights for potential model improvements.

Original: Many efforts focus on explaining models on images, text, and tabular data.
Revised: Many efforts focus on explaining models on images, text, and tabular data.

Original: In contrast, the explainability of models on graph data is yet underexplored.
Revised: In contrast, the explainability of models on graph data is yet underexplored.

Original: Since explainability can be especially critical for many graph tasks like drug discovery, and interest in deep graph models is growing rapidly, further investigation of graph explainability is warranted.
Revised: Since explainability can be especially critical for many graph tasks like drug discovery, and interest in deep graph models is growing rapidly, further investigation of graph explainability is warranted.

Original: In this work, we study graph ML explanation with graph neural networks (GNNs) as the target models, given their popularity and widespread use for graph machine learning tasks [42, 29, 38, 34, 33, 45].
Revised: In this work, we study graph ML explanation with graph neural networks (GNNs) as the target models, given their popularity and widespread use for graph machine learning tasks [42, 29, 38, 34, 33, 45].

Original: In ML explainability, important features are identified, and the Shapley value [30] has been deemed as a “fair” scoring function for computing feature importance.
Revised: In ML explainability, important features are identified, and the Shapley value [30] has been deemed as a “fair” scoring function for computing feature importance.

Original: Originally from cooperative game theory, many values, including the Shapley value, have been proposed for allocating a total payoff to players in a game.
Revised: Originally from cooperative game theory, many values, including the Shapley value, have been proposed for allocating a total payoff to players in a game.

Original: When used for scoring the feature importance of a data instance, the model prediction is treated as the total payoff and the features are considered as players.
Revised: When used for scoring the feature importance of a data instance, the model prediction is treated as the total payoff and the features are considered as players.

Original: In particular, for an instance with n features { x 1 , . . .
Revised: In particular, for an instance with n features { x 1 , . . .

Original: x n } , the Shapley value of its i th feature x i is computed via aggregating m ( i, S ) , which are the marginal contributions of x i to sets of other features x S ⊆ { x 1 , . . .
Revised: x n } , the Shapley value of its i th feature x i is computed via aggregating m ( i, S ) , which are the marginal contributions of x i to sets of other features x S ⊆ { x 1 , . . .

Original: , x n } \ { x i } .
Revised: , x n } \ { x i } .

Original: Each x S is called a coalition .
Revised: Each x S is called a coalition .

Original: Each m ( i, S ) is computed as the difference between model outputs for
Revised: Each m ( i, S ) is computed as the difference between model outputs for

Original: “is still quite good – natured and not a bad way to spend an hour” spendgood bad still natured and not hour to an is quite a way 3 1   3  31/ 1/ 1/ 1/ 1/ 1/  1/
Revised: “is still quite good – natured and not a bad way to spend an hour” spendgood bad still natured and not hour to an is quite a way 3 1   3  31/ 1/ 1/ 1/ 1/ 1/  1/

Original: HNweights
Revised: HNweights

Original: [Figure]
Revised: [Figure]

Original: x S ∪ { x i } and x S , e.g., difference of probability belonging to a target class for these two set of features, and it is meant to capture the interaction between x i and x S .
Revised: x S ∪ { x i } and x S , e.g., difference of probability belonging to a target class for these two set of features, and it is meant to capture the interaction between x i and x S .

Original: The Shapley value is widely used for explaining ML models on images, text, and tabular data, when the features are pixels, words, and attributes [22, 24].
Revised: The Shapley value is widely used for explaining ML models on images, text, and tabular data, when the features are pixels, words, and attributes [22, 24].

Original: The Shapley value has recently been extended to explain GNNs on graphs through feature importance scoring as above, where features are nodes [9] or supernodes [44].
Revised: The Shapley value has recently been extended to explain GNNs on graphs through feature importance scoring as above, where features are nodes [9] or supernodes [44].

Original: We argue that the Shapley value is a non-ideal choice for (super)node importance scoring because its contribution aggregation is non-structure-aware .
Revised: We argue that the Shapley value is a non-ideal choice for (super)node importance scoring because its contribution aggregation is non-structure-aware .

Original: The Shapley value aggregation assumes no structural relationship between x i and x S even though they are both parts of the input graph (a review of the Shapley value is in Section 2.2).
Revised: The Shapley value aggregation assumes no structural relationship between x i and x S even though they are both parts of the input graph (a review of the Shapley value is in Section 2.2).

Original: Since the graph structure generally contains critical information and is crucial to the success of
Revised: Since the graph structure generally contains critical information and is crucial to the success of

Original: GNNs, we consider properly leveraging the structure with a better structure-aware scoring function.
Revised: GNNs, we consider properly leveraging the structure with a better structure-aware scoring function.

Original: We propose Graph Structure-aware eXplanation (GStarX), where we construct a structure-aware node importance scoring function based on the Hamiache-Navarro (HN) value [15] from cooperative game theory.
Revised: We propose Graph Structure-aware eXplanation (GStarX), where we construct a structure-aware node importance scoring function based on the Hamiache-Navarro (HN) value [15] from cooperative game theory.

Original: Recall that GNNs make predictions via message passing, during which node representations are learned by aggregating messages from neighbors.
Revised: Recall that GNNs make predictions via message passing, during which node representations are learned by aggregating messages from neighbors.

Original: Message passing aggregates both feature and structure information, resulting in powerful structure-aware models [5].
Revised: Message passing aggregates both feature and structure information, resulting in powerful structure-aware models [5].

Original: The HN value shares a similar idea to message passing by allocating the payoff surplus generated from the cooperation between neighboring players (nodes).
Revised: The HN value shares a similar idea to message passing by allocating the payoff surplus generated from the cooperation between neighboring players (nodes).

Original: When used as a scoring function to explain node importance, the HN value captures both features and structural interactions between nodes (details in Section 4).
Revised: When used as a scoring function to explain node importance, the HN value captures both features and structural interactions between nodes (details in Section 4).

Original: Figure 1 (a) shows an example comparing the Shapley value and the HN value.
Revised: Figure 1 (a) shows an example comparing the Shapley value and the HN value.

Original: In this example, their difference boils down to different aggregation weights of marginal contributions, where the former is uniform and the latter is structure-aware (details in Section 3.2).
Revised: In this example, their difference boils down to different aggregation weights of marginal contributions, where the former is uniform and the latter is structure-aware (details in Section 3.2).

Original: In summary, our contributions are:
Revised: In summary, our contributions are:

Original: • Identify the non-structure-aware limitation of the Shapley value for GNN explanation.
Revised: • Identify the non-structure-aware limitation of the Shapley value for GNN explanation.

Original: • Introduce the structure-aware HN value from cooperative game theory to the graph machine learning community and connect it to the GNN message passing and GNN explanation.
Revised: • Introduce the structure-aware HN value from cooperative game theory to the graph machine learning community and connect it to the GNN message passing and GNN explanation.

Original: • Propose a new HN-value-based GNN explanation method GStarX, and demonstrate the superiority of GStarX over strong baselines for explaining GNNs on chemical and text graphs.
Revised: • Propose a new HN-value-based GNN explanation method GStarX, and demonstrate the superiority of GStarX over strong baselines for explaining GNNs on chemical and text graphs.

Original:  Preliminaries 2.1 Graph neural networks
Revised:  Preliminaries 2.1 Graph neural networks

Original: Consider a graph G with (feature-enriched) nodes V and edges E .
Revised: Consider a graph G with (feature-enriched) nodes V and edges E .

Original: We denote G as G =
Revised: We denote G as G =

Original: ( V , X , A ) , where X ∈ R n × d denotes d -dimensional features of n nodes in V , and A ∈ { 0 , 1 } n × n denotes the adjacency matrix specifying edges in E .
Revised: ( V , X , A ) , where X ∈ R n × d denotes d -dimensional features of n nodes in V , and A ∈ { 0 , 1 } n × n denotes the adjacency matrix specifying edges in E .

Original: GNNs make predictions on G by learning representations via the message-passing mechanism.
Revised: GNNs make predictions on G by learning representations via the message-passing mechanism.

Original: During message passing, the representation of each node u ∈ V is updated by aggregating its own representation and representations (messages) from its neighbors.
Revised: During message passing, the representation of each node u ∈ V is updated by aggregating its own representation and representations (messages) from its neighbors.

Original: We denote the set of neighbors as N ( u ) .
Revised: We denote the set of neighbors as N ( u ) .

Original: This aggregation is recursively applied, so u can collect messages from its multi-hop neighbors and produce structure-aware representations [5].
Revised: This aggregation is recursively applied, so u can collect messages from its multi-hop neighbors and produce structure-aware representations [5].

Original: With h ( l ) i denotes the representation of node i at iteration l , and AGGR ( · , · ) denotes the aggregation operation, e.g. summation, the representation update is shown in Equation 1.
Revised: With h ( l ) i denotes the representation of node i at iteration l , and AGGR ( · , · ) denotes the aggregation operation, e.g. summation, the representation update is shown in Equation 1.

Original: [Equation]
Revised: [Equation]

Original: 2.2 Cooperative games
Revised: 2.2 Cooperative games

Original: A cooperative game denoted by ( N, v ) , is defined by a set of players N = { 1 , . . . , n } , and a characteristic function v : 2 N → R .
Revised: A cooperative game denoted by ( N, v ) , is defined by a set of players N = { 1 , . . . , n } , and a characteristic function v : 2 N → R .

Original: v takes a subset of players S ⊆ N , called a coalition , and maps it to a payoff v ( S ) , where v ( ∅ ) :
Revised: v takes a subset of players S ⊆ N , called a coalition , and maps it to a payoff v ( S ) , where v ( ∅ ) :

Original: = 0 .
Revised: = 0 .

Original: A solution function ϕ is a function maps each given game ( N, v ) to ϕ ( N, v )
Revised: A solution function ϕ is a function maps each given game ( N, v ) to ϕ ( N, v )

Original: ∈ R n .
Revised: ∈ R n .

Original: The vector ϕ ( N, v ) , called a solution , represents a certain allocation of the total payoff v ( N ) generated by all players to each individual, with the i th coordinate ϕ
Revised: The vector ϕ ( N, v ) , called a solution , represents a certain allocation of the total payoff v ( N ) generated by all players to each individual, with the i th coordinate ϕ

Original: i ( N, v ) being the payoff attributed to player i .
Revised: i ( N, v ) being the payoff attributed to player i .

Original: ϕ ( N, v ) is also called the “value” of the game when it satisfies certain properties, and different values were proposed to name solutions with different properties [30, 35].
Revised: ϕ ( N, v ) is also called the “value” of the game when it satisfies certain properties, and different values were proposed to name solutions with different properties [30, 35].

Original: The Shapley value is one popular solution of cooperative games.
Revised: The Shapley value is one popular solution of cooperative games.

Original: The main idea is to assign each player a “fair” share of the total payoff by considering all possible player interactions.
Revised: The main idea is to assign each player a “fair” share of the total payoff by considering all possible player interactions.

Original: For example, when player i cooperates with a coalition S , the total payoff v ( S ∪ { i } ) may be very different from v ( S ) + v ( { i } ) because of i ’s interaction with S.
Revised: For example, when player i cooperates with a coalition S , the total payoff v ( S ∪ { i } ) may be very different from v ( S ) + v ( { i } ) because of i ’s interaction with S.

Original: Thus the marginal contribution of i to S is defined as by m ( i, S )
Revised: Thus the marginal contribution of i to S is defined as by m ( i, S )

Original: = v ( S ∪ { i } ) − v ( S ) .
Revised: = v ( S ∪ { i } ) − v ( S ) .

Original: Then the formula of the Shapley value for i is shown in Equation 2, where marginal contributions to all possible coalitions S ⊆
Revised: Then the formula of the Shapley value for i is shown in Equation 2, where marginal contributions to all possible coalitions S ⊆

Original: N \{ i } are aggregated.
Revised: N \{ i } are aggregated.

Original: The first identify in Equation 2 shows that the aggregation weights are first uniformly distributed among coalition sizes k (outer average), then uniformly distributed among all coalitions with the same size (inner average).
Revised: The first identify in Equation 2 shows that the aggregation weights are first uniformly distributed among coalition sizes k (outer average), then uniformly distributed among all coalitions with the same size (inner average).

Original: [Table]
Revised: [Table]

Original: v
Revised: v

Original: [Table]
Revised: [Table]

Original: Games with communication structures.
Revised: Games with communication structures.

Original: Although the Shapley value is widely used for cooperative games, its assumption of fully flexible cooperation among all players may not be achievable.
Revised: Although the Shapley value is widely used for cooperative games, its assumption of fully flexible cooperation among all players may not be achievable.

Original: Some coalitions may be preferred over others and some may even be impossible due to limited communication among players.
Revised: Some coalitions may be preferred over others and some may even be impossible due to limited communication among players.

Original: Thus, [26] uses a graph G as the communication structure of players to represent cooperation preference.
Revised: Thus, [26] uses a graph G as the communication structure of players to represent cooperation preference.

Original: A game with a communication structure is defined by a triple ( N, v, G ) , with
Revised: A game with a communication structure is defined by a triple ( N, v, G ) , with

Original: N being the node set of G .
Revised: N being the node set of G .

Original: This game formulation is more practical than fully flexible cooperation when cooperation preference is available.
Revised: This game formulation is more practical than fully flexible cooperation when cooperation preference is available.

Original: Several values with different properties have been proposed for such games [26, 2, 13, 18] including the HN value [15].
Revised: Several values with different properties have been proposed for such games [26, 2, 13, 18] including the HN value [15].

Original:  GNN explanation via feature importance scoring 3.1 Problem formalization
Revised:  GNN explanation via feature importance scoring 3.1 Problem formalization

Original: A general approach to formalize an ML explanation problem is through feature importance scoring [24, 6], where features may refer to pixels of images, words of text, or nodes/edges/subgraphs of graphs.
Revised: A general approach to formalize an ML explanation problem is through feature importance scoring [24, 6], where features may refer to pixels of images, words of text, or nodes/edges/subgraphs of graphs.

Original: Let f ( · ) denote a to-be-explained GNN, G =
Revised: Let f ( · ) denote a to-be-explained GNN, G =

Original: ( V , X , A ) denote an input graph, and 0 < γ < 1 denote a sparsity constraint to enforce concise explanation.
Revised: ( V , X , A ) denote an input graph, and 0 < γ < 1 denote a sparsity constraint to enforce concise explanation.

Original: GNN explanation via subgraph scoring is aimed to find a subgraph g that maximizes a given evaluation metric E VAL ( · , · , · ) , which measures the faithfulness of g to G regarding making predictions with f ( · ) , i.e.
Revised: GNN explanation via subgraph scoring is aimed to find a subgraph g that maximizes a given evaluation metric E VAL ( · , · , · ) , which measures the faithfulness of g to G regarding making predictions with f ( · ) , i.e.

Original: [Equation]
Revised: [Equation]

Original: When the task is graph classification and f ( · ) outputs a one-sum vector f ( G ) ∈ [0 , 1] C containing probabilities for G belongs to C classes, an example E VAL can be the prediction probability drop for removing g from G , i.e. E VAL ( f ( · ) , G , g ) =
Revised: When the task is graph classification and f ( · ) outputs a one-sum vector f ( G ) ∈ [0 , 1] C containing probabilities for G belongs to C classes, an example E VAL can be the prediction probability drop for removing g from G , i.e. E VAL ( f ( · ) , G , g ) =

Original: [ f ( G )]
Revised: [ f ( G )]

Original: c ∗ − [ f ( G\ g )]
Revised: c ∗ − [ f ( G\ g )]

Original: c ∗ with c ∗ =
Revised: c ∗ with c ∗ =

Original: arg max c [ f ( G )] c .
Revised: arg max c [ f ( G )] c .

Original: In practice, since the number of subgraphs is combinatorial in the number of nodes, the objective is often relaxed to finding a set of important nodes or edges first and then inducing the subgraph
Revised: In practice, since the number of subgraphs is combinatorial in the number of nodes, the objective is often relaxed to finding a set of important nodes or edges first and then inducing the subgraph

Original: A more tractable objective of finding the optimal set of nodes S ∗ ⊆
Revised: A more tractable objective of finding the optimal set of nodes S ∗ ⊆

Original: V 2 is given by
Revised: V 2 is given by

Original: [Equation]
Revised: [Equation]

Original: Existing methods often boil down to Equation 4 with different scoring functions (S CORE ), and finding a proper S CORE is non-trivial.
Revised: Existing methods often boil down to Equation 4 with different scoring functions (S CORE ), and finding a proper S CORE is non-trivial.

Original: One example of S CORE is to evaluate each node i directly as S CORE ( f ( · ) , G , i ) =
Revised: One example of S CORE is to evaluate each node i directly as S CORE ( f ( · ) , G , i ) =

Original: [ f ( { i } )] c ∗ .
Revised: [ f ( { i } )] c ∗ .

Original: However, this choice misses interactions between nodes and corresponds to a trivial case in GNNs where no message-passing is performed for { i } .
Revised: However, this choice misses interactions between nodes and corresponds to a trivial case in GNNs where no message-passing is performed for { i } .

Original: Another possibility is to use E VAL as S CORE , e.g., S CORE ( f ( · ) , G , i ) =
Revised: Another possibility is to use E VAL as S CORE , e.g., S CORE ( f ( · ) , G , i ) =

Original: [ f ( G )]
Revised: [ f ( G )]

Original: c ∗ − [ f ( G\{ i } )] c ∗ .
Revised: c ∗ − [ f ( G\{ i } )] c ∗ .

Original: However, this again fails to capture interactions between nodes; for example, two nodes i and j may be both important but also complimentary, so their contribution to G can only be observed when they are missing simultaneously.
Revised: However, this again fails to capture interactions between nodes; for example, two nodes i and j may be both important but also complimentary, so their contribution to G can only be observed when they are missing simultaneously.

Original: 3.2 Scoring functions from cooperative games
Revised: 3.2 Scoring functions from cooperative games

Original: Given the challenges for defining a proper S CORE , solutions to cooperative games, like the Shapley value, have been proposed with f ( · ) as the characteristic function, i.e. S CORE ( f ( · ) , G , i ) =
Revised: Given the challenges for defining a proper S CORE , solutions to cooperative games, like the Shapley value, have been proposed with f ( · ) as the characteristic function, i.e. S CORE ( f ( · ) , G , i ) =

Original: ϕ
Revised: ϕ

Original: i ( |G| , f ( · ))
Revised: i ( |G| , f ( · ))

Original: However, existing works only use the non-structure-aware Shapley value.
Revised: However, existing works only use the non-structure-aware Shapley value.

Original: In contrast, values defined on games ( N, v, G ) with communication structures G are naturally structureaware but were never considered GNN explanation.
Revised: In contrast, values defined on games ( N, v, G ) with communication structures G are naturally structureaware but were never considered GNN explanation.

Original: Below we discuss the non-structure-aware limitation of the Shapley value in detail and motivating structure-aware values with practical examples in GNN explanation.
Revised: Below we discuss the non-structure-aware limitation of the Shapley value in detail and motivating structure-aware values with practical examples in GNN explanation.

Original: The Shapley value is defined on games ( N, v ) , which by definition takes no graph structures.
Revised: The Shapley value is defined on games ( N, v ) , which by definition takes no graph structures.

Original: It assumes flexible cooperation between players and uniform distribution of coalition importance that only depends on | S | (see Equation 2).
Revised: It assumes flexible cooperation between players and uniform distribution of coalition importance that only depends on | S | (see Equation 2).

Original: Even if a G is given and the game is defined as ( N, v, G ) , the Shapley value will overlook G when aggregating m ( i, S ) .
Revised: Even if a G is given and the game is defined as ( N, v, G ) , the Shapley value will overlook G when aggregating m ( i, S ) .

Original: In contrast, structure-aware values on ( N, v, G ) can be interpreted as a weighted aggregation of coalitions with more reasonable weights.
Revised: In contrast, structure-aware values on ( N, v, G ) can be interpreted as a weighted aggregation of coalitions with more reasonable weights.

Original: Although different solutions ϕ ( N, v, G ) have their nuances in weight adjustments [13, 15, 26, 18], they share two key properties: (1) the weight is zero if i and S are disconnected because they are interpreted as players without communication channels [26], and (2) the weight is impacted by the nature of connections between i and S because it is easier for better-connected nodes to communicate.
Revised: Although different solutions ϕ ( N, v, G ) have their nuances in weight adjustments [13, 15, 26, 18], they share two key properties: (1) the weight is zero if i and S are disconnected because they are interpreted as players without communication channels [26], and (2) the weight is impacted by the nature of connections between i and S because it is easier for better-connected nodes to communicate.

Original: A synthetic example.
Revised: A synthetic example.

Original: We take the HN value (definition in Section 4.1) as an example structure-aware value and compare it to the Shapley value in a simple graph in Figure 1 (a) .
Revised: We take the HN value (definition in Section 4.1) as an example structure-aware value and compare it to the Shapley value in a simple graph in Figure 1 (a) .

Original: To compute ϕ 1 ( N, v, G ) , both values aggregates m (1 , S ) for S ∈ {∅ , { 2 } , { 3 } , { 2 , 3 }} .
Revised: To compute ϕ 1 ( N, v, G ) , both values aggregates m (1 , S ) for S ∈ {∅ , { 2 } , { 3 } , { 2 , 3 }} .

Original: The Shapley value first assigns a uniform weight 13 to three different | S | , and then splits weights uniformly for the | S | = 1 case to be 16 .
Revised: The Shapley value first assigns a uniform weight 13 to three different | S | , and then splits weights uniformly for the | S | = 1 case to be 16 .

Original: However, the HN value assigns weight zero for S = { 3 } because 1 and 3 are disconnected in coalition { 1 , 3 } and are assumed to be two independent graphs that shouldn’t interact (property (1) ).
Revised: However, the HN value assigns weight zero for S = { 3 } because 1 and 3 are disconnected in coalition { 1 , 3 } and are assumed to be two independent graphs that shouldn’t interact (property (1) ).

Original: Their interaction is rather captured in the S
Revised: Their interaction is rather captured in the S

Original: = { 2 , 3 } case, when 1 and 3 are connected by the bridging node 2, and this case is also downweighted from 13 to 14 , as 3 is relatively far from(property (2) ).
Revised: = { 2 , 3 } case, when 1 and 3 are connected by the bridging node 2, and this case is also downweighted from 13 to 14 , as 3 is relatively far from(property (2) ).

Original: A practical example.
Revised: A practical example.

Original: The good properties of structure-aware values can help explain graph tasks.
Revised: The good properties of structure-aware values can help explain graph tasks.

Original: The example in Figure 1 (b) is from GraphSST2 (dataset description in Section 5.1), where the graph for sentiment classification is constructed from the sentence “is still quite good-natured and not a bad way to spend an hour” with edges generated by the Biaffine parser [12].
Revised: The example in Figure 1 (b) is from GraphSST2 (dataset description in Section 5.1), where the graph for sentiment classification is constructed from the sentence “is still quite good-natured and not a bad way to spend an hour” with edges generated by the Biaffine parser [12].

Original: Assuming a model can correctly classify it as positive.
Revised: Assuming a model can correctly classify it as positive.

Original: Intuitively, “good” and “not a bad” are central to the human explanation.
Revised: Intuitively, “good” and “not a bad” are central to the human explanation.

Original: To compute the Shapley value of the word “good”, the coalition “not good” will diminish the positive importance of “good”, despite the two words lacking any direct connection.
Revised: To compute the Shapley value of the word “good”, the coalition “not good” will diminish the positive importance of “good”, despite the two words lacking any direct connection.

Original: A structure-aware value can instead eliminate the {“not”, “good”} coalition, and only consider interactions between “not” and “good” (in fact, “not” and any other word) when the bridging “bad” appears, hence better binding “not” with “bad” and improving the salience of “good”.
Revised: A structure-aware value can instead eliminate the {“not”, “good”} coalition, and only consider interactions between “not” and “good” (in fact, “not” and any other word) when the bridging “bad” appears, hence better binding “not” with “bad” and improving the salience of “good”.

Original: In Section 5.2, we revisit this example to observe impacts of structure-awareness empirically.
Revised: In Section 5.2, we revisit this example to observe impacts of structure-awareness empirically.

Original:  GStarX: Graph Structure-aware eXplanation
Revised:  GStarX: Graph Structure-aware eXplanation

Original: We propose GStarX, which uses a structure-aware HN-value-based S CORE to explain GNNs.
Revised: We propose GStarX, which uses a structure-aware HN-value-based S CORE to explain GNNs.

Original: We first state the definition of the HN value in cooperative game theory (4.1), and then connect it to the
Revised: We first state the definition of the HN value in cooperative game theory (4.1), and then connect it to the

Original: GNN message passing (4.2), and finally give the GStarX algorithm for GNN explanation (4.3).
Revised: GNN message passing (4.2), and finally give the GStarX algorithm for GNN explanation (4.3).

Original: 4.1 The HN value
Revised: 4.1 The HN value

Original: Let ( N, v, G ) be a game with a communication structure G and S ⊆
Revised: Let ( N, v, G ) be a game with a communication structure G and S ⊆

Original: N be a coalition.
Revised: N be a coalition.

Original: Let  S∗ = ∪ i ∈ S {N ( i ) }∪ S to be the union of S and its neighbors in G .
Revised: Let ¯ S = ∪ i ∈ S {N ( i ) }∪ S to be the union of S and its neighbors in G .

Original: Let S/ G be the partition of S containing connected components in G , i.e., S/ G = {{ i | i =
Revised: Let S/ G be the partition of S containing connected components in G , i.e., S/ G = {{ i | i =

Original: j or i and j are connected in S by E of G}| j ∈ S } .
Revised: j or i and j are connected in S by E of G}| j ∈ S } .

Original: Let G [ S ] be the induced subgraph of S in G .
Revised: Let G [ S ] be the induced subgraph of S in G .

Original: For example, in Figure 1 (b) , when S = {“is”, “an”, “hour”},  S∗ will be {“is”, “good”, “an”, “hour”, “spend”}, S/ G will be {{“is”}, {“an”, “hour”}}, and
Revised: For example, in Figure 1 (b) , when S = {“is”, “an”, “hour”}, ¯ S will be {“is”, “good”, “an”, “hour”, “spend”}, S/ G will be {{“is”}, {“an”, “hour”}}, and

Original: G [ S ] will be the subgraph with a two-node component (cid:3)(cid:2)
Revised: G [ S ] will be the subgraph with a two-node component (cid:3)(cid:2)

Original: (cid:0)(cid:1) an - (cid:3) (cid:2) (cid:0) (cid:1) hour and a single node component (cid:3) (cid:2) (cid:0) (cid:1) is .
Revised: (cid:0)(cid:1) an - (cid:3) (cid:2) (cid:0) (cid:1) hour and a single node component (cid:3) (cid:2) (cid:0) (cid:1) is .

Original: Definition 4.1 ( Surplus ) .
Revised: Definition 4.1 ( Surplus ) .

Original: The surplus p ( j, S ) generated by a coalition S cooperating with its neighbor j is defined as
Revised: The surplus p ( j, S ) generated by a coalition S cooperating with its neighbor j is defined as

Original: [Equation]
Revised: [Equation]

Original: Intuitively, p ( j, S ) is generated because S is actively cooperating.
Revised: Intuitively, p ( j, S ) is generated because S is actively cooperating.

Original: Thus, when evaluating a fair payoff to S , a portion of p ( j, S ) should be added to its own payoff v ( S ) .
Revised: Thus, when evaluating a fair payoff to S , a portion of p ( j, S ) should be added to its own payoff v ( S ) .

Original: This idea leads to the next definition of associated games regarding the original games, where surplus allocation is performed.
Revised: This idea leads to the next definition of associated games regarding the original games, where surplus allocation is performed.

Original: Definition 4.2 ( HN Associated Game ) .
Revised: Definition 4.2 ( HN Associated Game ) .

Original: Given 0 ≤ τ ≤ 1 representing the portion of surplus that will be allocated to a coalition S for its cooperation with other players.
Revised: Given 0 ≤ τ ≤ 1 representing the portion of surplus that will be allocated to a coalition S for its cooperation with other players.

Original: The HN associated game ( N, v ∗ τ , G ) of ( N, v, G ) is defined as
Revised: The HN associated game ( N, v ∗ τ , G ) of ( N, v, G ) is defined as

Original: [Equation]
Revised: [Equation]

Original: The HN value is a solution on ( N, v, G ) .
Revised: The HN value is a solution on ( N, v, G ) .

Original: It is computed by iteratively constructing a series of HN associated games until it converges to a limit game ( N, ˜ v, G ) .
Revised: It is computed by iteratively constructing a series of HN associated games until it converges to a limit game ( N, ˜ v, G ) .

Original: In other words, we first construct v ∗ τ from v by surplus allocation.
Revised: In other words, we first construct v ∗ τ from v by surplus allocation.

Original: Then we construct v ∗∗ τ from v ∗ τ by allocating the surplus generated from the v ∗ τ and so on.
Revised: Then we construct v ∗∗ τ from v ∗ τ by allocating the surplus generated from the v ∗ τ and so on.

Original: The convergence of the limit game is guaranteed and the result ˜ v is independent of τ under mild conditions as shown in [15].
Revised: The convergence of the limit game is guaranteed and the result ˜ v is independent of τ under mild conditions as shown in [15].

Original: The HN value of each player is uniquely determined by applying ˜ v to that player, i.e. ϕ i ( N, v, G ) =
Revised: The HN value of each player is uniquely determined by applying ˜ v to that player, i.e. ϕ i ( N, v, G ) =

Original: ˜ v ( { i } ) .
Revised: ˜ v ( { i } ) .

Original: We state the formal definitions of the limit game and the uniqueness theorem of the HN value in Appendix E.2.
Revised: We state the formal definitions of the limit game and the uniqueness theorem of the HN value in Appendix E.2.

Original: 4.2 Connecting GNNs and the HN surplus allocation through the message passing lens
Revised: 4.2 Connecting GNNs and the HN surplus allocation through the message passing lens

Original: Both the GNN message passing (MP) and the associated game surplus allocation (SA) are iterative aggregation algorithms, with considerable alignment.
Revised: Both the GNN message passing (MP) and the associated game surplus allocation (SA) are iterative aggregation algorithms, with considerable alignment.

Original: In fact, SA on each singular node set S = { i } is exactly MP: Equation 6 becomes an instantiation of Equation 1 with AGGR ( a, b ) = a + τ (cid:80) j b j on a scalar node value a and a neighbor set b .
Revised: In fact, SA on each singular node set S = { i } is exactly MP: Equation 6 becomes an instantiation of Equation 1 with AGGR ( a, b ) = a + τ (cid:80) j b j on a scalar node value a and a neighbor set b .

Original: These algorithms differ in that SA applies more broadly to | S |≥ 1 cases; it treats S as a supernode when nodes in S form a connected component in G , and handles disconnected S component-wise via Equation 7.
Revised: These algorithms differ in that SA applies more broadly to | S |≥ 1 cases; it treats S as a supernode when nodes in S form a connected component in G , and handles disconnected S component-wise via Equation 7.

Original: We illustrate SA using a real chemical graph example.
Revised: We illustrate SA using a real chemical graph example.

Original: The molecule shown in Figure 1 (c) is taken from MUTAG (dataset description in Section 5.1).
Revised: The molecule shown in Figure 1 (c) is taken from MUTAG (dataset description in Section 5.1).

Original: It is known to be classified as mutagenic because of the -NO2 group (nodes 1, 2, and 3) [8].
Revised: It is known to be classified as mutagenic because of the -NO2 group (nodes 1, 2, and 3) [8].

Original: When we compute v ∗ τ ( { 1 } ) , the surplus p (2 , { 1 } ) , p (3 , { 1 } ) , and p (4 , { 1 } ) are allocated to node 1 (like messages passed to a central node in GNN).
Revised: When we compute v ∗ τ ( { 1 } ) , the surplus p (2 , { 1 } ) , p (3 , { 1 } ) , and p (4 , { 1 } ) are allocated to node 1 (like messages passed to a central node in GNN).

Original: Then surplus are aggregated together with v ( { 1 } ) following Equation 6 to form v ∗ τ ( { 1 } ) .
Revised: Then surplus are aggregated together with v ( { 1 } ) following Equation 6 to form v ∗ τ ( { 1 } ) .

Original: For graphs, the SA approach has two advantages over the uniform aggregation approach used in the Shapley value: (1) The aggregated payoff in each v ∗ τ is structure-aware, like representations learned by GNNs [5], and (2) the iterative computation preserves locality, which is preserved by
Revised: For graphs, the SA approach has two advantages over the uniform aggregation approach used in the Shapley value: (1) The aggregated payoff in each v ∗ τ is structure-aware, like representations learned by GNNs [5], and (2) the iterative computation preserves locality, which is preserved by

Original: GNNs [3].
Revised: GNNs [3].

Original: In other words, these two properties mean close neighbors heavily influence each other due to cooperation in many iterations, while far away nodes less influence each other due to little
Revised: In other words, these two properties mean close neighbors heavily influence each other due to cooperation in many iterations, while far away nodes less influence each other due to little

Original: Input: Graph G with nodes V = { u 1 , . . . , u n } , trained
Revised: Input: Graph G with nodes V = { u 1 , . . . , u n } , trained

Original: GNN f ( · ) , empirical expectation f 0 , hyperparameter τ , max sample size m , number of samples J , sparsity γ .
Revised: GNN f ( · ) , empirical expectation f 0 , hyperparameter τ , max sample size m , number of samples J , sparsity γ .

Original: Get the predicted class c ∗ = arg max c [ f ( G )]
Revised: Get the predicted class c ∗ = arg max c [ f ( G )]

Original: c
Revised: c

Original: Define characteristic function v ( S ) =
Revised: Define characteristic function v ( S ) =

Original: [ f ( g S )]
Revised: [ f ( g S )]

Original: c ∗ − f 0 c ∗ if n ≤ m then ϕ =
Revised: c ∗ − f 0 c ∗ if n ≤ m then ϕ =

Original: Compute-HN ( G , V , v ( · ) , τ ) else ϕ
Revised: Compute-HN ( G , V , v ( · ) , τ ) else ϕ

Original: =
Revised: =

Original: Compute-HN-MC ( G , V , v ( · ) , τ, m, J ) end if
Revised: Compute-HN-MC ( G , V , v ( · ) , τ, m, J ) end if

Original: Sort ϕ in descending order with indices { π 1 , . . .
Revised: Sort ϕ in descending order with indices { π 1 , . . .

Original: , π n } k
Revised: , π n } k

Original: =
Revised: =

Original: ⌊ γ |V|⌋
Revised: ⌊ γ |V|⌋

Original: Return: S ∗ = { u π 1 , . . .
Revised: Return: S ∗ = { u π 1 , . . .

Original: , u π k }
Revised: , u π k }

Original: Algorithm
Revised: Algorithm

Original: Input: Graph instance G with nodes
Revised: Input: Graph instance G with nodes

Original: V = { u 1 , . . . , u n } , characteristic function v , hyperparameter τ .
Revised: V = { u 1 , . . . , u n } , characteristic function v , hyperparameter τ .

Original: for S in 2 N do
Revised: for S in 2 N do

Original: Compute payoff v ( S ) {Eq.(8)} end for
Revised: Compute payoff v ( S ) {Eq.(8)} end for

Original: Construct matrix H { τ,n, G} {Eq.(16)} repeat
Revised: Construct matrix H { τ,n, G} {Eq.(16)} repeat

Original: H = HH until H converges
Revised: H = HH until H converges

Original: Get the limit game ˜ v
Revised: Get the limit game ˜ v

Original: = H v {Eq.(17)}
Revised: = H v {Eq.(17)}

Original: Assign the first n entries of ˜ v to ϕ
Revised: Assign the first n entries of ˜ v to ϕ

Original: Return: ϕ cooperation.
Revised: Return: ϕ cooperation.

Original: In the MUTAG example, since the local -NO2 generates a high payoff for the mutagenicity classification, locally allocating the payoff helps us better understand the importance of the nitrogen atom and the oxygen atoms.
Revised: In the MUTAG example, since the local -NO2 generates a high payoff for the mutagenicity classification, locally allocating the payoff helps us better understand the importance of the nitrogen atom and the oxygen atoms.

Original: Whereas aggregating over many unnecessary coalitions with far-away carbon atoms can obscure the true contribution of -NO2.
Revised: Whereas aggregating over many unnecessary coalitions with far-away carbon atoms can obscure the true contribution of -NO2.

Original: We will revisit this example in Section 5.2.
Revised: We will revisit this example in Section 5.2.

Original: 4.3 The GStarX algorithm
Revised: 4.3 The GStarX algorithm

Original: We now state our algorithm for explaining GNNs with GStarX. Notice that GStarX scores nodes in a graph but not each dimension of node features.
Revised: We now state our algorithm for explaining GNNs with GStarX. Notice that GStarX scores nodes in a graph but not each dimension of node features.

Original: Feature dimension importance explanation is an orthogonal perspective that can be added on top of GStarX. We leave this extension as a future work.
Revised: Feature dimension importance explanation is an orthogonal perspective that can be added on top of GStarX. We leave this extension as a future work.

Original: GStarX formulates the GNN explanation problem as a feature importance scoring problem, where nodes are scored to find the optimal node-induced subgraph as we introduced in Section 3.1.
Revised: GStarX formulates the GNN explanation problem as a feature importance scoring problem, where nodes are scored to find the optimal node-induced subgraph as we introduced in Section 3.1.

Original: It essentially implements and solves the objective in Equation 4, where an HN-value-based S CORE is used.
Revised: It essentially implements and solves the objective in Equation 4, where an HN-value-based S CORE is used.

Original: To use such S CORE , we need to define the players and the characteristic function of the game, and then apply the formula in Equation 6 and 7.
Revised: To use such S CORE , we need to define the players and the characteristic function of the game, and then apply the formula in Equation 6 and 7.

Original: Suppose the inputs are a graph G with nodes
Revised: Suppose the inputs are a graph G with nodes

Original: V = { u 1 , . . . , u n } and label y ∈ { 1 , . . .
Revised: V = { u 1 , . . . , u n } and label y ∈ { 1 , . . .

Original: , C } , a GNN f ( · ) outputs a probability vector f ( G ) ∈ [0 , 1] C , and the predicted class c ∗ = arg max c [ f ( G )] c .
Revised: , C } , a GNN f ( · ) outputs a probability vector f ( G ) ∈ [0 , 1] C , and the predicted class c ∗ = arg max c [ f ( G )] c .

Original: Let V be players, and let the normalized probability of the predicted class be the characteristic function v :
Revised: Let V be players, and let the normalized probability of the predicted class be the characteristic function v :

Original: [Equation]
Revised: [Equation]

Original: Here the normalization term f 0 c ∗ =
Revised: Here the normalization term f 0 c ∗ =

Original: E [[ f ( G )] c ∗ ] is the expectation over a random variable G representing a general graph.
Revised: E [[ f ( G )] c ∗ ] is the expectation over a random variable G representing a general graph.

Original: In practice, we approximate it using the empirical expectation over all G in the dataset.
Revised: In practice, we approximate it using the empirical expectation over all G in the dataset.

Original: S CORE will be the HN value of the game, i.e., S CORE ( f ( · ) , G , i ) =
Revised: S CORE will be the HN value of the game, i.e., S CORE ( f ( · ) , G , i ) =

Original: ϕ
Revised: ϕ

Original: i ( V , v, G ) =
Revised: i ( V , v, G ) =

Original: ˜ v ( { i } ) .
Revised: ˜ v ( { i } ) .

Original: Given S CORE , we solve the objective by first computing the scores ϕ ∈ R n then selecting the top ⌊ γ |V|⌋ scores greedily as in Algorithm 1.
Revised: Given S CORE , we solve the objective by first computing the scores ϕ ∈ R n then selecting the top ⌊ γ |V|⌋ scores greedily as in Algorithm 1.

Original: Practically, like other game-theoretic methods, the exact computation of the HN value is infeasible when the number of players n is large.
Revised: Practically, like other game-theoretic methods, the exact computation of the HN value is infeasible when the number of players n is large.

Original: We thus do an exact computation for small graphs (the if-branch) and Monte-Carlo sampling for large graphs (the else-branch).
Revised: We thus do an exact computation for small graphs (the if-branch) and Monte-Carlo sampling for large graphs (the else-branch).

Original: The Compute-HN function is shown in Algorithm 2, where the H stands for a matrix form of the associated game defined in Definition 4.2.(See Appendix E.2 and E.3 for details of the matrix form and algorithms for Compute-HN-MC ).
Revised: The Compute-HN function is shown in Algorithm 2, where the H stands for a matrix form of the associated game defined in Definition 4.2.(See Appendix E.2 and E.3 for details of the matrix form and algorithms for Compute-HN-MC ).

Original: Also, even though the algorithm is stated for graph classification, GStarX works for node classification as well.
Revised: Also, even though the algorithm is stated for graph classification, GStarX works for node classification as well.

Original: This can be easily seen since GNNs classify nodes u i by processing an ego-graph centered at u i , so the task can be converted to graph classification with the label of u i used as the label of the ego-graph.
Revised: This can be easily seen since GNNs classify nodes u i by processing an ego-graph centered at u i , so the task can be converted to graph classification with the label of u i used as the label of the ego-graph.

Original: We focus on graph classification in the main text for simpler illustration and discuss more about node classification in Appendix B.
Revised: We focus on graph classification in the main text for simpler illustration and discuss more about node classification in Appendix B.

Original:  Experiments 5.1 Experiment settings
Revised:  Experiments 5.1 Experiment settings

Original: Datasets.
Revised: Datasets.

Original: We conduct experiments on datasets from different domains including synthetic graphs, chemical graphs, and text graphs.
Revised: We conduct experiments on datasets from different domains including synthetic graphs, chemical graphs, and text graphs.

Original: A brief description of the datasets is shown below with more detailed statistics in Appendix A.
Revised: A brief description of the datasets is shown below with more detailed statistics in Appendix A.

Original: • Chemical graph property prediction.
Revised: • Chemical graph property prediction.

Original: MUTAG [8], BACE and BBBP [39] contain chemical molecule graphs for graph classification, with atoms as nodes, bonds as edges, and chemical properties as graph labels.
Revised: MUTAG [8], BACE and BBBP [39] contain chemical molecule graphs for graph classification, with atoms as nodes, bonds as edges, and chemical properties as graph labels.

Original: • Text graph sentiment classification.
Revised: • Text graph sentiment classification.

Original: GraphSST2 and Twitter [43] contain graphs constructed from text.
Revised: GraphSST2 and Twitter [43] contain graphs constructed from text.

Original: Nodes are words with pre-trained BERT embeddings as features.
Revised: Nodes are words with pre-trained BERT embeddings as features.

Original: Edges are generated by the Biaffine parser [12].
Revised: Edges are generated by the Biaffine parser [12].

Original: Graphs are labeled as positive or negative sentiment.
Revised: Graphs are labeled as positive or negative sentiment.

Original: • Synthetic graph motif detection.
Revised: • Synthetic graph motif detection.

Original: BA2Motifs [25] contains graphs with a Barabasi-Albert (BA) base graph of size 20 and a 5-node motif in each graph.
Revised: BA2Motifs [25] contains graphs with a Barabasi-Albert (BA) base graph of size 20 and a 5-node motif in each graph.

Original: Node features are 10-dimensional all-one vectors.
Revised: Node features are 10-dimensional all-one vectors.

Original: The motif can be either a house-like structure or a cycle.
Revised: The motif can be either a house-like structure or a cycle.

Original: Graphs are labelled in two classes based on which motif they contain.
Revised: Graphs are labelled in two classes based on which motif they contain.

Original: GNNs and explanation baselines.
Revised: GNNs and explanation baselines.

Original: We evaluate GStarX by explaining GCNs [19] on all datasets in our major experiment in Section 5.2.
Revised: We evaluate GStarX by explaining GCNs [19] on all datasets in our major experiment in Section 5.2.

Original: In the ablation study in Section 5.3, we further evaluate on
Revised: In the ablation study in Section 5.3, we further evaluate on

Original: GIN [40] and GAT [36] on certain datasets following [44].
Revised: GIN [40] and GAT [36] on certain datasets following [44].

Original: All models are trained to convergence with hyperparameters and performance shown in Appendix A.2.
Revised: All models are trained to convergence with hyperparameters and performance shown in Appendix A.2.

Original: We compare with 5 strong baselines representing the SOTA methods for GNN explanation: GNNExplainer [41], PGExplainer [25],
Revised: We compare with 5 strong baselines representing the SOTA methods for GNN explanation: GNNExplainer [41], PGExplainer [25],

Original: SubgraphX [44], GraphSVX [9], and OrphicX [21].
Revised: SubgraphX [44], GraphSVX [9], and OrphicX [21].

Original: In particular, SubgraphX and GraphSVX use
Revised: In particular, SubgraphX and GraphSVX use

Original: Shapley-value-based scoring functions.
Revised: Shapley-value-based scoring functions.

Original: Evaluation metrics.
Revised: Evaluation metrics.

Original: Evaluating explanations is non-trivial due to the lack of ground truth.
Revised: Evaluating explanations is non-trivial due to the lack of ground truth.

Original: We follow [44, 43] to employ Fidelity , Inverse Fidelity ( Inv-Fidelity ), and Sparsity as our evaluation metrics.
Revised: We follow [44, 43] to employ Fidelity , Inverse Fidelity ( Inv-Fidelity ), and Sparsity as our evaluation metrics.

Original: Fidelity and Inv-Fidelity measure whether the prediction is faithfully important to the model prediction by removing the selected nodes or only keeping the selected nodes respectively.
Revised: Fidelity and Inv-Fidelity measure whether the prediction is faithfully important to the model prediction by removing the selected nodes or only keeping the selected nodes respectively.

Original: Sparsity promotes fair comparison by controlling explanations to have similar sizes, since including more nodes generally improves Fidelity and Inv-Fidelity , and explanations with different sizes are not directly comparable.
Revised: Sparsity promotes fair comparison by controlling explanations to have similar sizes, since including more nodes generally improves Fidelity and Inv-Fidelity , and explanations with different sizes are not directly comparable.

Original: Ideal explanations should have high Fidelity , low Inv-Fidelity , and high Sparsity , indicating relevance and conciseness.
Revised: Ideal explanations should have high Fidelity , low Inv-Fidelity , and high Sparsity , indicating relevance and conciseness.

Original: Equations 9-11 show their formulas.
Revised: Equations 9-11 show their formulas.

Original: [Equation]
Revised: [Equation]

Original: Fidelity and Inv-Fidelity are complementary and are both important for a good explanation g .
Revised: Fidelity and Inv-Fidelity are complementary and are both important for a good explanation g .

Original: Fidelity justifies the necessity for g to be included to predict correctly.
Revised: Fidelity justifies the necessity for g to be included to predict correctly.

Original: Inv-Fidelity justifies the sufficiency of a standalone g to predict correctly.
Revised: Inv-Fidelity justifies the sufficiency of a standalone g to predict correctly.

Original: As they are analogous to precision and recall, we draw an analogy to the F1 score to propose a single-scalar-metric “harmonic fidelity” ( H-Fidelity ), where we normalize them by Sparsity and take their harmonic mean; see Appendix A.3 for the formula.
Revised: As they are analogous to precision and recall, we draw an analogy to the F1 score to propose a single-scalar-metric “harmonic fidelity” ( H-Fidelity ), where we normalize them by Sparsity and take their harmonic mean; see Appendix A.3 for the formula.

Original: Hyperparameters.
Revised: Hyperparameters.

Original: GStarX includes three hyperparameters: τ for the allocated surplus in the associated game, m as the maximum graph size to perform exact HN value calculation, and J as the number of samples for the MC approximation.
Revised: GStarX includes three hyperparameters: τ for the allocated surplus in the associated game, m as the maximum graph size to perform exact HN value calculation, and J as the number of samples for the MC approximation.

Original: In our experiments, we choose τ = 0 .
Revised: In our experiments, we choose τ = 0 .

Original: 01 since we need τ < 2 n for convergence (Appendix E.2) and all graphs in the datasets above have less than 200 nodes.
Revised: 01 since we need τ < 2 n for convergence (Appendix E.2) and all graphs in the datasets above have less than 200 nodes.

Original: For m and J , bigger values should be better for the MC approximation, and we found m = 10 and J = n work well empirically.
Revised: For m and J , bigger values should be better for the MC approximation, and we found m = 10 and J = n work well empirically.

Original: 5.2 Evaluation results
Revised: 5.2 Evaluation results

Original: Quantitative studies.
Revised: Quantitative studies.

Original: We report averaged test set H-Fidelity in Table 1.
Revised: We report averaged test set H-Fidelity in Table 1.

Original: We conduct 8 different runs to get results with Sparsity ranging from 0.5-0.85 in 0.05 increments ( Sparsity cannot be precisely guaranteed, hence it has minor variations across methods) and report the best H-Fidelity for each method.
Revised: We conduct 8 different runs to get results with Sparsity ranging from 0.5-0.85 in 0.05 increments ( Sparsity cannot be precisely guaranteed, hence it has minor variations across methods) and report the best H-Fidelity for each method.

Original: GStarX outperforms others on 4/6 datasets and has the highest average.
Revised: GStarX outperforms others on 4/6 datasets and has the highest average.

Original: We also follow [44] to show the Fidelity vs. Sparsity plots for all 8 sparsity in Appendix A.4.
Revised: We also follow [44] to show the Fidelity vs. Sparsity plots for all 8 sparsity in Appendix A.4.

Original: Qualitative studies.
Revised: Qualitative studies.

Original: We visualize the explanations of graphs in GraphSST2 in Figure 2 and compare them qualitatively.
Revised: We visualize the explanations of graphs in GraphSST2 in Figure 2 and compare them qualitatively.

Original: We show explanations selected with high and comparable Sparsity on a positive (upper) graph and a negative (lower) graph. GStarX
Revised: We show explanations selected with high and comparable Sparsity on a positive (upper) graph and a negative (lower) graph. GStarX

Original: concisely captures the important words for sentiment classification without including extraneous ones for both sentences.
Revised: concisely captures the important words for sentiment classification without including extraneous ones for both sentences.

Original: Baseline methods generally select some-but-not-all important sentiment words, with extra neutral words as well.
Revised: Baseline methods generally select some-but-not-all important sentiment words, with extra neutral words as well.

Original: Among baselines, SubgraphX gives more reasonable results.
Revised: Among baselines, SubgraphX gives more reasonable results.

Original: However, it cannot cover two groups of important nodes with a limited budget because it can only select a connected subgraph as the explanation; e.g. is the should you lameness on bad in .
Revised: However, it cannot cover two groups of important nodes with a limited budget because it can only select a connected subgraph as the explanation; e.g. is the should you lameness on bad in .

Original: how the title ’s movie clue is the should you lameness on bad in . how the title ’s movie clue is the should you lameness on bad in .how the title ’s movie clue is the should you lameness on bad in .
Revised: how the title ’s movie clue is the should you lameness on bad in . how the title ’s movie clue is the should you lameness on bad in .

Original: how the title ’s movie clue is the should you lameness on bad in . how the title ’s movie clue is the should you lameness on bad in . 
Revised: how the title ’s movie clue is the should you lameness on bad in . how the title ’s movie clue is the should you lameness on bad in . how the title ’s movie clue is the should you lameness on bad in .

Original: how the title ’s movie clue
Revised: how the title ’s movie clue

Original: GNNExplainer
Revised: GNNExplainer

Original: PGExplainer
Revised: PGExplainer

Original: SubgraphX
Revised: SubgraphX

Original: GraphSVX “is still quite good – natured and not a bad way to spend an hour” “the title 's lameness should clue you in on how bad the movie is.”
Revised: GraphSVX “is still quite good – natured and not a bad way to spend an hour” “the title 's lameness should clue you in on how bad the movie is.”

Original: OrphicX
Revised: OrphicX

Original: GStarX
Revised: GStarX

Original: [Figure]
Revised: [Figure]

Original: an is quite a
Revised: an is quite a

Original: [Figure]
Revised: [Figure]

Original: spendgood
Revised: spendgood

Original: [Figure]
Revised: [Figure]

Original: still natured and not hour to an is quite a
Revised: still natured and not hour to an is quite a

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: still natured
Revised: still natured

Original: [Figure]
Revised: [Figure]

Original: not hour to an is quite a
Revised: not hour to an is quite a

Original: [Figure]
Revised: [Figure]

Original: to an is quite a
Revised: to an is quite a

Original: [Figure]
Revised: [Figure]

Original: not
Revised: not

Original: [Figure]
Revised: [Figure]

Original: to an is
Revised: to an is

Original: [Figure]
Revised: [Figure]

Original: a
Revised: a

Original: [Figure]
Revised: [Figure]

Original: to an is quite
Revised: to an is quite

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: “Ground
Revised: “Ground

Original: [Figure]
Revised: [Figure]

Original: to cover the negative word “lameness” in the lower sentence, SubgraphX needs at least three more nodes along the way, which will significantly decrease Sparsity while including undesirable, neutral words.
Revised: to cover the negative word “lameness” in the lower sentence, SubgraphX needs at least three more nodes along the way, which will significantly decrease Sparsity while including undesirable, neutral words.

Original: Moreover, we discussed in Section 3.2 that the Shapley value will downgrade the positive importance of the word “good” for the upper sentence.
Revised: Moreover, we discussed in Section 3.2 that the Shapley value will downgrade the positive importance of the word “good” for the upper sentence.

Original: Comparing the normalized contribution scores of our HN-value-based method GStarX and the Shapley-based method GraphSVX, contribution of “good” is higher in ours: 0.1152 vs. 0.0371.
Revised: Comparing the normalized contribution scores of our HN-value-based method GStarX and the Shapley-based method GraphSVX, contribution of “good” is higher in ours: 0.1152 vs. 0.0371.

Original: We visualize explanations selected with high and comparable Sparsity of a mutagenic molecule from MUTAG in Figure 3.
Revised: We visualize explanations selected with high and comparable Sparsity of a mutagenic molecule from MUTAG in Figure 3.

Original: Explanations on chemical graphs are harder to evaluate than text graphs as they require domain knowledge.
Revised: Explanations on chemical graphs are harder to evaluate than text graphs as they require domain knowledge.

Original: MUTAG has been widely used as a benchmark for evaluating
Revised: MUTAG has been widely used as a benchmark for evaluating

Original: GNN explanations because human experts recognize -NO2 as mutagenic [8], which makes MUTAG a dataset with “ground truth” 3 .
Revised: GNN explanations because human experts recognize -NO2 as mutagenic [8], which makes MUTAG a dataset with “ground truth” 3 .

Original: Surprisingly, we found that GStarX generates much better HFidelity / Fidelity / Inv-Fidelity than other methods and even the “ground truth” by only selecting the -O in -NO2 as explanations.
Revised: Surprisingly, we found that GStarX generates much better HFidelity / Fidelity / Inv-Fidelity than other methods and even the “ground truth” by only selecting the -O in -NO2 as explanations.

Original: In particular, the -0.234 Inv-Fidelity of GStarX means the selected subgraph has an even better prediction result than the original whole graph (0 Inv-Fidelity ) and the ground truth (-0.143 Inv-Fidelity ) because nodes not significant to the GNN prediction are removed.
Revised: In particular, the -0.234 Inv-Fidelity of GStarX means the selected subgraph has an even better prediction result than the original whole graph (0 Inv-Fidelity ) and the ground truth (-0.143 Inv-Fidelity ) because nodes not significant to the GNN prediction are removed.

Original: Fidelity metrics of baselines are inferior to GStarX because they include other non-discriminative carbon atoms despite they capture -NO2 to some extent.
Revised: Fidelity metrics of baselines are inferior to GStarX because they include other non-discriminative carbon atoms despite they capture -NO2 to some extent.

Original: This suggests that even though human experts identify -NO2 as the “ground truth” of mutagenicity, the GNN only needs -O to classify mutagenic molecules.
Revised: This suggests that even though human experts identify -NO2 as the “ground truth” of mutagenicity, the GNN only needs -O to classify mutagenic molecules.

Original: With the goal being understand model behavior, GStarX explanation is better.
Revised: With the goal being understand model behavior, GStarX explanation is better.

Original: Moreover, SubgraphX is the only baseline that has better H-Fidelity than the “ground truth”, but it
Revised: Moreover, SubgraphX is the only baseline that has better H-Fidelity than the “ground truth”, but it

Original: [Table] [Table]
Revised: [Table] [Table]

Original: can only capture one -NO2 because its search algorithm requires the explanation to be connected, so its Inv-Fidelity is not optimal.
Revised: can only capture one -NO2 because its search algorithm requires the explanation to be connected, so its Inv-Fidelity is not optimal.

Original: In fact, GNNExplainer, PGExplainer
Revised: In fact, GNNExplainer, PGExplainer

Original: [Table]
Revised: [Table]

Original: and SubgraphX can never generate explanations including only disconnected -O without -N like GStarX, because the former two solve the explanation problem by optimizing edges (as opposed to Equation 4), and the latter requires connectedness.
Revised: and SubgraphX can never generate explanations including only disconnected -O without -N like GStarX, because the former two solve the explanation problem by optimizing edges (as opposed to Equation 4), and the latter requires connectedness.

Original: More MUTAG explanation visualizations are in Appendix H.
Revised: More MUTAG explanation visualizations are in Appendix H.

Original: 5.3 Ablation study and analysis
Revised: 5.3 Ablation study and analysis

Original: Model-agnostic explanation.
Revised: Model-agnostic explanation.

Original: GStarX makes no assumptions about the model architecture and can be applied to explain various GNN backbones.
Revised: GStarX makes no assumptions about the model architecture and can be applied to explain various GNN backbones.

Original: We use GCN for all datasets in the major experiment above for consistency, and we now further investigate performance on two more popular GNNs: GIN and GAT.
Revised: We use GCN for all datasets in the major experiment above for consistency, and we now further investigate performance on two more popular GNNs: GIN and GAT.

Original: We follow [44] to train GIN on MUTAG and GAT on GraphSST2 4 , and show results in Table 2.
Revised: We follow [44] to train GIN on MUTAG and GAT on GraphSST2 4 , and show results in Table 2.

Original: For both settings, GStarX outperforms the baselines, which is consistent with results on GCN.
Revised: For both settings, GStarX outperforms the baselines, which is consistent with results on GCN.

Original: Efficiency study.
Revised: Efficiency study.

Original: The GStarX algorithm scales in O ( J ) with practical J ∝ |V| .
Revised: The GStarX algorithm scales in O ( J ) with practical J ∝ |V| .

Original: Following [44], we study the empirical efficiency of GStarX by explaining 50 randomly selected graphs from BBBP .
Revised: Following [44], we study the empirical efficiency of GStarX by explaining 50 randomly selected graphs from BBBP .

Original: We report the average run time in Table 3.
Revised: We report the average run time in Table 3.

Original: Our results for the baselines are similar to [44].
Revised: Our results for the baselines are similar to [44].

Original: GStarX is not the fastest method, but it is more than two times faster than SubgraphX. Since explanation usually doesn’t have strict efficiency requirements in real applications, considering GStarX generates higherquality explanations than the baselines, we believe the time complexity of GStarX is acceptable.
Revised: GStarX is not the fastest method, but it is more than two times faster than SubgraphX. Since explanation usually doesn’t have strict efficiency requirements in real applications, considering GStarX generates higherquality explanations than the baselines, we believe the time complexity of GStarX is acceptable.

Original: Explanation sparsity study.
Revised: Explanation sparsity study.

Original: To further study whether the obtained scores by GStarX are sparse, we follow [11] to evaluate an entropy-based sparsity measure on model output scores.
Revised: To further study whether the obtained scores by GStarX are sparse, we follow [11] to evaluate an entropy-based sparsity measure on model output scores.

Original: We show the average GStarX entropy-based sparsity on all datasets, and compare them with three reference score distributions on all n nodes in a graph. 1)
Revised: We show the average GStarX entropy-based sparsity on all datasets, and compare them with three reference score distributions on all n nodes in a graph. 1)

Original: An upper bound: Uniform(n), which represents the least sparse output.
Revised: An upper bound: Uniform(n), which represents the least sparse output.

Original: A practical lower bound: Uniform(0.25*n) which represents very sparse outputs with only top 25% of nodes.
Revised: A practical lower bound: Uniform(0.25*n) which represents very sparse outputs with only top 25% of nodes.

Original: 3) Poisson(0.25*n), which is a more realistic version of case 2).
Revised: 3) Poisson(0.25*n), which is a more realistic version of case 2).

Original: Results in Table 4 show the average entropy-based sparsity of GStarX is much lower than Uniform(n) and close to Poisson(0.25*n), which justifies the GStarX outputs are indeed sparse.
Revised: Results in Table 4 show the average entropy-based sparsity of GStarX is much lower than Uniform(n) and close to Poisson(0.25*n), which justifies the GStarX outputs are indeed sparse.

Original: A more detailed discussion of this metric and these three reference distributions is in Appendix A.5.
Revised: A more detailed discussion of this metric and these three reference distributions is in Appendix A.5.

Original:  Related work
Revised:  Related work

Original: GNN explanation aims to produce an explanation for a GNN prediction on a given graph, usually as a subgraph induced by important nodes or edges.
Revised: GNN explanation aims to produce an explanation for a GNN prediction on a given graph, usually as a subgraph induced by important nodes or edges.

Original: Many existing methods work by scoring nodes or edges and are thus similar to this work.
Revised: Many existing methods work by scoring nodes or edges and are thus similar to this work.

Original: For example, the scoring function of GNNExplainer [41] is the mutual information between a masked graph and the prediction on the original graph, where soft masks on edges and node features are generated by direct parameter learning.
Revised: For example, the scoring function of GNNExplainer [41] is the mutual information between a masked graph and the prediction on the original graph, where soft masks on edges and node features are generated by direct parameter learning.

Original: PGExplainer [25] uses the same scoring function as [41] but generates a discrete mask on edges by training an edge mask predictor.
Revised: PGExplainer [25] uses the same scoring function as [41] but generates a discrete mask on edges by training an edge mask predictor.

Original: SubgraphX
Revised: SubgraphX

Original: [44] uses the Shapley value as its scoring function on subgraphs
Revised: [44] uses the Shapley value as its scoring function on subgraphs

Original: [Table] [Table]
Revised: [Table] [Table]

Original: selected by Monte Carlo Tree Search (MCTS)
Revised: selected by Monte Carlo Tree Search (MCTS)

Original: [Table]
Revised: [Table]

Original: and GraphSVX [9] uses a least-square approximation to the Shapley value to score nodes and their features.
Revised: and GraphSVX [9] uses a least-square approximation to the Shapley value to score nodes and their features.

Original: While SubgraphX and GraphSVX were shown to perform better than prior alternatives
Revised: While SubgraphX and GraphSVX were shown to perform better than prior alternatives

Original: [Table]
Revised: [Table]

Original: as we show in Section 3, the Shapley value they try to approximate is non-ideal as it is non-structure-aware.
Revised: as we show in Section 3, the Shapley value they try to approximate is non-ideal as it is non-structure-aware.

Original: Although SubgraphX and GraphSVX use L -hop subgraphs and thus technically they use the graph structure, such structure usage are very limited in achieving structure-awareness as we show in Appendix G. While there are many other GNN explanation methods from very different perspectives, i
Revised: Although SubgraphX and GraphSVX use L -hop subgraphs and thus technically they use the graph structure, such structure usage are very limited in achieving structure-awareness as we show in Appendix G. While there are many other GNN explanation methods from very different perspectives, i

Original: [Table]
Revised: [Table]

Original: e. gradient analysis [28], model decomposition [1], surrogate models [37]
Revised: e. gradient analysis [28], model decomposition [1], surrogate models [37]

Original: [Table]
Revised: [Table]

Original: and causality [20, 21], we defer their details to Appendix
Revised: and causality [20, 21], we defer their details to Appendix

Original: C given their lesser relevance.
Revised: C given their lesser relevance.

Original: Cooperative game theory originally studies how to allocate payoffs among a set of players in a cooperative game.
Revised: Cooperative game theory originally studies how to allocate payoffs among a set of players in a cooperative game.

Original: Recently, certain ideas from this domain have been successfully used in feature importance scoring for ML model explanation [22, 32, 24].
Revised: Recently, certain ideas from this domain have been successfully used in feature importance scoring for ML model explanation [22, 32, 24].

Original: When used for model explanation, data features becomes players in the game, e.g. pixels for images, and the value of the game gives feature importance scores.
Revised: When used for model explanation, data features becomes players in the game, e.g. pixels for images, and the value of the game gives feature importance scores.

Original: The vast majority of works in this line, like the ones cited above, deem the Shapley value [30] to be the only choice.
Revised: The vast majority of works in this line, like the ones cited above, deem the Shapley value [30] to be the only choice.

Original: In fact, there are many other values with different properties and used in different situations in cooperative game theory.
Revised: In fact, there are many other values with different properties and used in different situations in cooperative game theory.

Original: However, to the best of our knowledge, only [4] mentions the Myerson value [26] in the context of proposing a connected Shapley (C-Shapley) value for explaining sequence data, and it is not directly comparable to ours for graph data.
Revised: However, to the best of our knowledge, only [4] mentions the Myerson value [26] in the context of proposing a connected Shapley (C-Shapley) value for explaining sequence data, and it is not directly comparable to ours for graph data.

Original: A detailed discussion of the Myerson value and the C-Shapley value can be found in Appendix
Revised: A detailed discussion of the Myerson value and the C-Shapley value can be found in Appendix

Original: F.
Revised: F.

Original: Our work follows the cooperative game theory approach to explain models on graph data using the HN value [15], which as we show is a better choice than the Shapley value given its structure-awareness.
Revised: Our work follows the cooperative game theory approach to explain models on graph data using the HN value [15], which as we show is a better choice than the Shapley value given its structure-awareness.

Original:  Conclusion and future work
Revised:  Conclusion and future work

Original: In summary, we study GNN explanation on graphs via node importance scoring.
Revised: In summary, we study GNN explanation on graphs via node importance scoring.

Original: We identify the non-structure-aware challenge of existing Shapley-value-based approaches and propose GStarX to assign importance scores to each node via a structure-aware HN value.
Revised: We identify the non-structure-aware challenge of existing Shapley-value-based approaches and propose GStarX to assign importance scores to each node via a structure-aware HN value.

Original: We also build connections between the HN value surplus allocation and GNN message passing.
Revised: We also build connections between the HN value surplus allocation and GNN message passing.

Original: GStarX demonstrates its superiority over strong baselines on chemical and text graph classifications.
Revised: GStarX demonstrates its superiority over strong baselines on chemical and text graph classifications.

Original: A limitation of GStarX is that the importance of different node feature dimensions is not explained.
Revised: A limitation of GStarX is that the importance of different node feature dimensions is not explained.

Original: One future work is to add this extension, which could be done by scoring a subset of nodes together with a subset of features each time.
Revised: One future work is to add this extension, which could be done by scoring a subset of nodes together with a subset of features each time.

Original: Another future direction is to exploit the rich cooperative game theory literature.
Revised: Another future direction is to exploit the rich cooperative game theory literature.

Original: Beyond the Shapley value, more values are possible for explaining ML models.
Revised: Beyond the Shapley value, more values are possible for explaining ML models.

Original: For graph data, edge-based values like [2] can potentially be applied to an alternative edge-based objective like Equation 4.
Revised: For graph data, edge-based values like [2] can potentially be applied to an alternative edge-based objective like Equation 4.

Original: Other values may be appropriate to more data types beyond graphs.
Revised: Other values may be appropriate to more data types beyond graphs.

Original: Acknowledgement
Revised: Acknowledgement

Original: This work was partially supported by NSF III-1705169, NSF 1937599, NSF 2119643, Okawa
Revised: This work was partially supported by NSF III-1705169, NSF 1937599, NSF 2119643, Okawa

Original: Foundation Grant, Amazon Research Awards, Cisco research grant USA000EP280889, Picsart Gifts, and Gifts
Revised: Foundation Grant, Amazon Research Awards, Cisco research grant USA000EP

Original: Checklist
Revised: Checklist

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] (c) Did you discuss any potential negative societal impacts of your work?
Revised: [Yes] (c) Did you discuss any potential negative societal impacts of your work?

Original: [No] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: [No] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 2.
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A] (b) Did you include complete proofs of all theoretical results?
Revised: [N/A] (b) Did you include complete proofs of all theoretical results?

Original: [N/A] 3.
Revised: [N/A] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: [Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
Revised: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
Revised: [Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] 4.
Revised: [Yes] 4.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] (b) Did you mention the license of the assets?
Revised: [Yes] (b) Did you mention the license of the assets?

Original: [Yes] (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [Yes] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No]
Revised: [No]

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Original: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
Revised: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Original: [N/A] 5.
Revised: [N/A] 5.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?[N/A] A Experiment details A.1 Dataset statistics
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: In Table 5, we provided the statistics of all datasets used in our experiments.

Original: [Table]

Original: A.2 Model architectures and implementation

Original: In Table 6, we provided the hyperparameters and test accuracy for the GCN model used in our major experiments.

Original: In Table 2, we provided the hyperparameters and test accuracy for the GIN and GAT model used in our analysis experiment.

Original: Most parameters are following [44], with small changes to further boost the test accuracy.

Original: We run all experiments on a machine with 80 Intel(R)

Original: Xeon(R) E5-2698 v4 @ 2.20GHz CPUs, and a single NVIDIA V100 GPU with 16GB RAM.

Original: Our implementations are based on Python 3.8.10,

Original: PyTorch 1.10.0, PyTorch-Geometric 1.7.1 [10], and DIG [23].

Original: We adapt the GNN implementation and most baseline explainer implementation from the DIG library, except for GraphSVX and OrphicX where we adapt the official implementation.

Original: For the baseline hyperparameters, we closely follow the setting in [44] and [9] for a fair comparison.

Original: Please refer to [44] Section 4.1 and [9] Appendix E for details.

Original: [Table]

Original: [Table]

Original: A.3 Exact formula for evaluation metrics

Original: Formulas for Fidelity , Inv-Fidelity , and Sparsity are shown in Equation 9, 10, and 11.

Original: In Equation 12, 13, and 14, we show formulas for normalized fidelity ( N-Fidelity ), normalized inverse fidelity ( N-Inv-Fidelity ), and harmonic fidelity ( H-Fidelity ).

Original: Both the N-Fidelity and N-Inv-Fidelity are in [ − 1 , 1] .

Original: The H-Fidelity flips N-Inv-Fidelity , rescales both values to be in [0 , 1] , and takes their harmonic mean.

Original: [Equation]

Original: [Equation]

Original: Let m 1 = N-Fidelity ( G , g ) , m 2 = N-Inv-Fidelity ( G , g )

Original: H-Fidelity

Original: [Equation]

Original: [Equation]

Original: =

Original: [Equation]

Original: A.4 Fidelity vs. sparsity plots

Original: In Table 1, we report the best H-Fidelity among 8 different sparsities for each method on each dataset.

Original: We also follow [44] to show the Fidelity vs. Sparsity plots in Figure 4 row1.

Original: Note that GraphSVX tends to give sparse explanations on some datasets, we still pick 8 different sparsities for it but mostly on the higher end.

Original: We also show the 1 - Inv-Fidelity vs. sparsity plots and the H-Fidelity vs. sparsity plots.

Original: Curves in all three plots are the higher the better.

Original: [Figure]
Revised: [N/A]

Original: corresponding to the results shown in Table 1.

Original: All three metrics are the higher the better.

Original: We see that

Original: GStarX outperforms the other methods

Original: A.5 Detailed entropy-based sparsity evaluation

Original: In Section 5.2 we study whether the obtained scores by GStarX are sparse and follow [11] to apply an entropy-based sparsity measure on scores.

Original: We now provide a more detailed discussion of this study.

Original: The entropy-based sparsity, as defined in Definition 2 in [11], is shown in the Equation 15 below.

Original: Here ϕ is the model output scores for a data instance, and ˜ ϕ

Original: [Equation]

Original: scores

Original: [Equation]

Original: The entropy-based sparsity helps us to understand how sparse an explanation is, before the scores are turned into hard explanation by thresholding or selecting top k. In Table 4, we show the average scores for GStarX on all datasets, and compare them with three reference cases.

Original: The entropy of uniform distribution over all n nodes in a graph, i.e., Uniform(n), which represents the least sparse output and is an upper bound of entropy-based sparsity. 2)

Original: The entropy of uniform distribution over the top 25% nodes in a graph, i.e., Uniform(0.25*n), where probabilities of the bottom 75% nodes are set to zero.

Original: This case is very sparse since 75% of nodes are deterministically excluded, which can be treated as a practical lower bound of entropy-based sparsity.

Original: The entropy of Poisson distribution with mean 0.25*n, i.e. Poisson(0.25*n), which is a more realistic version of the sparse output in case 2).

Original: Instead of setting all 75% of nodes to have probability zero, we assume the probabilities for tail nodes decrease exponentially as a Poisson distribution while the mean is kept the same as in case 2).

Original: Results in Table 4 show that the average entropy-based sparsity of GStarX is between

Original: Uniform(0.25*n) and Uniform(n) and close to Poisson(0.25*n), which justifies the GStarX outputs are indeed sparse.

Original: B GStarX for node classification

Original: Even though the GStarX algorithm is stated for graph classification, it works for node classification as well.

Original: This can be easily seen as the GNN node classification can be covert to classify an ego-graph.

Original: Given a graph G with V = { u 1 , . . .

Original: , u n } .

Original: Node classification on u i with an L -layer GNN can be converted to a graph classification.

Original: The target graph to classify will be the L -hop ego-graph centered at u i , because this is the receptive field of the GNN for classifying u

Original: i and nodes further away won’t influence the result.

Original: The label of the graph will be the label of u i .

Original: In this case, the final readout layer of the GNN will be indexing u

Original: i instead of pooling.

Original: Given this kind of conversion, everything we showed in Section 4 follows.

Original: C More related work

Original: GNN explanation continued Besides the perturbation-based method we mentioned in Section 6, there are several other types of approaches for GNN explanation.

Original: Gradient-based methods are widely used for explaining ML models on images and text.

Original: The key idea is to use the gradients as the approximations of input importance.

Original: Such methods as contrastive gradient-based (CG) saliency maps,

Original: Class Activation Mapping (CAM), and gradient-weighted CAM (Grad-CAM) have been generalized to graph data in [28].

Original: Decomposition-based methods are a popular way to explain deep NNs for images.

Original: They measure the importance of input features by decomposing the model predictions and regard the decomposed terms as importance scores.

Original: Decomposition methods including Layer-wise

Original: Relevance Propagation (LRP) and Excitation Backpropagation (EB) have also been extended to graphs [28, 1].

Original: Surrogate-based methods work by approximating a complex model using an explainable model locally.

Original: Possible options to approximate GNNs include linear model as in GraphLIME [17], additive feature attribution model with the Shapley value as in GraphSVX [9], and Bayesian networks as in [37].

Original: GNN explainability has also been studied from the causal perspective.

Original: In [20, 21], generative models were constructed to learn causal factors, and explanations were produced by analyzing the cause-effect relationship in the causal graph.

Original: D Properties of the Shapley value

Original: The Shapley value was proposed as the unique solution of a game ( N, v ) that satisfies three properties shown below, i.e. efficiency , symmetry , and additivity [30].

Original: These three properties together are referred as an axiomatic characterization of the Shapley value.

Original: The associated consistency properties introduced in Section 4.1 provides a different axiomatic characterization.

Original: Property D.1 ( Efficiency ) .

Original: [Equation]

Original: Property D.2 ( Symmetry ) .

Original: If v ( S ∪ { i } )

Original: = v ( S ∪ { j } ) for all S ∈ N \{ i, j } , then

Original: [Equation]

Original: Property D.3 ( Additivity ) .

Original: Given two games ( N, v ) and ( N, w ) ,

Original: [Equation]

Original: The efficiency property states that the value should fully distribute the payoff of the game.

Original: The symmetry property states that if two players make equal contributions to all possible coalitions formed by other players (including the empty coalition), then they should have the same value.

Original: The additivity property states that the value of two independent games should be added player by player.

Original: It is the most useful for a system of independent games.

Original: E Properties and calculation of the HN value E.1 Consistency and associated games

Original: One reason for the Shapley value’s popularity is its axiomatic characterization , indicating that it is the unique solution that satisfies a set of desirable properties (see Appendix D).

Original: Then [14] proposed a new axiomatic characterization of the Shapley value based on a different associated consistency property.

Original: The consistency property is a common analysis tool used in game theory [16, 7, 31, 27].

Original: The idea is to analyze a game ( N, v ) by defining other reduced games ( S, v S ) for S ⊆ N , and a solution function ϕ is called consistent when ϕ ( N, v ) yields the same payoff as ϕ ( S, v S ) on each S .

Original: When ( S, v S ) is defined with desired properties, these good properties can be enforced for a solution by requiring consistency.

Original: The associated consistency in [14] is a special case of consistency between ( N, v ) and only one other game ( N, v ∗ ) , which is called the associated game .

Original: [14] shows that a carefully designed associated game uniquely characterizes the Shapley value.

Original: Associated consistency is also the key idea of the HN value.

Original: E.2 Limit game and the axiomatic characterization

Original: The HN value is established on a special associated game as we discussed in Section 4.1.

Original: We can actually write this associated game in a more compact matrix form, where we slightly abuse notation and use v and v ∗ τ to represent vectors of payoffs for all S ⊆ N under the original and associated game respectively.

Original: In other words, v ( S ) , which is used to represent evaluating the coalition S using the characteristic function v , now can also be interpreted as indexing the vector v with index S .

Original: Lemma E.1.

Original: A matrix form of the associated game ( N, v ∗ τ , G ) is given by

Original: [Equation]

Original: The matrix H { τ,n, G} depends on the hyperparameter τ , number of players n , and the graph G .

Original: When these variables are clear from the context, we drop them and write v ∗ τ

Original: = H v .

Original: Please refer to [15] for the proof of Lemma E.1.

Original: With the matrix form, we can define the limit game.

Original: Definition E.2.

Original: Given a game ( N, v, G ) , its limit game ( N, ˜ v, G ) is defined by

Original: [Equation]

Original: Notice that although the matrix H is constructed from the associated game and depends on τ , the powers of H actually converge to a limit independent from τ , when τ is sufficiently small.

Original: The general condition depends on the actual graph, but 0 < τ < 2 n is proven to be sufficient for the complete graph case [14].

Original: As we discussed in Section 4.1, the limit game can be seen as constructing associated games repeatedly until the characteristic function converges.

Original: An axiomatic characterization of the HN value regarding its uniqueness is given by the following theorem based on the limit game.

Original: The associated consistency is the core property related to this work.

Original: We encourage the readers to check [15] for the other two properties.

Original: Theorem E.3.

Original: There exists a unique solution ϕ that verifies the associated consistency, i.e. ϕ i ( N, v, G ) =

Original: ϕ

Original: i ( N, v ∗ τ , G ) , inessential game, and continuity.

Original: ϕ is given by

Original: [Equation]

Original: E.3 The algorithm for computing the HN value

Original: We show the algorithm for Compute-HN-MC (Algorithm 3) mentioned in Section 4.3.

Original: The algorithm is a combination of Equation 16, 17, and 8.

Original: F The Myerson value and the C-Shapley value F.1 The Myerson value

Original: In the study of cooperative games, [26] proposed to characterize the cooperation possibilities between players using a graph structure G , which leads to the communication structure introduced in Section

Original: Input: Graph instance G with nodes V = { u 1 , . . . , u n } , characteristic function v , hyperparameter τ , maximum sample size m , number of samples J

Original: Let ψ 1 , . .

Original: . , ψ n be n empty lists for j = 1 to J do

Original: Sample g S j from G s.t.

Original: S

Original: j = { u j 1 , . . . , u j l } and l < m ϕ j = Compute-HN

Original: ( g S j , S j , v ( · ) , τ ) for k = 1 to l do

Original: Append ϕ jk to ψ j k end for end for

Original: Set ϕ

Original: i to be the mean of ψ

Original: i

Original: Return: ϕ 2.2 and the Myerson value as a solution for this special type of games ( N, v, G ) .

Original: The Myerson value is closely related to the Shapley value.

Original: In fact, it is the Shapley value on a transformed game where players are partitioned by the graph.

Original: We now formally introduce the partition and the transformed game.

Original: Definition F.1 ( Partition ) .

Original: Given a set of players N and a graph G .

Original: For any coalition S ⊆ N , the partition of S is denoted by S/ G and defined by

Original: S/ G = {{ i | i and j are connected in S by G}| j ∈ S } and a member of the set S/ G is called a component of S .

Original: Definition F.2 ( Transformed Game ) .

Original: Given a game ( N, v, G ) , we can transform it to a new game v/ G such that for all S ⊆ N

Original: [Equation]

Original: Intuitively, given a coalition S , the transformed game treats each connected component of S as independent, evaluates them separately, and sums their payoff as the payoff of S .

Original: The Shapley value has an axiomatic characterization that uniquely determines it as we introduced in Appendix D. Likewise, the Myerson value was proposed to be a unique solution that satisfies the component efficiency and the fairness property defined below.

Original: Property F.3 ( Component Efficiency ) .

Original: For a game ( N, v, G ) and any connected component S ∈

Original: N/ G , a solution is component efficient if

Original: [Equation]

Original: Property F.4 ( Fairness ) .

Original: For a game ( N, v, G ) and any edge ( i, j ) in G , let ˜ G be G with the edge ( i, j ) removed, a solution is fair if

Original: [Equation]

Original: The component efficiency property is an extension of the regular efficiency property to games with a communication structure.

Original: It requires efficiency to hold for each disconnected piece because these pieces are assumed as independent from each other.

Original: The fairness property states that if breaking an edge ( i, j ) changes the value of player i , then the value of player j should be changed by the same amount.

Original: Theorem F.5 ( Myerson Value ) .

Original: There exists a unique solution ϕ of game ( N, v, G ) satisfying component efficiency and fairness.

Original: With ˜ ϕ represents the Shapley value, the solution is given by the formula

Original: [Equation]

Original: For games with a communication structure, the Myerson value is a better choice than the Shapley value as it uses the graph structure.

Original: However, it also suffers from some criticisms.

Original: For example, the fairness assumption may not be realistic.

Original: When an existing edge is broken, the value changes for players on the two edge ends can be asymmetric.

Original: Intuitively, if the edge connects a popular hub player i to a leaf player j , then the change of i can be less significant than j since j becomes isolated when ( i, j ) is removed.

Original: This is also the case when the game value is used for model explanation.

Original: For example in Figure 1 (b), when the edge ("good", "quite") is broken, the value of "quite" should change a lot.

Original: It used to contribute positively together with "good", and thus gets some payoff allocation, but it now becomes an isolated node, which is neutral by itself.

Original: On the other hand, the word "good" can still contribute positively by itself and interact with other nodes through its other edges, and thus its value shouldn’t change too much.

Original: Because of such criticisms, we choose to use the HN value as our scoring function, which characterizes the value by associated consistency rather than fairness.

Original: F.2 The C-Shapley value

Original: The Myerson value was also mentioned in [4] for the model explanation on text, where the C-Shapley value was proposed as an approximation of the Shapley value, and it was claimed to be equal to the Myerson value.

Original: We have discussed why Shapley value and Myerson are not-ideal choices for explaining graph data in Section 3 and Appendix F.1.

Original: These are partially the reason why our HNvalue-based method is better than the C-Shapley value.

Original: However, the major reason why we don’t do a direct comparison to the C-Shapley value as a baseline is that its formula only works for line graphs like sequence data, and not even all nodes in line graphs.

Original: In contrast, our target task is general graph prediction for graphs with possibly complicated topological structures.

Original: We now clarify a mistake of the C-Shapley value formula and explain why it won’t work for general graphs.

Original: The notations are following the [4], where d is the number of players corresponding to n in our notation, and [ d ] corresponding to N .

Original: The formula for the C-Shapley value is given in Equation 6 in Definition 2 in the paper, and it is stated for "a graph G" without mentioning any assumptions of the graph.

Original: However, from the proof of this formula in Appendix B.2 in the paper, the line graph assumption can be seen in two places.

Original: The first place is Equation 20, where the set C is explicitly defined only for subsequences.

Original: The second place is Equation 22, the first line converts (cid:80) A : U S ( A )

Original: =

Original: U to (cid:80) d

Original: −| U |− 2 i = , which is implicitly saying

Original: V S ( A ) can be picked from all d but | U | − 2 nodes.

Original: However, this conversion is only possible when there are exactly 2 edges between U and [ d ] \ U , i.e. the middle part of a line graph.

Original: If there are l edges between U and [ d ] \ U , then the summation should go up to d − | U | − l .

Original: When l = 0 , i.e. U equals [ d ] or a connected component of [ d ] , no partition is needed and the coefficient simply evaluates to 1.

Original: By correcting all these cases, the final formula for the C-Shapley value coefficients of marginal contributions thus becomes

Original: [Equation]

Original: for l > 0 , and 1 for l = 0 .

Original: The correct formula for the C-Shapley value of general graphs will be

Original: [Equation]

Original: with l represents the edges between U and [ d ] \ U and C represents all connected subgraphs in [ d ] containing i .

Original: To verify this formula with the 3-node toy graph in Figure 5.

Original: When computing the value of node(left), the three connected components containing 0 are C

Original: = {{ 0 } , { 0 , 1 } , { 0 , 1 , 2 }} .

Original: Since 0 is an

Original: Shapley

Original: C_Shapley Myerson HN 1/ 1/ 1/ 1/ 1/ 1/ 1/  1/ 1/ 1/  1/ 1/ 1/ 1/

Original: Shapley

Original: C_Shapley Myerson HN 1/ 1/ 1/ 1/ 1/ 1/ 1/ 1/ 1/    1/ 1/ 1/ 1/ end node and has no leaf nodes to its left, l for these three components will be 1, 1, and 0 respectively.

Original: According to our new formula in Equation 22, the coefficients will be 12 , 16 , and 13 respectively, with the disconnected { 0 , 2 } case removed.

Original: This matches the original idea of Myerson value, where the { 0 , 2 } − { 2 } case is reduced to the { 0 } − ∅ case, which turns the Shapley coefficients from [ 13 , 16 , 16 , 13 ] to [ 13 + 16 , 16 , 16 − 16 , 13 ] , which is [ 12 , 16 , 0 , 13 ] .

Original: However, the original C-Shapley formula from Equation 6 in the [4] evaluates to [ 13 , 112 , 0 , 130 ] , which doesn’t match the Myerson value and not even sum up to 1.

Original: Another example of computing the value of node 1 is shown in Figure 5 right.

Original: The C-Shapley, even with the correct formula, eventually boils down to an approximation of the

Original: Shapley value or the Myerson value, which as we discussed are less ideal than the HN value.

Original: Also, the correct formula in Equation 22 requires generating all possible subgraphs

Original: U containing the node i and specify the edges between U and [ d ] \ U .

Original: This makes the computation very complicated, we thus skip the comparison to the C-Shapley value.

Original: G Use the graph structure via an L-hop cutoff

Original: Although the Shapley value itself is not structure-aware, we do note the existing Shapley-valuebased GNN explanation methods use an L-hop cutoff to help approximate the Shapley value [44, 9].

Original: Technically, this operation uses the graph structure, so we can’t strictly refer to these explanation methods as not structure-aware.

Original: However, we argue that the L-hop cutoff is a naive way of utilizing the graph structure.

Original: It has several concerns, and it is not the same structure-aware as the HN value.

Original: The L-hop cutoff approximates the Shapley value of node i by considering only the L-hop neighbors of i when explaining an L-layer GNN.

Original: The rationale of this operation is that an L-layer GNNs only propagate messages within L-hops so a node more than L-hop away from i has never passed any messages to i which means no interactions are possible.

Original: In existing Shapley-value-based GNN explanation methods, this L-hop cutoff operation was meant for reducing the exponentially growing computations of the Shapley value, and the ultimate goal is still to compute the Shapley value.

Original: The

Original: L-hop cutoff operation has several issues making it a less desirable choice.

Original: 1) Even meant to save computation, there are still many nodes involved in the computation after applying the L-hop cutoff since the number of nodes grows exponentially as L grows.

Original: For advanced GNNs, the L can be large.

Original: When L is larger than the diameter of the graph, which is actually the case for many recent deep

Original: GNNs, the L-hop cutoff is not effective anymore.

Original: When constructing coalitions of nodes within the local graph of L-hops, the computation still follows the Shapley value formula.

Original: This means the useful graph structure information among these nodes is forfeited which causes the structure-awareness concern of Shapley value as we discussed in Section 3,

Original: H More explanation visualizations

Original: Under the same setting as Figure 3, we visualize more explanations in 6.

Original: [Figure]

Original: [Figure]

Original: “occasionally funny , always very colorful and enjoyably overblown in the traditional almodóvar style

Original: [Figure]

Original: [Figure]