A True-to-the-model Axiomatic Benchmark for Graph-based Explainers
Authors: Corrado Monti
Paolo Bajardi
Francesco Bonchi
André Panisson
Alan Perotti
Venue: Accepted by TMLR ()

Abstract: Regulators, researchers, and practitioners recognize the urgency of explainability in artificial intelligence systems, including the ones based on machine learning for graph-structured data. Despite the large number of proposals, however, a common understanding of what constitutes a good explanation is still lacking: different explainers often arrive at different conclusions on the same problem instance, making it hard for practitioners to choose among them. Furthermore, explainers often produce explanations through opaque logic hard to understand and assess -- ironically mirroring the black box nature they aim to elucidate.

Recent proposals in the literature for benchmarking graph-based explainers typically involve embedding specific logic into data, training a black-box model, and then empirically assessing how well the explanation matches the embedded logic, i.e., they test truthfulness to the data. In contrast, we propose a true-to-the-model axiomatic framework for auditing explainers in the task of node classification on graphs.
Our proposal hinges on the fundamental idea that an explainer should discern if a model relies on a particular feature for classifying a node.
Building on this concept, we develop three types of white-box classifiers, with clear internal logic, that are relevant in real-world applications. We then formally prove that the set of features that can induce a change in the classification correctly corresponds to a ground-truth set of predefined important features. This property allows us to use the white-box classifiers to build a testing framework. 

We apply this framework to both synthetic and real data and evaluate various state-of-the-art explainers, thus characterizing their behavior. Our findings highlight how explainers often react in a rather counter-intuitive fashion to technical details that might be easily overlooked. Our approach offers valuable insights and recommended practices for selecting the right explainer given the task at hand, and for developing new methods for explaining graph-learning models.

Published in Transactions on Machine Learning Research (4/2024)
A True-to-the-model Axiomatic Benchmark for Graph-based Explainers
Corrado Monti
comndo.monti@ceetaa.ccc
CENTAI Institute, Turin, Italy
Paolo Bajardi
puolo.bajoordi@eeettaa.cc
CENTAI Instibute, Turin, Italy
Francesco Bonchi
Phacesso..honchi@@eentt...e.
CENTAI Institute, Turin, Italy
André Panisson
andrr.panisson@eeettad.e
CENTAI Institute, Turin, Italy
Alan Perotti
alam.perotti@centai.ce
CENTAI Institute, Turin, Italy
Reviewed on OpenReview: https: // opt/r/veew.eeeu net/ forum? id= HSQTv3R81z
Abstract
Regulators, researchers, and practitioners recognize the urgency of explainability in artificial intelligence systems, including the ones based machine learning for graph-structured data. Despite the large number of proposals, however, a common understanding of what constitutes a good explanation is still lacking: different explainers often arrive at different conclusions on the same problem instance, making hard for practitioners to choose among them. Furthermore, explainers often produce explanations through opaque logic hard to understand and assess ironically mirroring the black box nature they aim to elucidate.
Recent proposals in the literature for benchmarking graph-based explainers typically involve embedding specific logic into data, training a black-box model, and then empirically assessing how well the explanation matches the embedded logic, i.e., they lest trulhfulnes to the dala. In contrast, we propose prue-to-the-mode ariomatic fromework for auditing explainers in the task of node classification graphs. Our proposal hinges the fundamental idea that an explainer should discern if model relies on particular feature for classifying node. Building on this concept, develop three types of white-box classifiers, with clear internal logic, that are relevant in real-world applications. We then formally prove that the set of features that can induce a change the classification correctly corresponds to a ground- truth set of predefined important features. This properly allows us to use the white-box classifiers to build testing framework.
We apply this framework to both synthetic and real data and evaluate various state-of-the- art explainers, thus characterizing their behavior. Our findings highlight how explainers often react in a rather counter-intuitin fashion to technical details that might be easily overlooked. Our approach offers valuable insights and recommended practices for selecting the right explainer given the task hand, and for developing new methods for explaining graph-learning models.

---

Published in Transactions on Machine Learning Research (4/2024)
1 Introduction
The success of machine learning methods in solving problems over graph-structured data has spurred a great deal of applications in variety of domains, ranging from biology to finance, from social media to neuroscience. However, these models - and in particular graph neural networks (GNNs) - are often black boxes, as they have no intrinsic mechanism to provide lumann-undesstnndabbe explanations of their inner decision logic (et al. 2020). Increasing concerns from regulators, practitioners, algorithmic-assisted decision- takers, and subjects of algorithmic decisions (e.g. patients, customers etc.) boosted the research effort devoted to the development of novel explainability techniques for graph classification (sce Yuan ct al. (2020) for a recent survey). Regardless of the great deal of attention, common underslanding of what makes a good explanation is still lacking, As a consequence, different explainers often arrive to completely different conclusions on the same problem instance. Morcover, those explainers are themselves, more often than not, boxes producing explanations through opaque logic hard to understand and assess. Because of these reasons, while the usage of explainers for tabular data is now common (e.g., SHAP, Lundberg & Lee| (2017)) their adoption in practice for node classification in graphs is lagging behind: domain experts have no clear way to understand whether an explanation can be reliable, or to decide which explainer would be appropriate for specific problem.
Motivating example. Let us consider the following scenario. Imagine we have a model running in pro- duction in a banking system, that employs the network of financial transactions, together with attributes of its users, to determine a risk score for each user. The role of an explainer in this setting might be to assert the user's right to explanation as required by EU regulations (Selbst & Powles 2018 Goodman & Flaxman 2017). In particular, a user might want to know if a particular protected feature (e.g. ethnicity or gender) of a node, or of its neighbors, is employed by the model in making an automated decision. A watchdog might analyze such a model using an explainer. Our goal is therefore to ensure that a given explainer can correctly diagnose whether model using a feature or not. While this task might be straightforward in tabular data, in node classification in graphs specific feature of a node (e.g. ethnicity) might not be used directly to classify it, but it is still possible that the feature of its neighbors might be employed.
Proposed framework. In this work, we propose a rigorous framework aimed at assessing whether node- classification explainer can identify when given model uses a certain feature to produce the classification.
Recent efforts in this direction (Rathee et al. 2022 Longa et al., 2022 Amara et al. 2022 Jaume et al. 2021
typically adopt the following approach: they (1) implant some target patterns in synthetie training set,, (2) train black-box node classifier, (3) apply the explainer under scrutiny, (4) finally measure empirically the adherence of the produce explanation to the implanted logic, i.e., they test truthfulness to the data. Such an approach to auditing explainers in inherently complex, as it aims at assessing a software (the explainer), which in turn analyzes another ware (the model), which in turn is the output of another software (the training algorithm). This process has many dependencies; for instance, it depends on the specific learning algorithm and its parameters. All these dependencies can make the overall auditing process unstable. Moreover, there is degree of uncertainty about whether the trained black box has learned the intended logic (i.e., the planted patterns). Departing from this literature, we propose an axiomatic framework aimed at assessing whether a given explainer is true to the model (Chen et al. (020)): the explainer should capture the logic underlying the classification performed by the model, and not the patterns planted in the data. In fact, although present in the training data, those patterns might. not be used by the classifier. Our goal is thus to evaluate the accordancc of the explainer with the model itself, without making assumptions on what the model might have learned from the data. By measuring the concordance of explainers directly with the model logic, we can guarantee that our measure is not affected by any complex interaction between features in the data.
Our goal is formalized in a simple axiom: explainers should be able to differentiate whether a feature is used by a given model or not. Towards the operationalizaatinn of this inluition, we adopt a simple definition of what an important feature is within our framework. We define a feature f important for the classification of node e by model M, if and only if changing it, while maintaining all the other features unchanged, changes the classification of v. By no means this can be considered a general definition of feature importance in machine learning, however, within the strict limits of our framework we consistently use "important" with
2

---

Published in Transactions on Machine Learning Research (4/2024)
Node Classification
Graph
Features
True Importances
G (V,
X R Mi-
F'
Whitsbox node classifier
Ma,x
Explaining
Characterization
Explainer
3
Feature Importance Fidellity
Eslimated Importances
B
Our proposed framework. We start from a graph G with node features. We randomly mark subsets of features (F") as important: this is the ground truth for evaluations. We also white-box node classifier M implementing known logic, with ground truth importance for the features. We then feed these components to the explainer &, thus obtaining an explanation as estimated importance for the features, which are then compared to the ground-truth.
Figure 1:
this simple yet powerful semantic. We then axiomatize that an explainer should assign, to any important feature, higher explanation score than any unimportant feature.
We then devise three "white-boz models", ", i.c., classifiers whose internal logic is visible, in which able to hard-code which features are "important". Our white boxes have different logic and cover relevant scenarios in real-world applications of node classification: a local model, where the class of a node depends on a subset of its own features; a neighborhood model, where the class depends on the features of some of its neighbors; and a two-hop model where the class depends on the features of the nodes that are more likely to be reached in a two-hop random walk. All these models can also keep in consideration the degree of node, which often valuable for applications. For each white box we justify its practical relevance and discuss its PyTorch implementation allows us to prove (Theorem 1) that the set of features that are able to induce a change in the classification, correctly corresponds to predefined set of important features. Thanks to this theorem, we can develop our quantitative benchmark which, taking as input a given explainer, verifies whether it respects our axiom. A schematization of our framework is given in Figure
mossage-passing GNN. The simplicity of white boxes
Our findings. We apply our framework to several state-of ofate-of-the-rr explainers, carefully selected to cover all the main classes of approaches in the literature. Our analysis uncovers important phenomena emerging different explainers and highlights key differences semanties across explainers that often overlooked.
Our findings show that explainers based on surrogates can deliver an incomplete explanation when the surrogate they use does not faithfully represent the given model explain. For instance, our framework detects that GraphLIME (Huang al.] 2022) is not able to model correctly the feature of its neighbors.
neighborhood model, where the class of node depends
Our framework also shines light on peculiar behaviors that could be casily overlooked. For instance, we discovered that some explainers interpret feature importance features. When this is the case, zero-valued features cannot be, by definition, important.
difference from zero-valued
Furthermore, some explainers define important features only as those that positively contribute the classification of a node to the output class, while others consider both positive and negative contributions important.
3

---

Published in Transactions on Machine Learning Research (4/2024)
These crucial differences in semantics among explainers highlighted by our auditing framework, typically lead to completely different behaviors: without proper systematization, they can be an obstacle to clarity. The goal of our auditing framework is exactly to help assess and clarify the behavior of explainable graph learning techniques, aiding practitioners in choosing explainability methods that can correctly expose the criticality of graph learning applications.
Summary of contributions and paper roadmap. The technical contributions of this work can be summarized as follows:
We present quantitative benchmark based on three different white-box classifiers with known inner logic, to assess if an explainer can recognize whether a model is using some features (Section 3). For each white box we discuss its PyTorch implementation as message-passingg GNN. We prove (Theo- rem 1) that an explainer respects our axiom, then its output must necessarily be in accordance with the white-box ground truth.
We use our framework to audit five explainers, chosen representatives of the laxonomic classes of node-level explainers identified by Yuan et al. 20200) (Section
We show how the potential limits diagnosed by our framework are a product of the internal logic of each explainer (Section 5 we thus provide suggestions for their users and for designers of novel explainers. We are also the first to adopt salient deconwolutional networks (Mahendran & Vedaldi, 2016), an explainer that was originally introduced to explain image classification, to explain node classification on graphs. Such a method performs perfectly on our benchmark.
We provide code to let other rescarchers use our framework to test or develop new graph explainers. Our code available at httpps://github.ccmm http://ww.ccommoooooooooooooaa/a --a
Finally, a key contribution of this work is to show that radically different approach to assessing explainers, based on testing mussfuhnes-tt-the-moode through white boxes with hard-coded logic, is possible and can produce important insights. Next section discusses the most relevant related literature.
2 Related Work
Graphs serve as fundamental structures in myriad of real-world domains, and consequence graph neural networks (GNNs) have evolved as cornerstone in machine learning. Notable state-of-the art ap- proaches include Graph Convolutional Networks (GCNs)/Wellingg & Kipf, 2016), Graph Attention Networks (GATs) Veličković et al. 2017). and GraphSAGE (Hamilton et 2017). The effectiveness of these models comes al the cost of a remarkable opacily, which is especially problematic domains where understanding and trust in model predictions are crucial.
In response, several explainers for GNNs have been developed, using techniques like gradient-bosed bosed meth- ods (Integrated Gradients, Sundararajan et al. (211))) perturbation analysis (GNNExplainer, Ying et al. (019)), surrogate models (GraphLIME, [Haang et al. (0022) ), decomposition approaches (Layer-Relevane Relevance Propagation, Bach et al. (2015)), and substructure baseed methods (GraphShap, Perotti et al. (0023). Sub- graphX, Yuan et al. (2021)). (). In turn, these alternative solutions produce explanations which are hard to evaluate and compare, making it hard to decide which explainer would be appropriate for a specific problem.
This gap is particularly evident in the context of graah--tructured data due to its complexity and diversity. Several frameworks and desiderata have been suggested for evaluating explanations (Langer et al., 2021; van der Waa et al. 2021: Rosenfeld] 2021 Amparore et al., 2021 Agarwal et al. 2022a), but these often do not cater specifically to graph-based models, failing to capture their unique properties. The need for graph-specific evaluation frameworks is underscored in the taxonomic survey by (Yuan et al., 2020),, which, while focusing on explainability methods for GNNs, does not address the auditing problem in depth.
Auditing efforts, such as those by Agarwal et al. (0022b), provide valuable insights but often rely on synthetic empirical data. This reliance on approximations lead potential discrepancies between the uct and
4

---

Published in Transactions on Machine Learning Research (4/2024)
perceived effectiveness of explainers. Other proposals are strongly domain-specific: for instance, Jaume al. (2021) p propose a set of quantitative metrics based on class separability statistics using pathologically relevant concepts, thus quantifying the alignment of seprittmmicallyyggnerrtee explanations with known concepts in pathology. Besides the impossibility of generalising these metrics to other domains, these approaches highlight the lack of existing quantitative metrics for evaluating graph-based explanations. Other recent effort is devoted to develop benchmarks for the evaluation of graph-based explanation. BAGEL (Rathee et al. 2022) focuses both on explanation usability, measuring explanation sparsity and plausibility, and on explanation correctness, through the metries of faithfulness (ability to characterize model's inner logic) and correctness (ability to recognize the externally injected correlations). [onga et al. (2022) propose similar comparative survey, where the implemented metrics are plausibility (consistency between the explainer mask and ground truth mask) and fidelity (consistency between the model prediction on the full graph and on the explanation subgraph). Remarkably, edge masks are converted to node masks. Finally, Amara et al. (2022) introduce the characterization score, combining necessary-basce and sufficient-based fidelity metrics.
As already highlighted in the Introduction, all these approaches train black-box models, such as GCN and GAT, and fundamentally root their metrics querying these models on perturbed and/or masked data, i.c., they test truthfulness to the data. Interostingly, in all the mentioned papers, the authors cither explicitly acknowledge the strong limitation of assuming that the explanation ground truth is actually picked up by the model to make its decision or constrain their analysis to scenarios where the model almost memorized the injected patterns to achieve almost perfect classification. Conversely, we propose a true-to-the- model axiomatic benchmark by means of white-box models with known internal logic, which allows a direct comparison between models and explanations, by-passing the approximation and uncertainty introduced by the training process.
3 A True-to-the-modee Benchmark
We consider binary node classification on a node-attributed directed graph, defined over a set of nodes V and features F. In this setting a model M can be seen as function that takes graph G G= (V, F) with E' C 2VV, node-feature matrix X E RRVXX and node E V, it returns the probability that belongs to the positive class:
M: VVV x VIXIFI, x V + [0,1]]
Note that the graph G, being always defined over the same set of nodes V, is completely identified solely by the edge set E E 2VXV, For simplicity, we omit E and X from M(E,X,v), when clear from the context, and just denote M(v) the outcome of model M on node v.
In this setting, an instance-level explainer is a function that takes the model M, the edge set E, the node- feature matrix X, node and returns explanation for M(U), i.e., score of importance for each feature f E F. Again, we omit E and X when clear from the context and just denote the explainer as function EM V + F Here Ev(()= = 3 denotes the explanation for M(v), where the vector 3 determines the explanation scores: By is the explanation score given to the feature f E F.
3.1 The axiom: important features should receive higher importance score than non-important ones
We start simple, yet fundamental, intuition that we want the explainers to respect: features truly employed by the model to perform a classification, should receive higher explanation scores than features that are not used. We thus adopt a simple definition of what an importand feature is within our framework: a fra ʃ important for the classification of a node v by a model M, if and only if changing it, while maintaining all the other features unchanged, changes the classification of v.
Definition 1. Given the graph G = EV,E) with node-feature matriz X E VIXIIII we say that a feature I' E is important for the prediction of a model M on node u E iff il exists another matrir Y E RIVIxIFI which is identical to X ezcept for the column f (i.e., = = Iu.f, Yf & f such that M(X,0) #
M(Y, v). We denote (v) V the set of important features for the classification of
TTo the best of knowledge, not yet, reviewel..
5

---

Published in Transactions on Machine Learning Research (4/2024)
When clear from the context, we denote the set of important features for node v, simply by Ft. We can now formalize our starting intuition in an axiom of the desired behavior of explainers w.r.t. important features. Recall that By denotes the importance score given by the explainer to the feature f E F.
Axiom 1. Let EM be an explainer producing importance scores Em(r) = B when given model M and node v. Let F denote the set of important features (Defination 7). It must hold that B, < 3r vf & F ff" E F..
We next introduce a benchmark of three simple models aimed at stressing a given explainer w.r.t. Axiom 1. Such models are "white bozes" " (in contrast to black boxes) meaning that their internal logic is visible. In particular, they are geared on a predefined (ground truth) sct F' C F of important features, which is encoded inside each model as binary vector 3 E {0, 131FI where By = 11 After introducing such white boxes (Section 3.2), we prove that, for each of them, the ground truth set F" and the set of important features coincide (Theorem 1). We finally (Section 3.3 measure how much the explanation 3 produced by the explainer adheres to the ground truth B, or in other terms, to which extent the explainer respects Axiom 1. It worth stressing that our test is a nccessary, but not sufficient, condition for an explainer to be considered valid tool: passing our lest, does not imply that an explainer might not have other issues, on the other hand, failing our simple test clearly highlight some important drawbacks of the given explainer.
3.2 The white-box models
We first provide a general schema common to all three white boxes, then we introduce them in details. For each of them, we discuss (i) their relevance to real-world applications of node classification, and (ti) their PyTorch implementation as wressage-pssingg graph neural networks.
As pointed out earlier, each white box receives, as part of the input, the set. F' C F of ground-truth features which is encoded as a binary vector B. Inside cach white-box model, 3 is used as a kernel: i.c., instead of considering the feature vector F, of a node v, the model considers 3 x, (indicating with x, the w-th row of x). The general schema for the white-box models M(E,X, v) is as follows:
M(E,X,U) = ŌK (M(E,
(1)
where M is a base model and 0x, is the sigmoid function
0k.O(2) = ((((0=))+1)
(2)
The sigmoid function serves two purposes: first, it allows to obtain y E [0, 1] even if the base model M has range in R; second, it normalizes such scores as explained next.
Consider 5 as the sequence of outputs 9 = M(EE X, v) of the base model M for all the nodes & E V. We use S to adjust the midpoint 9 and steepness K of the sigmoid function (x). Specifically, we set & as the mean of S, and its steepness K as the inverse of the standard deviation of S. This adjustment serves as a normalization mechanism, ensuring that the rescaled output effectively has zero mean and unit variance. By doing so, we standardize the distribution of the base model's outputs across different datasets and models, ensuring that on average, nodes have an equal probability of being classified into one class or the other and that the variance in scores is consistent, facilitating comparability and interpretability across different modelling contexts.
As in real-world sccnarios, node classifiers might employ a subsct of the edges for their classification, according to some internal criteria, and this selection might have an impact on the performance of the explainers, our white boxes consider a randomly chosen subset E' c E, obtained by including each edge with 1 probability.
We next present in detail three different base models M, which differ in their usage of the information coming from neighborhood nodes. In the following (v) indicates the set of predecessor of e which are connected to u through an edge in E': i.c., N(v) = {u E Vl(u, v) E E").
Local model (LM). In this model, the class of a node v E V depends solely on its features Ty. However, we also introduce dependence on the in-degree of controlled by parameter y E R. The model assigns
6

---

Published in Transactions on Machine Learning Research (4/2024)
the probability of belonging to the positive class
((E,X,0)=><(((111 +T
(3)
The PyTorch implementation straightforward. First, the in-degree component is obtained via one-layer message-passingg network where each message is and the aggregation function is the sum. Then, the output of this network is given to the main model that integrates the local parts (i.e. adding Txx, passing the output to the sigmoid).
X2 and
This model is the simplest one we consider, but it represents a variety of real-world situations. Information from local attributes of a node and from its degree might often be sufficient to identify popular users in social media (Hansen et al. 2010) or high-quality web pages (Upstill et al., 2003).
Neighborhood model (NM). In this model, the class of a node e E V depends on the features Ta of the neighboring nodes (w).. Moreover, as the previous model, we introduce a dependence regulated by y on the in-degree. The model is thus defined by
8x. X [Nw)
()) +|
E
(4)
WEN(w)
Intuitively, in this model, the classification of node in the positive class depends on the average of the important features of that node's neighbors.
To implement this model, the main PyTorch module uses two onc-layer message-passingg networks. The first is the degree one, as in the local model previously explained. The second layer's message through (u, E E' is the feature vector Tus while the aggregation function is the average. Then, these two components are aggregated according to Eq. (4).
This model represents a typical situation several application scenarios in which nodes are classified based on their neighbors. For instance, in opinion mining on social media, it is common to classify the opinion of using their neighbor's observable features [Barberá] 2015 leveraging the assumption of homophily. More in general, GNN models composed by single GraphSAGE convolution (Hamilton et al. 2017 be implemented as a combination of a local and neighborhood models, and have been used in node classification tasks for citation networks, social media, and graph classification for
interactions.
Two-hop model (2HM). Our third white- model represents a generalization of the former neighborhood model to random walks of length 2. While this behavior could easily be generalized to longer paths, we restrict our attention to this case since it is by far the most common in practice (Tang et al. 2015). Let P7: (v, u) indicate the probability that a random walk of length starting in V ends up in E V on the graph EV.
Then, we define model as
Σ P (v, (v, AT
(5)
where
P (V, (1) (uEV I
0}.
(6)
In other words, this model classifies nodes positive according to the important features of the nodes that can be reached with two steps on the graph. The implementation of this model involves two layers of message-pasinng net works for the degree component, and two for the feature aggregation, which finally are aggregated according to Eq. (5). Each single layer shares the same structure as the one used by NM. Node classification using random-walk-ban wask- analysis of node attributes has been well explored in the literature (Huang et al. 2219). Moreover, the most scalable methods typically leverage information coming from the two-hop neighborhood (Tang et al. 2015

---

Published in Transactions on Machine Learning Research (4/2024)
3.3
The auditing framework
Now that we have presented our white-box models, we are ready to present our axiomatic benchmark. The first needed step is to prove that the above white boxes indeed have the desired behavior; that is, the features F' C F that are encoded as important inside each model (through 3), coincide with the important features node v E V, as in Definition This proved
F (v) of an explanation M,E, X, v), for the classification
in Theorem 1, whose formal proof is deferred to Appendix A
Theorem 1. Let M be one of our three white-bo box node classifiers (LM, NM, or 2HM) defined over a graph G G (V, E) with no zero-degree nodes and node-feature matriz X R R Let X₁ be the of matrices that might differ from X only in the feature f: X3 {X' E RRIIIIFI Vu V,Vi E P. ..ss & Wug s]. For the classification M(E,X, X,2) with E V, it holds that
By 1 EX.S E X2 M(X' M(X,
or in other terms, a feature f a ground-truth truth important feature F", iff changing has the potential of changing the classificalion of
Thanks to Theorem we know that the three white boxes such that the features F (Definition D that should be recognized important, correspond exactly to the ground-truth important features F' that control. Thus, we can use them to measure how much the explanation 3 produced by given explainer adheres to the ground truth P. We frame such benchmark as a clssty-clastiiication task where the explainer, given a white box, needs to classify the features as important or not.
Testing Explainers: The Feature Importance Fidelity. Since Axiom 1 poses that 83 < 3--- for all F", AUC ROC is the most natural measure, as evaluates how well the scores can separate
I F*, F from F Ft. We take the mean value among all the test nodes. This measure that ranges from 0 to is dubbed feature importance fidelity. A fidelity of 1 means that the explainer scores are perfectly able to separate the important features from the non-important ones. A fidelity of 0.5 corresponds to random performance: the importance scores are random with respect to the true importance of features. A fidelity of 0, instead, indicates anticorrelation: important features consistently receive lower importance scores than non-important ones.
f F*
To summarize, our framework, whose schematic representation is given in Figure 1 consists of the following steps.
1. Given a directed graph = (V, E) and node-feature matrix X E R VIxIF,, generate randomly of important features F² C F.
2. Using F', define a white-box model M, which treats as important features the set F", encoded
as By = (f)
3. Run the explainer E on M for set of test nodes V, obtaining importance score vector 3, that should express (according to E) the importance of cach feature for the model M when classifying a node v.
4. For each test node, compute the ROC AUC between & (the importance assigned by the explainer) and 3 (the ground truth importance of features for the white-box model M). We call the mean of these measures the feature importance fidelity.
4 Experimental settings
In this section, we present the main external components we use in our experiments: explainers and graphs.
Explainers. In order to characterize representative set of explainers, we choose one algorithm for each of the four classes identified in the survey by Yuan et al. (2020): gradient-based, perturbation, surrogate,

---

Published in Transactions on Machine Learning Research (4/2024)
and decomposition. From the Gradient-based class, select Integrated Gradients (Sundararajan et al, 2017), because of its implementation provided in Captum P Albeit not included in the survey (Yuan et al. 2020) due to its recency, it belongs to this class as it uses the gradient values to approximate importance. Among the Perturbation explainers, we use GNNExplainer (Ying et al., 2019), the most widely-known representative of this class. Perturbation explainers, largely employed in image classification, analyze how an input perturbation impacts the model outputs in order to discover important features for the model. From the Surrogate class, we employ GraphLIME (Huang which is able to assign importance to features, and for which an implementation exists As representative of Decomposition explainers, we select Layer-wise Relevance Propagation (Bach et al. 2015
al., 2022), ; the only method in this class
(T.RP). Its importance scores are computed by decomposing the final output of the model layer-by layer according to layer-specific rules, until the input layer is reached.
Furthermore, we also consider a fifth method, Deconvolution (Mahendran & Vedaldi 2016), an explainer that displays a hybrid decompasition gradicnt- based approach. This method was originally introducod to explain image classification and, to the best of our knowledge, it has not been considered yet. in literature as explainer for node classification.
All of these methods share a common interface: they output an importance score for all features when given a model and a specific target node to classify As such, they fit in the framework summarized at the end of the last section.
Datasets. Although our white boxes are not trained, our framework still needs data. For our purposes, we use an array of Erdös-Rényi graph with 100 nodes and a set random 50 binary features for each node. We generate an array of such data sets by varying the fraction of positive features. We opted for this kind of random graph in order to test the explainers on a networked system with topological features drastically randomly select given fraction of features
different from the real data set. In our experiments, important (varying among experiments) for the model M, and we set the value of y to 1.
Finally, we also test real-world dataset (He & McAuley 2016) with 786 anonymized Facebook users (with 319 binary fcatures) and 14024 edges between. Some features (c.g., gender) are considered protected for many tasks, and this data set has been widely used in studies of fairness in graphs (Dong et al. 2022). Our framework can therefore answer the question: if a model M using a protected feature J, for instance using the gender of user to classify whether their ads should gain more visibility, is a given explainer able to detect it?
5 Results
We test the explainers presented above varying the values of important features and, for the synthetic data sets, the number of positive ones. We run each setting 4 times. From each experiment, we obtain a feature importance fidelity score that ranges from (cxplainer scores are anticorrolated with important fcatures) to (all important features have a higher score than unused features, and thus Axiom 1 is respected). Values around 0.5 indicate random performance in capturing features' importance. We present our results on the array of synthetic data sets in Figure? 2 In each plot, each dot represents the average fidelity obtained under each setting: whiskers extend to the maximum and minimum scores obtained. The different percentages of positive and important features are indicated on the x-axis of each plot. Figure 3 shows results for our framework on the Facebook data set; here, adherence to our axiom means that an explainer is able to detect whether the white-box models are employing a certain real, possibly protected, feature (such as the gender of users).
Phttps: /cwp:///.pppta////c////
hhttps://githbb hwt.c///ww.iiacacaaa/aarrrraaaa the original paper (Huang et 2022
double-checked that the implementation is correct with respect to
Teature importance is reflected by score magnitude, we applied an absolute value function to their outputs, ensuring consistency with our framework's assumption larger scores indicating more important features.
'For some explameers

---

Published in Transactions on Machine Learning Research (4/2024)
Local Model
Neighborhood Model
Two-hop Model
1.0
0.8
Fidelty
0.8
0.7
0.6
0.5
posilive fealures
postive
posilive
Fidelty
0.8
0.7
0.8
0.5
0.2
0.3
0.5
0.1
0.2
0.3
0.5
0.1
0.4 Fraction important festures
0.3
Fraciion of important fealures
Fraction of important features
Deconvolution
GNN Explamer
Graphiime
Integrated Gradients
LRP
Feature importance fidelity measured by our framework three white-box models to explain, one per column. Each explainer is represented by different color. In each plot, the obtained fidelity is on the y-axis, and each dot represents the average of 4 experiments, whiskers extend to the minimum and maximum value obtained each setting. In the top row, we vary the fraction of positive features (on the z-axis), and the fraction of important features is set to 50%. In the bottom row, we vary the fraction of important features, and the fraction of positive features set to 50%.
Figure 2:
array of synthetic data sets, with
Local Model
Neighborhood Model
Twa-hop Model
1.0
0.9
0.8
0.7
0.6
0.3
Fraction important
Fraction important
Fraction important
Deconvolution
GNN Explainer
Graphiime
Integrated Gradients
LRP
Figure Feature importance fidelity measured by our framework on the Facebook users dataset with different fractions of randomly assigned important features, and three white-box models to explain, one per plot. In each plot, the fraction of important features is on the z-axis, and the obtained fidelity is the y-axis. Each explainer is represented by different color.
We next present these results explainer by explainer, highlighting under which conditions they respect Axiom 1, investigating the rationale beyond their performance, and discussing the consequences of their practical application.
5.1 GraphLIME
Among the explainers we assess, GraphLIME the only one for which not able to find any setting where it fully respects Axiom 1. Its best results are obtained the local model; in other contexts its performance often approaches a random baseline.
Rationale. These results show that GraphLIME suffers from the inability of its surrogate model to deal with non-local charactoristics in node classification. To better understand the reasons, we have to analyze the way GraphLIME works. First, GraphLIME extracts a set of n nodes in a k-hop subgraph around the explained node i E V (k being a parameter, that we set to 2). Then, it considers the rows of the feature matrix X corresponding to this of nodes, and computes distance matrix K R| between them: intuitively, the element K2 represents measure of similarity between 2; and Dy (we observe that this is computed for each pair of nodes and in the k-hop subgraph, regardless of whether (u, v) E
10

---

Published in Transactions on Machine Learning Research (4/2024)
not). Following the same logic, GraphLIME computes matrix L E RR using the model's outcomes for each node: the element Ti,j represents measure of similarity between and Vj- Finally, GraphLIME uses these two matrices to explain predictions by using a regression method (i.e., its surrogate model); it assumes that if two nodes obtain a similar classification score when their feature f is similar, then f is important; otherwise, it not.
We can see how this approach does not account for non-local correlations. Two nodes might have different features, but if the features of their respective neighbors very similar, a message-passingg model might assign them similar labels. In this case, correlation would be detected by GraphLIME between the similarity of specific feature and the model's outcome; even in the case that the outcome depends only on that feature. Moreover, we observe that even in the local model, GraphLIME obtains worse performances when there are many important features. This phenomenon is due to the regularization term in GraphLIME regression model, which favors sparse solutions.
Fixes and warnings. In the general case, GraphLIME might not be able to explain models that consider non-local features. Therefore, users should carefully evaluate if the regression model proposed by GraphLIME can suit their use
The performance of GraphLIME on our axiomatic benchmark highlights the defining bias of surrogate explainers: their simplified model can be incapable of explaining a model that follows a logic that cannot be modeled by the surrogate one. This hints at a promising direction for future research.
5.2 LRP
LRP respects Axiom 1 in the local model and on the two-hop model; on the neighborhood model, it does so only when the data set presents few positive features. As we will see next, this crucially depends on how we choose to represent negative fcatures.
Rationale. The fundamental reason behind the scores obtained by LRP in our axiomatic benchmark, is that this method is not always able to correctly identify the zero-valued features as important. In the original paper Bach al. (2015), in fact, authors normalize their image data set that relevant fcatures are not encoded as weros. The importance of this step is nowhere acknowledged, for instance, it does not seem to be mentioned in the documentation of the implementation we use happ:s//aaptumaaaj.. ai/aii/ii////iiiaee hap/lrp.html),. If this fact is not clear to users, it might lead to an incorrect interpretation of LRP explanations.
Let us consider a practical example: opinion mining of social media users. Suppose to have a feature ʃ representing whether they have blocked a consorvative politician account: encodes them having blocked them and 0 they have not. If model uses this feature to classify an individual i, when the feature Xi, is 0, the explanation offered by LRP will not assign high importance to that feature.
This issue is more present when there are more zero-valued features, and less impactful when there are more positive features, e.g., on the two-hop model than on the neighborhood one. Finally, since the possibility of a component being exactly zero greatly decreases when all the neighborhood features are averaged, this phenomenon less severe when explaining non-local models. We observe that with LRP (contrarily to Inte- grated Gradient, as we will show below) this issue does not affect the local model but only the neighborhood one. We conjecture that this issue affects the propagation rule of the importance decomposition inside LRP: when the model is composed of only one layer- in the local one such issue does not appear.
Fixes and warnings. Users should be warned that this explainer will not work as expected with zcro- valued features. A simple solution is to choose a different encoding. For example, if we encode binary features with {-1,1} instead of {0, the feature importance fidelity of this explainer becomes almost perfect. We test this claim with an experiment, with the same exact sctting as the one previously used, but encoding binary features with 1- instead of {0, 13. Figure shows our results: as expected, LRP fidelity shifts from around 0.5 to close to 1 against all models, thus confirming our hypothesis.
11

---

Published in Transactions on Machine Learning Research (4/2024)
Local Model
Neighborhood Model
Two-hop Model
0.1
0.8
0.9
Fraction positive features
Fraction of pos features
Frastion of positive feetures
0.9
Fidelly
0.8
0.7
0.1
0.2
0.3
0.4
0.5
0.1
0.2
0.3
0.4
0.5
0.1
0.2
0.3
0.5
Fraction of ipportnn features
Fraction of Important features
Fraction important features
Deconvalution
GNN Explainer
Graphime
Integreted Gradients
LRP
Figure 4: Feature importance fidelity when binary features are encoded with (-1, 1} instead of {0, 1}, for each of the three white-box models (one per column). In each plot, the fidelity is the y-axis. In the top row, the z axis reports different fractions of positive features; in the bottom row, it reports different fractions of important features.
5.3 Integrated Gradients
Integrated Gradients shows in general results similar to LRP. The only difference lies in the local model in the case features are encoded as zero, where Integrated Gradients obtains worse results than LRP.
Rationale. Integrated Gradients suffers from the same issue LRP when important features are encoded zeroes: this phenomenon for Integrated Gradients is very clear even in the local model, where fidelity scales proportionally with the fraction of positive features. This behavior confirms that zero- aero-valued features do not behave as expected with this explainer. We dub explainers with this issue as zero-baseline explainers: in fact, Integrated Gradicnts' authors Sundararajan et al. (2017) explicitly state this requirement, observing that the explanation depends on a basetine inpul that "for image networks, [-] could be the black image, while for text models could the zero embedding vector. [..] A key step in applying integrated gradients is to select a good baseline. We recommend that developers check that the baseline has near-zero score." "
Fixes and warnings. As for LRP, Figure 4 shows that by encoding binary features with {-1,1} instead of {0, 1}, Integrated Gradients' performance shifts to perfect one.
5.4 GNNExplainer
We observe that GNN Explainer obtains fidelity score proportional to the number of positive features when they are encoded as (0, 1} (similarly to Integrated Gradients). With non-local models, instoad, it obtains fair yet not-perfect results (fidelity ~ 0.75), regardless of the type of encoding. This problem is alleviated (fidelity ~ 0.8 - 0.9) when the important features are fewer.
Rationale. First of all, we recognize that also GNN Explainer is unable recognize important features when their value is zero; such, we focus on the experiments where features encoded as {-1,1}. We show such results in Figure Here, the imperfect results obtained by this explainer are due to combination of different factors. First, we find that GNNExplaincr has different semantic than other explainers. To better explain this semantic difference we use the notation introduced in Section 3 We consider a model when M(X, v) > for some threshold [0, 1]. An explainer analyzing this classification instance will output Em(X,, v) 3, assigning an importance score By to each feature f. Now, such scores can obey one of the two distinct semantics that we define next.
M that takes as input the features and classifies node v as class c e {0,1), , in particular
12

---

Published in Transactions on Machine Learning Research (4/2024)
(1) Signed attribution. Importance scores 3 should identify all features that have any impact, either positive or negative, in the classification of the node e class This is equivalent to what is required by Axiom
(2) prrisionn-aligme attribution. Importance scores 3 should identify only features that move the classifica- tion of the node u towards the actual class c. In other words, By should be high when changing Ze, increases M(X, e) the case MMX,u) v) > 0, or decreases M v) in the case M(X, e) < 0.
While other explainers implement the former semantic, GNNExplainer follows the latter: only features that reinforce the given class assignment are deemed important.
Even besides this semantic difference, the results obtained by GNNExplainer are not perfect. Upon additional experiments, we observe that the presence of a mask on the edges siguificantly impacts the performance of GNNExplainer. In other words, the presence of edges where varying a feature does not affect the output (even when there are edges where it does) makes harder for this explainer to recognize a feature as important. Finally, the constraint imposed by default by GNNExplainer to select a low number of important features often leads to lower number of features recognized important. For this reason, its performance degrades when there is high number of important features.
Fixes and warnings. Regarding semantics, while both semantics have their applications, we emphasize the importance of distinguishing the two, so that practitioners can understand correctly the obtained expla- nations.
Regarding sparsity, we find out that GNNExplainer is very sensitive to some hyperparameters, such as KM and Ky that control the size of subgraph and feature explanations respectively. In this work the default values of hyperparameters are used across all experiments, i.e., the regularization hyperparameters for subgraph size is 0.005 and for feature explanation is 1. Optimal values should be informed by prior knowledge about the dataset, and as can be seen from experiments, fixed values of hyperparameters might result in degraded performance if they are not optimal for the number of features in the ground truth.
5.5 Deconvolution
In every configuration, Deconvolution obtains perfect results with a fidelity of 1.0, showing that it is the only explainer fully adhere to Axiom in all tested scttings. Its perfect results, however, do not come for free. Since it is not model-agnostic, its application is fundamentally limited to graph neural networks. Also, it might require modifications at the implementation level in the logic of the classifier in order to be applicable. However, the reliability of the explanations offered by Deconvolution for graph machine lcarning models constitutes a promising and novel result.
6 Discussion
This work proposes axiomatic approach to audit node classification explainers through model-driven framework. In particular, our proposal is to assess whether an explainer is able to detect as important, features that are hard-coded as such, in three white-box models with known internal logic. The simplicity of the three white boxes that we define allows us to prove that the features that exhibit the capability of changing the classification, correctly correspond to the hard-coded set of ground-truth important features. Therefore, we can directly assess the concordance of explainers with the model logic, avoiding the training of black-box on some syntectic data as done by the other proposals in the literature. The main advantage of our approach over this literature lies exactly in bypassing the training phase with its inherent uncertainty about whether the intended logic implanted in the training data has been indeed maintained in the Iearned model. Moreover, one main contribution of our paper is showing that this different approach, assessing tussffulhes--toccheemmdddd is indeed possible and can produce interesting insights.
We applied our framework to five explainers, covering the main classes of approaches in the literature. Our analysis brought to light the existence of some subtle differences among explainers that have an important impact on the quality of the explanations and that are easily overlooked.
13

---

Published in Transactions on Machine Learning Research (4/2024)
Model Name
GNNExplainer
Intended Use Case
The explainer intended to be used gain insights into how GNN models make predictions, and increase transparency and trust in the predictions made by these models.
Model Aware/Agnostic
The algorithm is defined as model agnostic, the available implementation is model (and library) specific.
Zero Awareness
GNNExplainer is zero-baseline explainer, vero-valued features as a baseline: their importance might go undetected. Feature encoding is critical (e.g. use {-1,1} instead of {0, 1} for binary features).
treats
Attribution Semantics
A feature considered important only if it contributes in the direction of the current classification.
Employed Feature Detection
Fully respected only when node classification is driven by local features.
Table 1: Example of the contribution of the axiomatic framework to an explainer model card for GNNEx- plainer.
Limitations. Our contributions should be viewed within the limits of our work. Firstly, our framework follows a binary logic: does not determine which explainers provide a better approximation of the different degrees of importance of each feature used by the model, but only which explainer able to correctly identify important features, i.., features that used to make prediction. It is worth noting that our simple definition of important features does not capture complex scenarios, e.g., prediction that depends on a disjunction of important features. Yet, we show that this elementary criterion is nevertheless sufficient to develop a significant complexity in our analysis, bringing to light some important differences and pitfalls of different explainers. We recognize that our work only operationalizes one notion of truthfulness to the model, from which one metric derived. However, our metric should not be the sole criterion for sclecting among different explanation algorithms. Importantly, human aspects such as usability and adherence to the desired semantics must be considered in making such choices. In the light of these limitations, consider our proposal as first, yet important, step towards developing more nuanced axiomatic frameworks.
Future work. In our future work, we aim to tackle such limitations by extending our framework in several directions. First, as discussed above, we would like to generalize the framework to handle the notion of non-binary importance of features. Second, we aim to devise more complex white boxes, for instance where important features could be dilferent on different nodes, or including non-linear transformations of the features: towards this goal, it is needed to strike a balance between the complexity of GNNs and the simplicity required to achieve formal guarantees as the ones provided by Thcorem 1, whose extension to general GNNs is not easy. Third, we aim to generalize our framework to different tasks: e.g., graph classification or link prediction.
Broader impact. Besides these technical developments, we believe that fundamental aim for the rescarch community's s future goals should be securing the reliability of analyses on machine learning systems. We hope that our framework represents a step in this direction, by allowing any researcher to use the proper tools verify machine learning model is using or not feature possibly protected, such as their gender, religion, or their political ideas to guide choices that could impact individuals. In this regard, our framework also be ingredient towards building model card for explainers, similarly to what has
14

---

Published in Transactions on Machine Learning Research (4/2024)
been done by Mitchell et al. (2019) for trained machine learning models. A model card provides information about a model for the sake of transparency and accountability for its creators and users, mitigating bias, fairness issues, and other ethical and power-balance concerns in their deployment. Similarly, an explainer model card should warn the user of an explainer on the limitations, the semantics, and the intended use of a given explainer-alloe allowing practitioners to assess whether that explainer fit for their particular purpose. In this sense, as exemplified Table 1 our framework allows to check whether an explainer is fit to detect the usage of a given, possibly protected, feature in the context of graph-based machine learning. Building and analyzing explainers, and offering formal guarantees their behavior, is step towards making anyone able, in principle, to open the black boxes of key decision-making systems.
References
Chirag Agarwal, Eshika Saxena, Satyapriya Krishna, Martin Pawelczyk, Nari Johnson, Isha Puri, Marinka Zitnik, and Himabindu Lakkaraju. Openxai: Towards a transparent evaluation of model explanations. arXiv preprinl Xie:22206. 1.0...11111 11104, 2022a.
Chirag Agarwal, Marinka Zitnik, and Himabindu Lakkaraju. Probing gnn explainers: A rigorous theoretical and empirical analysis of gun explanation methods. In International Conference on Artificial Intelligence and Statistics, pp. 8996 PMLR, 2022b.
Kenza Amara, Rex Ying, Zitao Zhang, Zhihao Han, Yinan Shan, Ulrik Brandes, Sebastian Schemm, and Ce Zhang. Graphframex: Towards systematic evaluation of explainability methods for graph neural networks. arXiv preprinl (rf::2206009677 2022.
Elvio Amparore, Alan Perotti, and Paolo Bajardi. To trust or not to trust an explanation: using LEAF evaluate local linear XAI methods. PeerJ Compuler Science, 7:e479, Apr 2021.
Scbastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müllrr, and Wojeiech Samek. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS one, 10()):e0130140, 2015.
Pablo Barberá. Birds of the same feather tweet together: Bayesian ideal point estimation using twitter data. Political analysis, 23(1):76 91, 2015.
Hugh Chen, Joseph D Janizok, Scott Lundberg, and Su-In Leo. True to the model or true to the data? arXiv preprint ar Xic:20006.1220 2020.
Yushum Dong, Jing Ma, Chen Chen, and Jundong Li. Fairness in graph mining: A survey. orXie preprint arXiv:2204.0090 098888, 2022.
Alejandro Barredo Arricta al. Explainable artificial intelligence (xai): Concepts, taxonomics, opportunities and challenges loward responsible ai. Information Fusion, 58:82 115, 2020.
Bryce Goodman and Seth Flaxman. European union regulations on algorithmic docision makiing and a "right to explanation". Al magazine, 38(3):50 57, 2017.
Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in neural information processing systems, 30, 2017.
Derek Hansen, Ben Shneiderman, and Mare A Smith. Analyzing social media networks with Xo:eXL: Insights from connected world. Morgan Kaufmann, 2010.
Ruining He and Julian McAuley. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In proceedings of the 25th international conference on world wide web, pp- 507-517, 2016.
Qiang Huang, Makoto Yamada, Yuan Tian, Dinesh Singh, and Yi Chang. Graphlime: Local interpretable model explanations for graph neural networks. IEEE Transactions on Knowledge and Data Engineering. 2022.
15

---

Published in Transactions on Machine Learning Research (4/2024)
Xiao Huang, Qingquan Song, Yuening Li, and Xia Hu. Graph recurrent networks with attributed random walks. In Proceedinggs of the 25th ACM SIGKDD International Conference on Knowledge Discovery g Data Mining, pp. 732 740, 2019.
Guillaume Jaume, Pushpak Pati, Behzad Bozorgtabar, Antonio Foncubierta, Anna Maria Anniciello, Florinda Feroce, Tilman Rau, Jean-Philippe Thiran, Maria Gabrani, and Orcun Goksel. Quantifying explainers of graph neural networks in computational pathology. In Proceedings of the IEEE/CVI con- ference on computer vision and pattern recognition, pp. 8106 8116, 2021.
Markus Langer, Kevin Baum, Kathrin Hartmann, Stefan Hessel, Timo Speith, and Jonas Wahl. Explain- ability auditing for intelligent systems: A rationale for multi-disciplnnary perspectives. In 2021 IEEE 29th International Requirements Engineering Conference Workshops (REW), pp. 164 168, 09 2021. doi: BAAO100//RE33555522.22222
Antonio Longa, Steve Azzolin, Gabriele Santin, Giulia Cencetti, Pietro Liò, Bruno Lepri, and Andrea Passerini. Explaining the explainers in graph neural networks: comparative study. arv preprint arXiv:2210.15. 2022.
Scott M Lundberg and Su-In Lec. A unified approach to interpreting model predictions. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (ods.), Advances in Neural Information Processing Systems 30, pp. 4765-4774 Curran Associates, Inc., 2017.
Aravindh Mahendran and Andrea Vedaldi. Salient doconvolutional networks. European conference on computer vision, pp. 135. Springer, 2016.
Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timmit Gebru. Model cards for model reporting. In Proceedings of the conference on fairness, accountability, and transparency, pp. 220-229, 2019.
Alan Perotti, Paolo Bajardi, Francesco Bonchi, and André Panisson. Explaining identity-aware graph classi- fiers through the language of motifs. In International Joint Conference Neural Nelworks, IJCNN 2023, Gold Coast, Australia, June 18-23, 2023, pp. 1-8. IEEE, 2023.
Mandeep Rathee, Thorben Funke, Avishek Anand, and Megha Khosla. Bagel: A benchmark for assessing graph neural network explanations. Xiv preprint arXiv:2206. 110..19999 13983, 2022.
Avi Rosenfeld. Better metrics for evaluating explainable artificial intelligence. In Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems, AAMAS '21, pp. 45-50, Richland, SC, 2021. International Foundation for Autonomous Agents and Multiagent Systems. ISBN 450333333
Andrew Selbst and Julia Powles. "meaningful information" and the right to explanation. In Conference on Fairness, Accountabilly and Transporency, pp. 48-48. PMLR, 2018
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In Proceedings of the 34th International Conference on Machine Learning Volume 70, 1CML'17, 17, pp. 3319 3328. JMLR. 2017.
Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. Line: Large-scale information network embedding. In Proceedings of the 24th international conference on world wide web, pp. 1067 1077, 2015.
Trystan Upstill, Nick Craswell, and David Hawking. Predicting fame and fortune: Pagerank or indegree? In Proceedings of the Australasian Document Computing Symposium, ADCS2003, pp. 31-40, 2003.
Jasper van der Waa, Elisabeth Nieuwburg, Anita Cremers, and Mark Neerinex. Evaluating xai: com-
parison of rulc-basod and example-basod explanations. Artificial Intelligence, 291:1034044, 2021. ISSN 0004-3702. doi: http:://doo.org//1.000jj...... 2000..0444... URL https:://ww/ http:://wwww.ccenccrrrcccccc
16

---

Published in Transactions on Machine Learning Research (4/2024)
Petar Veličkovié, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks, 2017.
Max Welling and Thomas N Kipf. Semi-supervised classification with graph convolutional networks. In J. International Conference on Learning Representations (ICLR 2017), 2016.
Zhitao Ying, Dylan Bourgoois, Jiaxuan You, Marinka Zitnik, and Jure Leskovec. Gnnexplainer: Generating explanations for graph neural networks. Advances in neural information processing systems, 32, 2019.
Hao Yuan, Haiyang Yu, Shurui Gui, and Shuiwang Ji. Explainability in graph neural networks: A taxonomic survey. arxiv preprint 0rx:222012. 15, 15445, 2020.
Hao Yuan, Haiyang Yu, Jie Wang, Kang Li, and Shuiwang Ji. On explainability of graph neural networks via subgraph explorations. In Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pp. 12241-12252. PMLR, 2021.
A Proof of Theorem 1
Proof. We proceed by proving each direction of this equivalence (+,, { for each of our three white boxes (LM, NM, 2HM).
LM (=): Assume By 1. We construct X' E N2 as T'os =F, ky Vu E V, for a constant k o O. By definition of LM we have that M(X' = = + Liis Berle + Byre,,) and by construction of X', it follows M(X', v) o(IN Eizy Brsn,t + 3Y + +Byk), which can be rewritten grouping the two terms o([Ng + _Tx, x, + Byk).. Since B, 1, then M(X', v)
T x. k).. Now, since *
and & (as defined in Equation 2) is injective, we have
M(X', u) = (E + 8T x, + kk) #
+ = -M(x
LM (+): Assume EX' E M(X + M(X, Now assume by absurd that 3,= 0. By definition of LM have that: M(X,u) u) of xx,)) <((Ne< + Eits Brrn, ByZuF)) As Bn 0, then u) = (v) + Eits Bitns)). Now consider X'. It holds that
=x (v))
By definition of Xs, we have that E,ys Biste, = Eins Betors and therefore
M(X' ()) (((NN: +
Bioo 8y2%,.)
iff
As 3, = 0, then M(X',)) (vN + Eies BiEr.)) = M(),u), VX E X, which contradicts the initial assumption.
NM (=): Assume 3, = 1. We construct X' E X₂ as T'e = Io,f + h. Vo E V, for a constant k * O. By definition of NM we have that M(X,v) ())N(())+ + (oंo EmENoo Eier Bru.)). For simplicity we rewrite it M(X, v) σ(Q, Q2 Qs) where:
Q [N(w)]:
Q2 [N(x)
Σ
Σ
Bruss
uEN(v) it
Q3
Σ B
(M)(v)
17

---

Published in Transactions on Machine Learning Research (4/2024)
Similarly we denote M(X', 2) + We have that Q1 Qi and Q2 Q22 due to definition of X" E Ky. For what concerns Q'3 instead we have that
Q's
Σ
Byd'us
[N(']
mEN(o)
Σ BrZu. +
Σ
Byk
[N(0]
[N(v)]
NEN(v)
NEN(u)
Qs +
Σ Byk.
[N (EN(v)
Since 3, 1, k + 0 and (v) is not empty (no singleton nodes), it follows that EUEN( 3yk + O. Therefore, since & is injective, (QQ + Q2 + Qa) * ((Q + Q2 + Q3 + EmeN(o) [N(O] and M(X, v) # M()'
NM (=): Assume EX' E Xy : :M v) + M(X, v). Now assume by absurd that By = Using the same rewriting as above we denote M v) σQ - Q2 + Qs) and M(X' = o(Qi + +02 Q2), with Q1 Qi and Q2 = Q/2 as the case above. Instead, contrarily to the case above, since we are assuming that 3, = 0, we also have that VX' Xj. Qs = Q/ and thus M(X, v) M(X u), VX E Xy, which contradiction with the assumption.
2HM (=): Assume 3 1. We construct X' X2 as Lees To,f + k, Ve E V, for constant k + O. By definition of 2HM we have that M(X, = N (v) + En P2 (v, u) B x.). For simplicity we rewrite
it as M)X,v) = sQ1 + Q2 + Q2) where:
Q = (x)l:
Q2 ifS
PU u) Brouusi
Q3 Σ Pr (V, u) 33
Similarly we denote M(X, b) = (QQ 0 + (s). For all X' E X,, we have that Q1 = Qi = =|N and Q2 = Q2 Instead, Q/4 is
O's
Σ (U, Byy'us
Σ (v, B (u)
= Qs + (P, 3i
As the graph has no zero-degree nodes the random walk is well-defined and 3u (v, > 0. Moreover, we assumed 3y and A: + 0, and thus En PV, (v, Byk + Therefore, since & injective, Φ(Q1 + Q2 Qs) + o(Q1 Q2 Qs Σ,, PV u) 8yk) and M(E, X, M(E,X',o)
2HM (+): Follows exactly the proof of the case NM (+).
This concludes the proof.
18


Reviews:
[{"review_id": "IRPiWY0mlt", "reviewer": "Reviewer_GwDM", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n\nExplaining black-box models is a challenging problem with many different approaches, and GNNs seem to be particularly difficult. Certain metrics are widely used for tabular and image explanations, but it doesn't seem like there are reliable benchmarks for GNN explainers. It is therefore a worthwhile direction to explore, and the authors design a new approached anchored in a property that seems generally reasonable (although with some shortcomings described below). Their experiments consider a range of different methods, and are generally well executed and nicely presented.\n\n## Weaknesses\n\nSome concerns about the premise of this benchmark:\n\n- Unlike explainers for tabular and image datasets, graph explainers have substantial variety in the format of their output. For example, some provide attribution scores for all input features, and others select important subsets of nodes or edges. Meanwhile, the benchmark tests methods via their ability to properly rank the important/unimportant features; many of the methods tested here are not designed to output single scores for each feature (i.e., scores that are not node-specific). It therefore seems that the benchmark provides a limiting and poorly matched way to evaluate and compare explanation approaches.\n- The stated motivation for this \"true to the model\" benchmark (see the \"Proposed framework\" paragraph) is that previous evaluations are somehow not true to the model: they inject a dependency into the data, fit a black-box GNN, assume that the pattern is used by the model, and then test whether the explanation identifies it. It seems to me that the black-box model would in most cases use the injected pattern, do the authors believe that's not the case in the four cited works? Can they explain why or provide any evidence, because it seems like not the strongest motivation to design a new benchmark. Can the authors compare their findings to those works and explain whether they derive any new conclusions that are specifically enabled by their benchmarking approach, and not simply the inclusion of a different set of explainers?\n- The definition of \"important\" features and axiom 1 seem a bit subjective, it's not obvious that these are desirable properties. For example, consider a GNN that classifies nodes based on the individual's `income` and `race`, so that the prediction is 0 if `income < X` or `race = Y`. If both conditions hold, neither feature would be deemed \"important,\" and that seems like a bad thing. Are we sure this is a good definition of importance to base a benchmark on? Separately, it's worth contextualizing this view of \"importance\" via some existing notions in the literature, particularly the related axioms for SHAP and IntGrad (Lundberg & Lee, 2017 and Sundararajan et al, 2017).\n- A smaller complaint about Axiom 1 is that the inequality between importance scores depends on the sign, but I'm not sure it should. For methods that provide signed importance scores (many feature attribution methods), an \"important\" feature should have a large negative score. So the axiom should if anything have $|\\beta_f| < |\\beta_{f^*}|$.\n- Another complaint about Axiom 1: what is the relevance of this axiom for real models, where it seems very unlikely that any input feature is unable to have any effect on the model output?\n\nAbout the white-box models and theoretical results:\n- In the description of the white-box models, I couldn't follow the bit about considering the sequence $S$ of outputs of the base model for all nodes. I also couldn't follow why there was a softmax involved (mentioned in the Local Model section). \n- Theorem 1 is a very intuitive result, I'm not sure it deserves to be called a \"theorem.\" Maybe a claim, or a proposition instead. For all three models, the first direction is obvious. The second direction is also obvious from the contrapositive: if $\\beta_f = 0$, the feature cannot change the output. These results shouldn't take 2 pages.\n\nAbout the conclusions:\n- Some of the insights from the experiments are known design choices of each method that have been thoroughly discussed in the literature. For example, the failure of some methods to identify important features when they're equal to the baseline value (e.g., zero); that some methods distinguish between positive and negative effects on the prediction; that some methods are sparse by design, whereas others identify all influential features. After reading the results, many of these claims seem like they would have been knowable without relying on the benchmark.\n- Overall, the benchmark provides little information about each explanation's correction outside its assignment of \"important\" features. For example, it provides no way to evaluate the relative ranking of important features.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work considers the current literature on explainers for graph neural networks (GNNs): that is, algorithms that explain why a GNN made a specific prediction for a given input, usually by scoring the relevance of all input features, or in some cases identifying a subset of relevant nodes/edges/features. Many such approaches have been developed in recent years, and this work aims to provide a consistent means of comparison via a new benchmark. The idea of the benchmark is to design a couple simple GNN models, which have no learnable parameters and are completely specified in 1-2 layers, and test whether each explanation algorithm correctly identifies the set of \"important\" features. These important features are defined by the authors as those that can take on values that change the model's prediction when all other features are fixed. Under this evaluation, an explanation algorithm is deemed accurate if the important features have higher scores than the unimportant features.\n\nThe authors test five different graph explainers under their proposed approach. The results reveal certain insights about each method. The authors use these insights to provide some recommendations about how and when each method should be used.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Several questions and concerns are mentioned above. It would help to clarify these points in the paper.", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "vZ5c5wEI5j", "reviewer": "Reviewer_STRH", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n-\tThis paper introduces a framework for evaluating and auditing explainers used in graph-based models.\n\n-\tThe framework is based on three white-box classifiers, implemented based on message-passing GNN and covering models relying on local features, one-hop and two-hop neighbors for node classification.\n\nWeakness:\n\n-\tThe framework is focused on auditing in terms of the feature importance. However, graph structure is also important for graph learning tasks, and it seems that the framework proposed in this paper cannot analyze the importance of graph structure.\n\n-\tAlthough three white-box classifiers can be implemented with message passing GNN, these white-box classifiers may differ from the classifiers learned from data. Therefore, it is necessary to discuss the differences in applying explainers to white-box models and black-box models.\n\n-\tThe reason for using these three white box models is not very clear. Is there any way to verify their performance on real datasets?\n\n-\tIt is better to analyze different explainers on real datasets, such as showing the actual meanings corresponding to some important features.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces a framework for evaluating and auditing explainers used in graph-based models. They establish a benchmark using three white-box classifiers implemented as message-passing GNNs and prove a theorem ensuring explainer accuracy. The authors audit five explainers, identifying limitations and providing recommendations. They also adapt salient deconvolutional networks for graph node classification, performing perfectly on the proposed benchmark.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "See weakness", "broader_impact_concerns": "N/A.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "bnO8ELahlI", "reviewer": "Reviewer_MeAS", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strong aspects\n\n#### Strength: Impact of Results\nThe authors have highlighted that typical explanation methods such as Integrated Gradient, GNNExplainer, GraphLIME, and LRP tend to fail to identify important features in simple GNNSs. This finding suggests the need for careful consideration of the reliability of these methods when applied in practical scenarios.\nFurthermore, the authors revealed that in methods based on zero baselines like Integrated Gradient and LRP, accuracy improvement is possible by transforming features to become non-zero.\nInsights gained from these discoveries and improvements can be considered an important step forward in Explainable Artificial Intelligence (XAI) research.\n\n### Weak aspects\n\n#### Weakness 1: Global vs. Local Feature Importance\nThe proposed simple GNNs operate under the assumption that important features are consistent across all nodes. This implies that the evaluation method assumes GNNs make decisions based on globally important features consistently across all decisions. However, a key advantage of Neural Network-based models is their ability to adaptively change decisions based on different features depending on the data (nodes). Therefore, evaluating GNNs that make decisions based on locally important features, where important features vary for each data (node), would better suit real-world problem settings. Thus, the proposed evaluation model oversimplifies the situation and seems to overlook the scenario where the important features assumed by many explanation methods vary locally.\n\n#### Weakness 2: Existence of Trivial Solutions\nThe proposed GNNs (3), (4), (5) are sparse linear models. Consequently, it's easy to identify important and non-important features by perturbing the input vector x. For instance, a simple gradient, i.e., differentiating the model's output with respect to the input vector x, makes non-important features to have zero derivatives, easily achieving Axiom 1. Similarly, with popular explanation methods like Permutation Importance, permutations for non-important features don't alter the model's output, hence their contribution can be deemed zero. In the proposed evaluation method, these classical and extremely simple explanation methods can trivially achieve the highest accuracy. Does this fact suggest the effectiveness of these classical methods, or does it imply a missing crucial aspect in this evaluation method?\n\n#### Weakness 3: Theorem 1 is trivial.\nTheorem 1 seems trivial based on the definition of the models (3), (4), (5). Devoting many pages to demonstrating this trivial result doesn't seem meaningful. It would be more beneficial to allocate pages to expanding experiments, analyzing their results, and providing insights rather than proving obvious theorems.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "In this research, the authors proposed a method to evaluate the faithfulness of feature-based explanations provided for Graph Neural Networks (GNNs). This method aims to assess whether these explanations effectively capture the truly important features in the model.\nSpecifically, they constructed simple GNNs that encapsulate the truly important features. Then, they applied various explanation methods to these simple GNNs to determine how accurately they could identify the truly important features.\nThe authors conducted evaluations on representative explanation methods such as Integrated Gradient, GNNExplainer, GraphLIME, LRP, and Deconvolution, demonstrating experimentally that many of these methods could not accurately identify the true important features with sufficient accuracy.\nFurthermore, they provided qualitative and quantitative insights into the reasons behind the low accuracy of these methods.\nThe authors also reported that Deconvolution exhibited the most promising results in this evaluation.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Most of the followings are the questions to the authors rather than changes.\n\n#### Request 1:\nThe proposed GNNs all operate under the assumption of globally important features. Does this assumption pose any risk of bias in evaluating explanation methods? Would it be more appropriate to design GNNs based on the assumption of locally important features and evaluate on them?\n\n#### Request 2:\nIn the proposed evaluation method, simple classical techniques like Gradient or Permutation Importance are trivially effective. Does this fact indicate the effectiveness of these classical methods, or does it suggest a missing crucial aspect in this evaluation method?\n\n#### Request 3:\nIt might be more beneficial to remove Theorem 1 and allocate pages to expanding experiments, analyzing their results, and providing insights rather than proving obvious theorems.", "broader_impact_concerns": "There is no ethical concern.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: During the discussion, two of the reviewers were leaning towards accepting the paper and one towards rejecting it. Overall, the reviewers tend to agree that the impact of the work may be limited since the proposed method is of limited in capacity and highly simplistic. However, as per TMLR guidelines, estimated potential impact should not be a criteria for the acceptance decision. Moreover, the reviewers also tend to agree that the problem tackled by this paper is highly relevant and should be studied more thoroughly in the literature. Hopefully, this paper will be first step towards a more rigorous evaluation of graph explainers. This is why I decide to recommend accepting the paper.
Decision: Accept (poster)