Graphon-Explainer: Generating Model-Level Explanations for Graph Neural Networks using Graphons
Authors: Sayan Saha
Sanghamitra Bandyopadhyay
Venue: Accepted by TMLR ()

Abstract: Graph Neural Networks (GNNs) form the backbone of several state-of-the-art methods for performing machine learning tasks on graphs. As GNNs find application across diverse real-world scenarios, ensuring their interpretability and reliability becomes imperative. In this paper, we propose Graphon-Explainer, a model-level explanation method to elucidate the high-level decision-making process of a GNN. Graphon-Explainer learns a graphon—a symmetric, continuous function viewed as a weighted adjacency matrix of an infinitely large graph—to approximate the distribution of a target class as learned by the GNN. The learned graphon then acts as a generative model, yielding distinct graph motifs deemed significant by the GNN for the target class. Unlike existing model-level explanation methods for GNNs, which are limited to explaining a GNN for individual target classes, Graphon-Explainer can also generate synthetic graphs close to the decision boundary between two target classes by interpolating graphons of both classes, aiding in characterizing the GNN model’s decision boundary. Furthermore, Graphon-Explainer is model-agnostic, does not rely on additional black-box models, and does not require manually specified handcrafted constraints for explanation generation. The effectiveness of our method is validated through thorough theoretical analysis and extensive experimentation on both synthetic and real-world datasets on the task of graph classification. Results demonstrate its capability to effectively learn and generate diverse graph patterns identified by a trained GNN, thus enhancing its interpretability for end-users.

Published in Transactions on Machine Learning Research (10/2024)
Graphon-Explainerr Generating Model-Level Explanations for Graph Neural Networks using Graphons
Sayan Saha
1yyonsaha11196688 com
Machine Intelligence Unit
Indian Statistical Institute, Kolkata
Sanghamitra Bandyopadhyay
sunghamiiiiscaal.cm ac.
Machine Intelligencee Unit
Indian Statistical Institute, Kolkata
Reviewed on OpenReview: openreveee... net/ fortm id= gHUutuvo.Q
Abstract
Graph Neural Networks (GNNs) form the backbone of several state-of-the-arl methods for performing machine learning tasks graphs. As GNNs find application across diverse real-world scenarios, ensuring their interpretability and reliability becomes imperative. In this paper, we propose Graphon-Expoo Explainer, model-level explanation method to elucidate the high-level decision-making process of a GNN. Graphon-Fxplainer learns a graphon a symmetric, continuous function viewed as a weighted adjacency matrix of an infinitely large graph-to approximate the distribution of target class as learned by the GNN. The learned graphon then acts a generative model, yielding distinct graph motifs deemed significant by the GNN for the target class. Unlike existing model-level explanation methods for GNNs, which are limited to explaining GNN for individual target classes, Traphon-Explainee can also generate synthetic graphs close to the decision boundary between two target classes by interpolating graphons of both classes, aiding in characterizing the GNN model's decision boundary. Furthermore, Graphon- -xppann-xxpainerr is model-agnostic, does not rely on additional black-box models, and does not require manually specified handcrafted const raints for expla- nation generation. The effectiveness of our method is validated through thorough theoretical analysis and extensive experimentation on both synthetic and real-world datasets the task of graph classification. Results demonstrate its capability to effectively learn and generate diverse graph patterns identified by trained GNN, thus enhancing its interpretability for end-users.
1 Introduction
Graph Neural Networks [Zhang et al. 2020 Wu et al.) 2020) have attained state-of-the-art perfor- mance on various graph tasks such as node classification, graph classification and link prediction. They are increasingly being deployed the real world in diverse applications such as drug discovery and molec- ular structure prediction (Merchant (Meerction et al.) 2023), weather forecasting(Lam al.) 2023) and recommender
systems WWu et al. 2022). The integration of GNNs into real-world applications comes with the dire need to make them interpretable and trustworthy for stakeholders who regularly interact with rely such models. Although substantial rescarch has boen conducted on explainability in domains like vision and text analysis (Kim et. al. 20188 Selvaraju et al.; 2017 Ribeiro el. al. 2016 Chen et al.) 2018), these met are challenging to directly apply to graphs. This difficulty stems from the discrete nature of graph data, which poses challenges for optimization using gradient-based tochniques. Morcover, unlike images, graph data exhibits irregularity as each graph may contain varying number of nodes, and its validity often hinges upon adhering to domain specific structural constraints.

---

Published in Transactions on Machine Learning Research (10/2024)
Existing post hoc explainability methods for GNNs can be categorized into two classes Yuan al. 2022 Instance-Level methods and Model-level methods. Instance-level methods offer explanations for the model s predictions on individual inputs, while model-level methods investigate the general behavior of the model without reference to specific inputs. When the aim is to evaluate the reliability and trustworthiness of a model, one needs to examine instance-level explanations across several instances to draw a rigorous conclu- sion about the behavior of the model. Therefore, model-level explanations are more apt such contexts. Furthermore, it has been demonstrated in Faber et al. (2021) | that instance-level explanations fail to produce faithful explanations when the GNN model is trained on binary classification problem and suffers from the bias-attribution issuc. Model-level explanations can not only produce faithful explanations in this scenario but also diagnose the bias attribution issue. Although model-level explanation methods offer substantial benefits over instance-level approaches, they have received comparatively less altention in the literature. Current methods such as XGNN (Yuan et al., 2020). D4Explainer (Chen et al., 2024). and GnNtntrrrerer (Wang& & Shen 2022) focus on providing explanations for specific target classes but fall short in clucidating the decision boundary of the underlying model. On the other hand, GNNBoundary (Wang & Shen 2024) can
generate graphs near the decision boundary between two target classes but lacks class-specific explanations.
In this paper, we introduce Graphon-Expia Explainer, a holistic model-level explanation method that can not only generate explanations for individual target classes but also produce synthetic motifs that help characterize the decision boundaries between two classes. To the best of our knowledge, this is the first method capable of offering both class-specific explanations and insights into the decision boundary Sraphon-Explaing approximates the distribution of graphs classified to a particular target class by the GNN model by estimating a graphon function for that target class. The learned graphon serves a generator, producing graph motifs that capture the distinctive features learned by the GNN about the target class. Unlike graphs, graphons possess regular structure and aligmment, facilitating the interpolation of graphons from two target classes to produce a graphon that can generate synthetic graphs lying close to the decision boundary between the two classes. Graphon-Explainer has several advantages over current state-of-the-ar model-level methods. Unlike, XGNN((uan et al. 2020) and D4Explainer (Chen et al. 2024 it does not need another black box deep learning model to generate explanations. Similarly, unlike GNNInterpreter (Wang & Shen 2022), which requires access training data embeddings to generate explanations, only roquires access to the emboddings of the query dataset that the user possessos, while secking to explain the model's behavior.
the target classes.
2 Preliminaries
We introduce some graphon related mathematical concepts in this section that we use later for the theoretical analysis.
Graphon: A graphon S [0, [0, 1] is symmetric, continuous, and measurable function which can be viewed as a weighted adjacency matrix of a graph with infinitely many nodes, Given two points x,, æy E [0, 1], 5(i,j) can be interpreted as representing an weighted edge between nodes and j. Graphons arise as limits of sequences of dense graphs sharing a common set of topological attributes in the graph theory literature and can serve as generator of graphs having similar topological properties. For instance, sampling n points uniformly from the unit interval and setting S(z, v) = p Vxx, y E [0, 1], where p e [0, 1] is constant generates Erdös-Réns Rényi random graph G(n, p).
Cut Norm: The cut norm of a graphon S can be defined as:
Vsay S(x,8)
| = sup
x.YC(111) XXY
For two graphons,St and 52, measure of distance between them can be defined using the cut norm - -l Lower values of the cut norm indicate that the graphons S and 5a have a high degree of strue- tural similarity. Graphon estimation methods frequently use step functions as it has been proven (Lvássz 2012) that graphons can always be approximated with arbitrary accuracy the cut norm using step func- tions.
Graph Homomorphism: Given two graphs F and G, let their node sets be denoted V(F) and
2

---

Published in Transactions on Machine Learning Research (10/2024)
taaplig Fen 8
g0
4 A 00
Traiel Clasdior
Ap
Expleiatnn for Clas Libeedi
Figure 1: demonstration the Caaphon-Explaine workflow on a binary classification problem. First, a trained classifier categorizes dataset of graphs into two classes ei and c2. Then, a graphon is estimated for each class ( and 8 using the graphs assigned that class, These estimated graphons are sampled to generate explanations for both classes by optimizing their class scores, with the classifier evaluating the explanations. Additionally, the estimated graphons interpolated to create new graphon Sh which is sampled to produce graphs that lie near the decision boundary of both the classes. These graphs exhibit a mixture of motifs from both classes.
V(G) respectively and their edge sets be denoted as E(F) and E(G) respectively. A graph homomor- phism ub is a map from ((F) + V(G) such that if (u E E(F) then (((u),((w) E E(G). Let, hom(F, G) denote the total number graph homomorphisms from to G. Then, the homomorphism density ((F, G) can be defined as t(F, G) [VGG]jIV(S)-- Homomorphism density measures the fraction of struckure-preserving maps from F to G. The notion of homomorphism density can be naturally extended to graphons. Given graph F, its homomorphism density with respect graphon S can be defined ((F, fo. HIJER(P) S(1;, (FE dz;.
hom((F,G)
3 Related Work
Graph Neural Networks: GNNs work by learning a node's representation by a message passing mech- anism. Each node's representation is learned by aggregating features of neighboring nodes and combining them with its own features. Different architoctures (Veličković et al. 2018 Kipf & Welling 2016 Gilmer
et al. 2017) differ in their aggregation functions and message passing functions but the learning strategy remains similar across architectures.
Instance-Level Explanations: Instance level methods provide an explanation corresponding to each in- put. Typically, they aim to ideutify salient subgraphs, nodes or edges that deemed important by the GNN for the graph to be classified into a particular class. According to the survey (Kakkad al. 2023 they (nopo
can be categorized into: decomposition based methods (Pope al. 2019 Feng et al. 2023), gradient based
methods
methods(Baldassaar (Baldassarre & Azizpour 2019 Huang et al.] 2022). surrogate BethodssZZanme et al.] 2021 Vu &
Thai 2020) and perturbation based methhods(Schlichtkk
et al. 2020 Yuan et al. 2021 Lucic et al. 2022:
Lin et al. 2022 Sharing a similar goal as our method, MotilExplaaner(Yin & Gao 2023) employs domain
specific motif extraction rules to identify a candidate set of explanation motifs and uses an attention based mechanism to identify motifs from the candidate set in an instance graph that is important for the catego- rization of the graph particular class by the GNN classifier. However, our approach differs significantly in two key ways. First, while MotifExppaannr relies on domain-specific rules for motif extraction, our method does not. Second, MotifExplainer provides instance-level explanations, which are not generalizable across different graphs. In contrast, our model-level explanation method generates motifs that reflect the features the GNN has learned for an entire target class, rather than focusing on individual instance graphs.
Model-Level Explanations: The goal of model-level explainability is to undersland a model's general behavior by generating graph patterns that trigger specific predictions. Different model-level methods differ in their graph generation mothods. XGNN et al., (2000) trains a reinforccment learning model using the policy gradient method to maximize a reward function designed using handcrafted constraints to gen- erate graphs that serve explanations of target class. D4Explainer al. 2024) generates graphs
3

---

Published in Transactions on Machine Learning Research (10/2024)
by training diffusion model which requires significant computational resources and time to generate explanation. GNNnneeppreter(War Wang & Shen 2022) was the first model-level method that did not rely on another black-box neural network architecture to generate explanations. However, during training, it re- quires the embeddings of the GNN's training data as input for an objective function it aims to maximize, necessitating access both the training data and the hidden layers of the GNN. It worthwhile to mention here that crafted constraints or employ another black-box model to generate explanations. Another recent method GNNBoundary (WangaarryWaaggggggggg & Shen 2024) is model-level explanation method that illustrates the decision mak- ing process of a GNN by generating graphs that lie close to the decision boundary between two classes, however, it cannot produce explanations for individual target classes. Other notable works in model-level explainability include GDM (Nian et. al. (2024). which generates model-level explanations by minimizing the distance between its explanations and training graph embeddings during the GNN's training process, and MAGE (Yu & Gao 2024), which provides motif-based modcl-level explanations for classification tasks molecular datasets. However, is important to note that these methods also share the limitation of producing explanations only for individual target classes, without illustrating the decision boundary.
does not require access to the GNN's internal components, does not use hand-
Other Methods of GNN Explanations: There are also GNN explanation methods that do not strictly belong to either the instance-level or model-level categories, and some approaches combine both. A notable example of the former is GraphChef (Müller et al. 2024), which develops an intrinsically explainable GNN capable of explaining its decision- decision-making process on any dataset by generating a decision tree. An example of the latter category (Azzolin et al.] 2022) uses instance-level explanations as input and combines them using Boolean operations construct model-level explanations.
Graphons: Graphons have been extensively studied limits of graph sequences in the literature Lovász
& Szegedy 2006 Lovász 2012 They have also been explored in the contexts of Graph Neural Networks (GNNs) (Han et al. 2022; Ruiz et al. 2020) and network theory Vizuutee et al. 2021). Approaches to
approximating graphons can be categorized into two main lines of work. The first utilizes stochastic block models, including methods like stochastic block model approximation (SBA) (Airoldi ct al. 2013) and sorting and smoothing (SAS)
Airoldi 2014). The second line employs low-rank matrix approximations, such
as matrix completion (Keshavvnn et al. 2010 and universal singular value thresholding (USVT) CChtterreee
2015).
4 Method
Algorithm Generating Explanations using Grappon-Explaines
Hyperparameters:
Neamples: Number of samples to be generated by the estimated generative model.
K: Number of nodes in the generated explanation
X: Weight of a graphon during linear interpolation of graphons of two target classes
Input: Trained GNN graph classifier model f()), Sets of graphs (De,,... Dea} labeled by f as belonging to classes (c.,... (k) Graphon Estimator g
Procedurc:
Generating explanations for class c:
Estimate graphon 5 for class e, on K partitions and USVT graphon extimator using Algorithm
Sample Nsampies from the cstimated graphon using as described Algorithm
Select the sample having the highest target class score as the explanation.
Generating graphs close to the decision boundary of two classes Ci and e2:
Combine estimated graphons 5 and S linearly with weights À to get the boundary graphon a)
5h = AS +
Sample Nsampies from 5 using equation
Select the sample that minimizes equation 3
4

---

Published in Transactions on Machine Learning Research (10/2024)
4.1 Problem Setup:
The goal of model-level explanation is to pinpoint discriminative graph patterns that trigger specific class prediction from a trained graph classification model f(). Formally, the goal of producing model level explanation for a target class can be stated as identifying G for which,
GO argmax P(f(G) =))
(2)
G
Assuming, we have an unknown joint distribution of graphs and class labels P)G,C), the optimization takes place over the samples drawn from the class conditional distribution P(G|C c) which we assume in line with prior works (Wang& & Shen 2022 2024) can be written as P(XIC c) = c), the product of the distribution of node feature matrices and adjacency matrices. In this work, the distribution of the adjacency matrices is approximated by the estimated graphon and the distribution of the node feature matrices is approximated by a probabilistic weighting mechanism on the pool of node features of graphs that are classfied to particular target class by f(-). It should be noted that there might exist multiple distinct motifs G that maximize the equation P Such motifs shed light on the graph patterns learnt by the model f() for the particular target class.
Beyond the generation of explanations for individual target classes, Sraphon-Explaingg can also generate graphs that lic close to the decision boundary of two target classes. Formally stated, for a pair of classes c1. e2 the objective is to find motifs G2, such that
GB argmin PP(f, (G)) (G))) subject to (G), ((ee(GG) Vi [1.2], Vj 1,2
(3)
G
P(fe, (G)) denotes the probability assigned by f(.) to G' for class c. Minimizing this objective ensures that Gg lies close to the decision boundary of both classes.
4.2 Implementation Setting
Graphon-Explainge is a model-agnostic method that can be used to explain any GNN classifier model. It does not need access to hidden layers of the GNN classifier or knowledge of the process by which the GNN classifier outputs a label for an input graph. Graphon-Fxplainee only assumes the ability to query the trained GNN classifier for predictions. Figure] Iillustrates the workflow of Graphon-Explaing which we will explain in detail below.
We begin with a dataset of graphs, represented as D, wherein each graph is associated with a distinct class label, denoted by with e taking on values within the set (1,..., C). Additionally, a trained GNN given graph within this dataset.
graph classifier, represented as f()), can be queried to predict the class Consequently, when presented with a collection of graphs De S D, identified by f()) as belonging a specific target class c, Craphon-Explainee proceeds to generate explanations for the class Graphon- Explainer generates explanations in two steps. First, it estimates graphon for the target class using the graphs in D.- Second, it samples from the estimated graphon while optimizing the objective in Equation 2 to generate graphs that exhibit the discriminative motifs characteristic of class as identified by f(). Further, when provided with datasots De, and De, of graphs that belong to two target classes cl and d2 according to f((), Graphon-Explainee can generale examples that lie close to the decision boundary of the two classes. Generating examples that lie on the decision boundary involves estimating the graphon for each class from the corresponding datasets as a first step. This step is followed by taking a convex combination of the graphons of both these classes to estimate the graphon of graphs that lie on the boundary. Finally, sampling from this graphon while optimizing the objective in Equation B yields examples that lie on the decision boundary. We contain a combination of discriminative motifs present in classes Ci and cz.
4.3 Graphon Estimation
The first step in generating explanations using target class using graphs classified into target class by the trained classifier f. Estimating graphon for
involves approxinating graphons of each
5

---

Published in Transactions on Machine Learning Research (10/2024)
class of real-world graphs is inherently difficult as it is an unknown function lacking a closed-form expression.
Therefore, in line with previous research (Xu et al. 2021 Han et al. 2022 Airoldi et al. 2013) we employ a step function method for graphon estimation. Let, P = (...... .k) be a partition of [0. 1] into K disjoint intervals. A step function Sp [0, 1]2 - [0,1] can be defined as Sp(z, 3) Ex Lk SkK (2, Px where each SKA' E 1] and II is an indicator function which is equal to 1 when (z, E Ps x PE: otherwise it is equal to 0. Hence, the step function Sp over K partitions of the unit interval can also be seen as a matrix Sp = [sRRr] E [0, 1],1] For our purposes, we set K to be equal to the median number of nodes in the dataset D across all target classes. We use the USVT method (Chatterjee 2015) to estimate the step function. This method aligns the nodes of the graph in cach dataset based on certain node properties and then proceeds to estimate the step function from the aligned adjacency matrices. The weak regularily lemma of graphons (Lovász & Szegedy 2006) guarantees that any graphon can be accurately approximated | describes the method we use to approximate graphons.
well in the cut norm by step functions. Algorithm The algorithm and the lemma are detailed in Appendix
4.4 Generation of Explanation for Target Class
Once, the graphon for the target class e is approximated by the step function S using graphs De, one can sample from this graphon to generate explanations for the target class c. Given 5 we can generate a N node graph through the following two-step sampling procedure:
Uh ~ Uniform([[, 1)) Onn' (ern(S
Vn E (1,"., N) (w)) un)) Vn,n' E (1,".,,
(4)
The first step samples N nodes from a uniform distribution on the unit interval [0, 1]. The second step generales adjacency matrix A= [amwr] E {0, 1) NxN by sampling each entry from a Bernoulli distribution with parameters defined using the estimated graphon S Graphs sampled from S using this sampling process would contain discriminative motifs that are significant for the classificr f(-) to label the generated graphs as belonging to the target class c. The generation of node features of synthetic graphs is done in two phases. During the graphon estimation phase, we align the original node features with the adjacency matrices, resulting in a set of aligned node features for each graphon. We pool these aligned features and assign probabilistic weights to them depending the their frequency of occurence. Node features are then sampled according to their probabilistic weight from the pool. The graph generation process is detailed in Algorithm 4
4.5 Generation of Graphs near the Decision Boundary
Generation of synthetic examples lying on the decision boundary of two target classes requires estimation of the graphon functions of the corresponding target classes as a first step. Given two target classes ei and 02 and their estimated graphons S and S we take a convex combination of these graphons: : Sb AS + (1 - 2) Interpolating graphons of two target classes yields a graphon which is mixture of the graphons of both classes. Graphs can be sampled from Sh through the same sampling procedure as shown in Equation ! The node features of the boundary motifs are generated by first sampling the node features from each graphon, as outlined in Algorithm 4 Next, the sampled features in each node feature matrix are reweighted according to the mixup parameter of each graphon. Finally, each node of the generated graph is resampled using the corresponding weights as probabilities. Graphs sampled from the interpolated graphon partially contain discriminative features of both the classes which places them close to the decision boundary of the two target classes. In multi-class classification problem, evaluation is performed to determine if two classes share a decision boundary. This process begins by randomly sampling N pairs of graphs from the two target classes. For each pair, the corresponding graph embeddings are extracted from the penultimate layer of the classifier, before the final linear layer that produces class-specific logits. Next, these embeddings are interpolated, and the classifier's output on the interpolated embeddings is analyzed to see if crosses into the decision region of another class. A score is then calculated for each pair basod on this analysis. If the cumulative score across the K pairs exceeds a predefined threshold, the two target classes are considered adjacent, sharing decision boundary. The pseudocode of this procedure is detailed
6

---

Published in Transactions on Machine Learning Research (10/2024)
in Algorithm 5 in Appendix E Syuthetic graphs close to their decision boundary can then be obtained by sampling from their interpolated graphon of obtained by mixup of graphons of both the target classes. Algorithm 1 describes the method to generate explanations and boundary motifs using Eraphon-Explainer.
5 Theoretical Analysis
We give theoretical guarantees demonstrate that graphs generated from a graphon that is a convex combination of the estimated graphons of two target classes would contain discriminative motifs belonging to both classes. We have shown (in Section 4.5 how such graphons can be used to generate synthetic graphs that lie close to the decision boundary of the two target classes. This validates the intuitive notion that graphs that lie on or close to the decision boundary of two target classes contain discriminative features of both classes. We first give a concrete definition of a discriminative motif and state a few basic assumptions on which the theoretical guarantees rely on. Definition A discriminative motif Me of graph G the smallest subgraph of that gives G its class identity. In, other words Me is a subgraph with a minimal number of nodes and edges, which is essential for G to be identified as belonging to its corresponding class.
Note that, if there exists graph homomorphism from a motif Mg to graph G, then the motif Ma exists in graph G. Thus, the homomorphism density t t(Mc, G) gives a mcasure of the frequency of occurence of Ma in G We assume that every graph belonging to a target class e contains a discriminative motif M.. Further, we also assume that any collection of graphs De that has been identified by a classifier as belonging to a class e contains a finite set of discriminative motifs Me. The following thcorem shows that the boundary graphon partially contains discriminative motifs belonging both the larget classes.
Theorem 5.1. Let De and Dez be two collections of graphs belonging to the target closses e1 and 02, respeclively, with their eslimated graphons Se1 and S Define the boundary graphon as Sh = X5 + (1 A)SSa For any discriménative motif Mc, E Me, and Mc, E Moo, the difference in the homomorphism density of Me, and Moz with the boundary graphon Sh compared to their respective graphons S and S is bounded above by:
( t(Me, 8*) S (1 A))E(M)))) - -50 ( ) - 1(Me,, S*)] S X F(M - -501
Proof. The proof is detailed in Appendix D.I
Theorem 5.1 establishes an upper bound on the difference in homomorphism densitics between a discrimi- native motif of target class and both the graphon of the larget class and the boundary graphon. In other words, the difference between the frequency of occurence of a motif in the graphon of a target class and in the boundary graphon is upper bounded. This upper bound is solely dependent on the hyperparameter X, since the values of [E(M) and (E(M),)) are constants determined by the discriminative motifs.
Next, we present a theoretical guarantee that graphs generated from the graphon 5 will inherit the motifs contained in 56.
Theorem 5.2. Let Sh be the boundary graphon and assume that it contains discriminative motif Mr. Then any random graph G on n nodes generated from sb satisfies:
en2
G)
< 2exp
Proof. The proof is detailed in Appendix D.3
Theorem 5.2 demonstrates that the topology of random graph sampled from the boundary graphon closcly resembles the topology of the boundary graphon itself, which includes discriminative motifs from both classes. Consequently, the theoretical analysis in this section confirms that graphs generated by the boundary graphon 5h will contain discriminative motifs from both target classes. Further, a time complexity analysis and a theoretical guarantee of diversity in the generated explanations of Graphon-Explainer is provided in A Appendix D2. and D.2
7

---

Published in Transactions on Machine Learning Research (10/2024)
6 Experimental Analysis
We evaluate the explanation generation capabilities of Graphon-Explainer through meticulously designed experiments using three real-world datasets and three synthetic datasets. Initially, we train a GNN classifier on each dataset and employ Traphon-Explaingg synthetic motifs that lie close the decision boundary between two target classes.
generale explanations for specific larget classes and
6.1 Metrics for Quantitative Evaluation of Generated Explanations
Unlike instance-level explanations, model-level explanations lack a corresponding ground truth because it's impossible to determine the exact graph patterns the classifier has learned to distinguish between classes. For generating explanations for individual largel classes, the objective is to maximize the class score for the target class when the generated explanation is provided as input to the classifier. For generating graphs that lic on the decision boundary, the objective is to achieve a class score indicating that the generated graphs belong to all classes with equal probability according to the classifier. With this goal in mind, we evaluate our method using the metrics established in the literature & Shen 2022 Chen al., 2024 Yuan al., 2020) to assess model-level explanations. We report the performance based on the following metrics: Target Class Score: The target class score of generated explanation represents the probability with which the explanation belongs to the larget. class, determined by the classifier f(.). When generating explanations for target class c, each generated explanation is inputted into f(-), and the probability for class is recorded. We report the mean and standard deviation for 50 generated explanations for each target class. For syuthetic graphs generated on the decision boundary, we report the class score of the generated graph across all classes. For target class explanations, a. higher class score suggests that the gencrated explanation includes discriminative motifs identified by the classifier as belonging to the target class, resulting in the classifier confidently classifying the explanation into the target class. Conversely, for synthetic motifs intended to lie close the decision boundary, the goal is to achieve class scores that assign the generated motif an equal probability of belonging to both the classes.
Density: This metric assesses the sparsity of the generated explanations. In a graph G, density is calculated [E] Density = T where E represents the edge set and V denotes the vertex set of G. It is crucial that explanations generated for class include only edges that distinctly characterize that class and are essential for classification, thereby minimizing non-vital edges. Lower-density explanations facilitate clearer interpretation for users, enhancing model transparency and trustworthiness. We provide the average density and standard deviation of 50 explanation graphs for each target class.
Time Efficiency: Following the evaluation setting in GNNInterpreter (Wang & Shen 2022 we also note the time taken to generate 10 explanations for each target class and report the mean time taken across all target classes in a particular datasct. We denote this using Tio in Table 2
25hapes
IMDB-B
Reddit-B
Classd
Close
Lollipop
Tree
Classl (Action)
Classi (Romance)
QQestion!
(Discussion)
4Shapes
Bar
Windamill
Wheel
Star
MUTAG
BA.Mootff
Carbon
Fluorine Bromine
Nitragen
Indine
Ovygen
Chlorine
Mutagenic
Non- Mustagenic
Cycle
House
Figure 2: Examples of Graphs in all Target Classes on all Datasets
8

---

Published in Transactions on Machine Learning Research (10/2024)
6.2 Metrics for Evaluation of the Decision Boundary of the Classifier
The boundary margin (Yang et al. 2020) measures the minimum distance weew the embeddings of graphs from a class c1/02 and those at the boundary between classes and For a class C1, it defined as:
Boundary Margin:
(ffa))
min (l((Ga)) - ((G+)]
(Ga
where (Ge Gb) represents pair of graphs such that Ga belongs class by having the highest class for class 1 according to and G2 is the boundary between class and C2. Here, & denotes the embedding function of the GNN classifier f. A wider boundary margin implies better chance of generalization properlies of the classifier since the risk of misclassification decreases during test time. In this paper, for the choice of & we use the embedding of the last hidden layer, after which a lincar layer is applied to get the class specific logits. This ensures that we compute this measure with linearly separable embeddings.
Boundary Thickness: A thicker boundary signifies greater robustness while thinner boundary is more prone to adversarial attack. The boundary thickness (Yang ct al.) 2020) between target class c c2/02 and the decision boundary separating the two classes, can be delined for the class 61,
[Id(Ge)) ((bi)l L 1(6 o(h(t)) ohh(t))cs dt
(( B(G.,
where: h(t) = (1- ( + (GG)) is the linear interpolation between embeddings, (hh(t)), and (h(()) are the probabilities for classes ej and 02 respectively, II the indicator function, a threshold, commonly set 0.75.
Boundary Complexity: The boundary complexity Guan & Loew 2020 quantifies the intricacy of the data and the classifier's decision-making process by assessing the entropy of the eigenvalue distribution from the covariance matrix of boundary embeddings. It is expressed as:
P
02)
log K
where: μ are the eigenvalues of the covariance matrix of the boundary embedding set d(G,), || is the L1 the entropy of the normalized eigenvalues, K is the dimensionality of the embeddings. Datasets with weal-world complexity or more classes typically exhibit more intricate decision boundaries compared to synthetic datasels or those with fewer classes.
norm of μ1, H [
6.3 Datasets
We conduct experiments on three synthetic and three real-world datasets.
Table 1: Dataset Properties and Classifier Accuracy
Dataset
Classes
#Graphs
Average # Nodes
Average Friges
Chussiiee Accuracy
IMDB-B B
1000
18.77
96.53
0.7050
Reddit-B
2000
429.43
497.76
0.8250
MUTAG
188
17.93
19.79
0.88297
2Shapes
9.32
10.37
1.0
Motii
1030
25,88
0.958
48hapos
4000
15.89
22.88
0..9
Synthetic Datasets:
We generate the 2Shapes and 4Shapes datasets. 2Shapes contains graphs belong-
ing to two classes: Lollipop and Tree. 4Shapes contains graphs belonging 4 classes: Windmill, Wheel,

---

Published in Transactions on Machine Learning Research (10/2024)
Star, and Barbell. A graph belonging to particular class has the shape described by the name of the corresponding class. Examples of graphs in all the target classes are shown in Figure 2 The method to generate the datasets is detailed in Appendix A We also demonstrate the effectiveness of our method on the BA2Motif (Luo et al.) 2020) dataset which consists of graphs with two classes of motifs: house and cycle. Real Datasets: For real datasels, we conduct. experiments using the IMDB-BINARY (IMDB-B), REDDIT-BINARY (REDDIT-B), and MUTAG datasets. The IMDB-B dataset comprises 1,000 movie collaboration networks, where nodes represent actors or actresses and edges denote co-appearances in movies. The graphs are categorized into Action or Romance genres. The REDDIT-B dataset represents Reddit discussions, where nodes correspond to users and edges indicate replies. Each graph is labeled as belonging to either a dnvestion/answer-beee compound graphs, with nodes as atoms and edges as bonds, labeled as either mulagenic or non-mulagenic. Examples of graphs from all datasets are shown in Figure 2 Dataset properties and classifier test accuracy are summarizod Table with the architecture of the GNN classifiers detailed in Appendix
or discussion-based based community. The MUTAG dataset contains chemical
6.4 Results
We assess the performance of Graphon-Explainee both qualitatively and quantitatively. We also demonstrate comparative results by comparing our method with GNNInterpreter and XGNN. Our code repository is provided at htttps///githb.. wwt.coom/cmc/eeee
Saapphnn-Expaainer Results all metrics presented in Table 2 for synthetic datascts and real datascts. An overarching trend observed across almost all datascts is that Graphon-Explainee can generate explanations around 10 times faster than GNNnnterpreter. Table 3 presents the results for motifs generated near the decision boundary. The class scores clearly shows that Graphon-IExplainer produces boundary motifs with competitive, or even superior, boundary scores compared to those generated by GNNBoundary. It should be noted that the decision boundary metrics in Table 3 with respect to the classifier are computed using the motifs generated by Craphon-Fxplainee Qualitative results in Figure 3a| and (3b] show the estimated graphon for each target class and the corresponding explanation generated by
and GNNInterpreter. Since XGNN, on most classes generates a single node or line graph as explanations, the generated explanations are deferred to Figure G in the Appendix
F The boundary motifs generated by GNNBoundary is also deferred to the Appendix K It can be observed that the estimated graphons of each target class clearly look different. Hence, each class has a distinct motif based on which the classifier categorizes them into separate classes. The visualization of the estimated graphons of different target classes in Figure 3a and 3b gives added insight into the structure of graphs contained in various target classes and help make the generative model itself more interpretable.
Table 2: Quantitative Results
Clos Seree
Deseity
To
Clos Sioee
Beuily
5o
Closs Stoee
Deacity
T2
Silqprs
Edllpop
10 0.90
0148 à 0.821
0.31
6J075 is 0.71
0.085 & B.041
17553
6/66 a 0.30
835 000
415315
BAZ30att
Houe
10 0.00
0JM57 : ULIAAZ
0:47
U1003 2 MIB4
2233
Baars 20 300
035 (M))
30656
Carl:
09035
0.907
Barbell
8M 9.000
010922 c WEMOS
0M00 - HU901
11405
10 2000)
Wisholl
0
0198
60000
Wheel
0.88 0,0006
(U33) - (600
0.1100 64005
6n 60007
035 400
Biar
0:000
0.0053 0.0008
32001
01765 AM8RO
00 01000
Milae
Mulagysk
Ln 0.00
0,075 0L0N77
37031
0JIN4 W02E
935
Ln 0,0
01723 10000
116
Nop Mutagiii
0.000
0.0900 2 0.0002
0060: a 00002
01166 à 10907
10:1 0.00
83536 10010
Real Delast
DMDB BSnary
(Actia)
Choul
18 0.0
02114 (L00
3.34
00003 - 00080
07763 00105
6:16) 0000
01:+0:00
418
(Bemanee)
Chual
0.00
0.2034 0.0014
0,2100 M09G
0J033 1000
Roddii-Bisas
(Quntias/ Answe))
03834 2 0.000G
IU3166 4L9X0
2.46
IUD23 a 001122
0.0154 0.002L
15.35
0V04
03356 1042
(Dacusion)
Synthetic Datasets: The quantitative results in demonstraae that Craphon-Explaines consistently outperforms GNNInterpreter and XGNN across nearly all metrics. This indicates that Graphon-Fxplainee is
10

---

Published in Transactions on Machine Learning Research (10/2024)
Table 3: Quantitative Results on all Datasets for the Decision Boundary
Dataset
Graphon-Taplaliee
GINNooundby
Boundary Complexity
Boundary Margin
Boundry Thicknes
Sevees
15
Kio)
p/o1
zloe]
Khapes
Fdlipe
The
1/0900
17700
6/6861
733
0003111
0.1912
0.0002
0.5128 + 0.0001
0 01968 4 2507
0,7010
0.3000
BAGMoxir
Cysl:
0,114
1998677
0.1387
9,335 DC450
0.0400
0.40914 0.0841
Chus)
LMDB-B
10038
015894
1/652
8.7237
0.2345
0.813 B0002
0,5025 0.000)
0,4994
0.0000
0.0000
(Artino)
(Bomatce)
Clesi
Cloel
Ro
0.2739
0.31
1.84
2.46
0,482
0.5006
0.0000
0.5008 0,0000
0. 0.4888
0.5184
0.0213
Quaticu/ Asses]
(Docsokus)
Martag
Maragerl
19:333
31.7
03.400
31811
0.096
0.5014
0.0522
0.1882 1. 0.0018
0:9506 1 0020007
0.7H 0022961
Butell
Whedl
0.731
0.0000
1/41066
3.077
0.02936
0.4762
0.4812 0.0091
0.0084
IShapee
Buleil
0.0079
0.07444
0,6001
0
0.59GG
0.5AAL 0.2%1
Washuil
Whrdi
0,813
0,000
2,4115
4.629
0.885
0.4990
0.0010
0.3052 d: 0.0014
0.7563 a: 0.1627
0.3000 0.1495
Wasimlll
Ste
0.4313
3:445
5.300
0.200)
0.4812 4: 0.0232
0.4852 4: 0.0150
(L85T a: 11
1.3122
(11490)
not only capable of generating discriminative motifs that closely align with the classifier's learned patterns but does so more efficiently, producing sparser explanations in significantly less time than existing methods.
The qualitative analysis in Figure 3a further supports these findings, showing that the explanations generated by Graphon-Explaing align well with the class identities when the classifier performs accurately, as seen with the 2Shapes dataset. The classification task on this dataset is relatively straightforward for the classifier, as evidenced by the minimal boundary complexity shown in Table 3 and the confusion matrix in Figure 4
However, the 4Shapes datasct presents a more challenging scenario. While the explanations for the Barbell, Windmill, and Wheel classes may not perfectly match the class identities, they still achieve high class scores, suggesting greater likelihood of misclassification for these classes. Analysis of the adjacency scores in Figure 5 and decision boundary metrics in Table 3 reveals that within the 4Shapes dataset, the Windmill and Wheel classes share the thinnest boundary margins and the highest boundary complexity. Additionally, the Windmill class has a thinner boundary in its decision boundary with the Wheel class, indicating a higher propensity for misclassification as Wheel rather than the other way around. This suggests that the embeddings of these classes are more closely aligned, making it more difficult for the classifier to distinguish between them, as reflected in both the confusion matrix in Figure 4 and the similarity of their explanations in Figure 3a.
The highest boundary complexity among the syuthetic datasets is observed for the BA2Motif dataset, likely due to the more complex task of classifying random graphs based on the presence of a specific motif a prob- lem that is inherently more challenging due to the noise within these graphs. Moreover, graphs containing a House motif are more likely to be misclassified as Cycle, as indicated by the thinner boundary margin for the House class. This finding is corroborated by the confusion matrix in Figure 4 Finally, an analysis of the motifs generated near the decision boundary in Figure [aa shows that motifs close to the boundary between two target classes often represent mixture of motifs from both classes.
Real Datasets: The results in Table 2 clearly highlight the superiority of Graphon- Explainer over existing methods. A qualitative analysis of the explanations from Figure 3b on the MUTAG dataset shows that GrapaionEExplainer can generate more realistic molecular features compared to GNNlnterpreter. It's worth noting that XGNN also performs well on this dataset, producing more realistic explanations than GNNIn- terpreter, as ratustrated in Figure 6 For the Mutagenic class, fusod ring structure and the NO₂ subgroup key distinguishing fcatures of this class whereas these are ab- sent in other explanations. For the Non-Mutagenic class, no O alom is present in the explanations, aligning with the class identity. An analysis of the boundary metrics from Table 3 reveals that the Non-Mntagenic class has a lesser boundary thickness, which increases the likelihood of misclassification of Non-Mutagenic compounds as Mutagenic. This observation is supported by the confusion matrix in Figure 4 Moreover, the boundary complexity score is the lowest on this dataset compared to other real datasets, indicating that the decision rules for class assignment on the MUTAG dataset are simpler.
explanations include the
On the REDDIT-B dataset, both Explanations for Class 0 (the Question-Answe class) in Table 3b reveal a pattern of one high-degree
and GNNInterpreter perform well, as shown in Table
2
11

---

Published in Transactions on Machine Learning Research (10/2024)
Duteset
Lellpe
Tree
Beitian Boredary
25huges
BAIModl
Witee Dicktón Barndary
Bateel Ster Deckker Hlunedery
4hhupes
Winf
Irindeell Schedd Decidan Berndayy
Kisdadi Dociden Rousdaay
Chille
Pepiien
(a) Synthetic Dataset
Classl (Action)
Class1 (Romance)
Decision Houndary
IMDB-B
Apurodraed
Coiglie
Clasadi (Questionn Amswer)
Clast (Discuston)
Decision Boundary
Reddit-B
Mutagenic
Mun-Muiagenic
Dreinion Boundary
Appesincce Lragiim
Darge Cilelie
Oogpe Cilelee
Dogin Clinke.
(b) Real Dataset
Figure 3: Qualitative results across all datasets. Graphon-Explainer explanations have class scores of 1, and boundary motifs score close to 0.5. The top-scoring GNNnteeprrote explanations are also shown. the approximated graphons, colors transition from dark to bright, reflecting values from to with brighter colors indicating higher values.
node connected to several low-degree nodes, capturing the characteristic of a single user answering multiple questions-4 key feature of this class. In contrast, explanations for Class 1 (the Discussion class) show few users frequently interacting among themselves, with other
on the fringe engaging with only a single user, typical of discussion groups. Boundary analysis from Table 3 indicates that the boundary margin and thickness are both smaller for the Question-Answer class, increasing the risk of misclassification as belonging to the Discussion class. This finding is corroborated by the classifier's confusion matrix in Figure
On the IMDB-B dataset, Graphon-Exppainee Explainer comprehensively outperforms competing methods, as demon- strated in Table Explanations for Class (Action genre) in Table 3b feature the signature motif of multiple co-actors who have previously worked together, reflecting the typical structure of Action movie with mul- tiple popular actors in key roles. In contrast, explanations for Class (Romance genre) show distinct motif
12

---

Published in Transactions on Machine Learning Research (10/2024)
of one popular actor interacting with multiple groups of actors, consistent with the genre's focus two central characters. The boundary graph appears to combine these two distinct motifs. Additionally, boundary metrics analysis indicates that this task has the highest boundary complexity, suggesting that the classifier's 's decision-making process to distinguish between these two classes of graphs is particularly complex. Furthermore, graphs the Action class are more prone to misclassification due to lesser boundary thickness, which is validated by the confusion matrix in Figure
6.5 Sensitivity Analysis
Figure 12 in Appendix L illustrates how class scores vary with changes in Graphon-Explainer's hyperpa- rameters as listed in Algorithm [ For explanations of individual target classes, we adjust the mumber of partitions K used to approximate the graphon, which alters the number of nodes in the generated graphs. For graphs generated to belong close to the decision boundary, we vary the linear interpolation parameter À and report the class score variations for a single class. We also detail the Asammle hyperparameter used for each dataset in Table g in Appendix L It can be concluded from the plots in Figure [2] and Table g that the target class scores for explanations do not vary much with change in K, class scores for boundary motifs remain in the desirable range when À is around 0.5 and Rsamples used is reasonably low which makes our method robust to hyperparameter changes.
7 Conclusion
This study introduces graphons, approximating one for each target class, to serve as generators for discriminative motifs identified by the GNN as indicative of each class. Unlike existing model-level techniques limited to generating expla- nations for individual target classes, motifs close to the decision boundary of two target classes. This capability not only elucidates what the GNN has learned for each class but also aids delineating the decision boundary of the trained GNN, Through exlensive experiments and theoretical analysis, we demonstrate that Graphon-Explaine faithfully elucidates any GNN, highlighting its strengths and pitfalls in graph classification tasks. Notably, it achieves this while generating diverse explanations significantly faster than current statc-of-the-art methods.
model-level explanation approach for GNNs that leverages
Sraphon--xplaine
transcends this constraint by producing synthetic
We believe Exxpino-Explainee can be used in many real-world applications to inform users about the induc- tive biases and underlying classification logic of a GNN model deployed in practical scenarios. For example, our experiment on the MUTAG dataset demonstrates how the classifier identifies fused rings as key motifs for mutagenic classes. Similarly, our method could explain a GNN classifying toxic and non-toxic drugs, revealing the basis on which the GNN classifies a molecule as toxic. Analyzing the decision boundary of such a GNN could provide critical insights into the model's robustness and the risk of misclassifying toxic molecules non-toxic.
Furthermore, as demonstrated by additional experiments in Appendix H even when two classifiers perform accurately, their interpretations of a single class can vary siguificantly based on their inductive biases. This insight can help users choose between classifiers with equal accuracy based on their unique interpretative properties in real- world settings. Moreover, our approach can generate explanations tailored to the specific datasot a user possesses since estimated graphons effectively capture the dataset's 's distribution. This person alized approach offers tailored insights depending on the types of instances present in user's query dataset, rather than providing uniform explanations regardless of dataset variations.
Overall, -xxlaancr not only advances our understanding of GNN behavior but also equips users with a powerful tool to assess, interpret, and trust the decisions made by GNNs in diverse application domains.
Acknowledgments
SB acknowledges the JC Bose Fellowship grant No. /0000/////// from ANRF- Govt. of India.
13

---

Published in Transactions on Machine Learning Research (10/2024)
References
Edo M Airoldi, Thiago B Costa, and Stanley H Chan. Stochastic blockmodel approximation of graphon: Theory and consistent estimation. Advances in Neural Information Processing Systems, 26, 2013.
Steve Azzolin, Antonio Longa, Pictro Barbiero, Pictro Liò, and Andrea Passerini. Global explainability of gnns via logic combination of learned concepts. arXiv preprint xrXi:22200007147, 2022.
Federico Baldassarre and Hossein Azizpour. Explainability techniques for graph convolutional networks.
arXiv preprinl Xr:: 13686, 2019.
Stanley Chan and Edoardo Airoldi. A consistent histogram estimator for exchangeable graph models. International Conference on Machine Learning, pp. 208-216 PMLR, 2014.
Sourav Chatterjee. Matrix estimation by universal singular value thresholding, Annals of Statistics, 2015.
Jialin Chen, Shirley Wu, Abhijit Gupta, and Rex Ying. D4explainer: In-distribution explanations of graph neural network via discrete denoising diffusion. Advances in Newral Information Processing Systems, 36, 2024.
Jianbo Chen, Le Song, Martin Wainwright, and Michael Jordan. Learning to explain: An information- theoretic perspective on model interpretation. In International conference on machine learning, pp. 883- 892. PMLR, 2018.
Lukas Faber, Amin K. Moghaddam, and Roger Wattenhofer. When comparing to ground truth is wrong: On evaluating gmn explanation methods. In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining, pp. 532-341, 2021.
Qizhang Feng, Ninghao Liu, Fan Yang, Ruixiang Tang, Mengnan Du, and Xia Hu. Degree: Decomposition based explanation for graph neural networks. ar preprint arXiv:2305.12890 2023.
Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In International conference on machine learning, pp. 1263 1272. PMI.R, 2017.
Shuyue Guan and Murray Loew. Analysis of generalizability deep neural networks based on the complexity of decision boundary. In 2020 19th IEEE international conference on machine learning and applications (ICMLA), pp. 101-106. IEEE, 2020.
Xiaotian Han, Zhimeng Jiang, Ninghao Liu, and Xia Hu. G-mixup: Graph data augmentation for graph classification. In International Conference on Machine Learning, pp. 8230-8248. PMLR, 2022.
Qiang Huang, Makoto Yamada, Yuan Tian, Dinesh Singh, and Yi Chang. Graphlime: Local interpretable model explanations for graph neural networks. JEEE Transactions on Knowledge and Data Engineering, 2022.
Jaykumar Kakkad, Jaspal Jannu, Kartik Sharma, Charu Aggarwal, and Sourav Medya. A survey on ex- plainability graph neural networks. arXiv preprint orXiv:2306.0190 2023.
Raghunandan H Keshavan, Andrea Montanari, and Sewoong Oh. Matrix completion from few entries. IEEE transactions on information theory, 56(6):2980 2998, 2010.
Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, et al. Inter- pretability beyond fcature attribution: Quantitative testing with concept activation vectors (tcav). In International conference on machine learning, pp.
PMLR, 2018.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. ar preprint arXiv:1412. 6980, 2014.
Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint ar X 160. 02907, 2016.
14

---

Published in Transactions on Machine Learning Research (10/2024)
Remi Lam, Alvaro Gonzalz,, Matthew Willson, Peter Wirnsberger, Meire Fortunato, Ferran Alet, Suman Ravuri, Timo Ewalds, Zach Faton-Rosen, Weihua Hu, et al. Learning skillful medium-range global weather forecasting. Science, 312(6677) 1421, 2023.
Wanyu Lin, Hao Lan, Hao Wang, and Baochun Li. Orphicx: A causality-inspired latent variable model for interpreting graph neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13738, 2022.
László Lovész. Large networks and graph limils, volume 60. American Mathemalical Soc., 2012.
László Lovász and Balázs Szegedy. Limits of dense graph sequences. Journal of Combinatorial Theory, Series B, 96(6):933 957, 2006.
Ana Lucic, Maartje A Ter Hoeve, Gabriele Tolomei, Maarten De Rijke, and Fabrizio Silvestri. Cf- gnnexplainer: Counterfactual explanations for graph neural networks. In International Conference on Artificial Intelligence and Statisties, pp. 4499 4511. PMLR, 2022.
Dongsheng Luo, Wei Cheng, Dongkuan Xu, Wenchao Yu, Bo Zong, Haifeng Chen, and Xiang Zhang. Pa- rameterized explainer for graph neural network. Advances in neural information processing systems, 33: 19620-19631, 2020.
Amil Merchant, Simon Batzner, Samuel Schoenholz, Muratahan Aykol, Gowoon Cheon, and Ekin Dogus Cubuk. Scaling deep learning for materials discovery. Nature, 624((990):80 85, 2023.
Peter Müller, Lukas Faber, Karolis Martinkus, and Roger Wattenhofer. Graphchef: Decision-tree recipes to explain graph neural networks. In The Twelfth International Conference on Learning Representations, 2024.
Yi Nian, Yurui Chang, Wei Jin, and Lu Lin. Globally interpretable graph learning via distribution matching. In Proceedings of the ACM on Web Conference 2024, pp. 992-1002, 2024.
Phillip E Pope, Soheil Kolouri, Mohammad Rostami, Charles E Martin, and Heiko Hoffmann. Explainability methods for graph convolutional neural networks. In Proceedings of the IEEE/CVE conference on computer vision and pallern recognition, pp. 10781, 2019.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. whhy should i trust you?' explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1135 1144, 2016.
Luana Ruiz, Luiz Chamon, and Alejandro Ribciro. Graphon neural networks and the transferability of graph neural networks. Advances in Neural Information Processing Systems, 33:1702 1712, 2020.
Michael Sejr Schlichtkrull, Nicola De Cao, and Ivan Titov. Interpreting graph neural networks for nlp with differentiable edge masking. arXiv preprinl arXim:2010.005: 2020.
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision, pp. 618 626, 2017.
Ana Vasilcoiu, T.H.F. Stesson, Thics Kersten, and Batu Hclvacioglu. [re] GNNInterpretor: A probabilistic generative movel-level explanation for graph neural networks. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL htttp:: https:://peenreiier wtt.://www.......eomoo///o/aaaaa
Petar Veličković, Guillem Cucurull, Aranixa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In International Conference on Learning Representations, 2018. URL https: wttperrrrrie.eeetffouuurrrrddrdr
Renato Vizuete, Foderica Garin, and Paolo Frasca. The laplacian spectrum of large graphs sampled from graphons. IEFF Transactions on Network Science and Engineering, 8(2):1711-1721, 2021.
15

---

Published in Transactions on Machine Learning Research (10/2024)
Minh Vu and My T Thai. Pgm-explainer: Probabilistic graphical model explanations for graph neural networks. Advances in neural information processing systems, 13:12225-122333 2020.
Xiaoqi Wang and Han Wei Shen. GNNInterpreter: A probabilistic generative model-level explanation for graph neural networks. In The Eleventh International Conference on Learning Representations, 2022.
Xiaoqi Wang and Han Wei Shen. GNNBoundary: Towards explaining graph neural networks through the lens of decision boundaries. In The Twelthh International Conference Learning Representations, 2024. URL https://openrevier www.eewwweeeeeeeeeeeeeeeeeeeeeee net.forrmmi
Shiwen Wu, Fei Sun, Wentao Zhang, Xu Xie, and Bin Cui. Graph neural networks in recommender systems: a survey. ACM Compuling Surveys, 55(5):1-33) 37, 2022.
Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. A comprehensive survey graph neural networks. TEEE transactions on neural networks and learning systems, 32(1) 4-422,,, 2020.
Hongteng Xu, Dixin Luo, Lawrence Carin, and Hongyuan Zha. Learning graphons via structured gromov- wasserstein barycenters. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 10505-10511 2021.
Yaoqing Yang, Rajiv Khanna, Yaodong Yu, Amir Gholami, Kurt Keutzer, Joseph E Gonzalez, Kannan Ramchandran, and Michael W Mahoney. Boundary thickness and robustness in learning models. Advances in Neural Information Processing Systems, 33:6223-6234 2020.
Zhaoning Yu and Hongyang Gao. Motifexplainer: a motif- based graph neural network explainer, 2023. URL http://www....eeooooooo///////a.
Zhaoning Yu and Hongyang Gao. Mage: Model-level graph neural networks explanations via motif-based graph generation. arxiv preprint 0rxiv:2405. 1225..220 12519, 2024.
Hao Yuan, Jiliang Tang, Xia Hu, and Shuiwang Ji. Xgnn: Towards model-level explanations of graph neural networks. In Proceedings of the 26th ACM STGKDD international conference on knowledgee discovery & data mining, pp. 430-438, 2020.
Hao Yuan, Haiyang Yu, Jie Wang, Kang Li, and Shuiwang Ji. On explainability of graph neural networks via subgraph explorations. In International conference on machine learning, pp. 12241-12252. PMLR, 2021.
Hao Yuan, Haiyang Yu, Shurui Gui, and Shuiwang Ji. Explainability in graph neural networks: A taxonomic survey. IEEE Iransactions on pallern analysis and machine intelligence, 45(5))578225777
2022.
Yue Zhang, David Defazio, and Arti Ramesh. Relex: A model-agnostic relational model explainer. In Proceedings of the 2021 AAAT/ACM Conference on AT, Fthics, and Sociely, pp. 1042-1049, 2021.
Ziwei Zhang, Peng Cui, and Wenwu Zhu. Deep learning graphs: A survey. IEEE Transactions on Knowledge and Data Engineering, 34(1):249 270, 2020.
A Synthetic Dataset Generation
We detail the method to generate the datasets 2Shapes and 4Shapes in Algorithm 2 and Algorithm respectively.
3
16

---

Published in Transactions on Machine Learning Research (10/2024)
Algorithm 2 2shapes dataset generation
Graph Classcs:
[LOLLIPOP, TREE}
Output: A collection of pairs consisting of graph and its label
Procedure:
For every class, C Graph Classes, do 1000 times:
-ff LOLLIPPP:
Sample lollipop graph Giollipop with random number of head nodes E (2,-... 6} and random number of tail nodes k (4,..., 16]
Return (Go, C).
if TREE:
Sample r-ary tree Giree with random E (2,3, and random number of nodes n E {3, ..... ,13 Return (Gc, C).
Algorithm 3 4Shapes datasct generation
Graph Classes: (BARBELL, WINDMILL, WHEEL, STAR]
Output: A collection of pairs consisting of graph and its label Procedure:
For every class, C Graph Classos, do 1000 times:
if C -BARBELL:
Sample barbell graph GBrreell with random number of head nodes m, -1..... .4) and random
number of connector nodes (4....,
Return (Go, C).
if C WINDMILL:
* Sample windmill graph Gwiindmill with random number of cliques A E (3.... ,9 and each clique of random size s (3,...,)
* Return (Gc,C).
if - -HEEEL
* Sample wheel graph Gwheel with central node and random number of non-central nodes
{4, ... 16)
* Return (Go, C).
if C = STAR:
* Sample star graph Gatar with random number of non-center nodes n E (6,-..,30 Return (Go, C)).
17

---

Published in Transactions on Machine Learning Research (10/2024)
B Classifier Architecture
On all datascts except MUTAG, we implement deep GCN architecture comprising three GCN layers, cach with a dimension of 64. This is followed by global mean pooling layer, batch normalization layer, and finally linear layer to obtain the class logits. We utilize Leaky ReLU as the activation function between the GCN layers and within the MLP. The model parameters are initialized using the Kaiming method. Training is conducted using the Adam optimizer (Kingma & Ba 2014 with an initial learning rate of 0.01. Additionally, we use a learning rate scheduler to halve the learning rate every hundred epochs. On the MUTAG dataset, the model architecture consists of three graph convolutional layers designed to extract features from imput graphs. The first and third layers Graph Convolutional Networks (GCN), while the second layer is Graph Attention Network (GAT) with two attention heads, enabling the model to focus important nodes and edges. A residual connection from the input features to the output of the third layer ensures better gradient flow and feature retention. The model employs Jumping Knowledge mechanism in concatenation mode, aggregating features from all three layers to capture hierarchical information. The aggregated features are globally pooled, batch normalized, and passed through a dropout layer with a 0.5 probability prevent overfitting Finally, these processed features are fed into lincar classifier that outputs the class logits. This architecture is trained the same way the above archilectures.
The confusion matrix for each of the trained classifier is shown in Figure
2Shapes
4Shapes
BA2Motif
MUTAG
IMDB-B
Reddit-B
Figure 4: Confusion Matrix of the Trained Classifier on Datasets
C Graphon Estimation and Related Guarantees
The algorithm for estimating the graphon of target class detailed in Algorithm ??.
The weak regularity lemma for graphons Lovvss 2012) can be stated as
Theorem C.1. Let be any gruphon and let 21 E Z. Then, there erisls a slep fanction Sp over a partition (P,,... ,P] of the wnit intervol such that |S Splla S Vlog ||
This theorem guarantees that there would always exist step function and partition with which any given graphon can be approximated well in the cut norm. Also note that by increasing the number of partitions, one can arbitrarily increase the accuracy the approximation.
D Theoretical Guarantees
D.1 Proofs of the Theorems
Theorem D.1. Let De, and De, be two collections of graphs belonging the target classes e₁ and C2 respectively, with their eslimated graphons Sei and Se2 Define the boundary graphon Sh A + (1
18

---

Published in Transactions on Machine Learning Research (10/2024)
Algorithm Estimating the Graphon of a Target Class and Generating Graphs from the Graphon
Set of graphs De belonging to class c, threshold for clipping singular values, desired number of nodes N (where N must not exceed the maximum number of nodes in any graph from De)
Input:
Init:
List of sorted adjacency matrices A, List of sorted node features X
Output: Estimated graphon S for class c, Node feature of the generated graph Xe, Adjacency matrix of the generated graph A.
Procedure:
Align and Sort Nodes:
For each graph G (A in the set De:
Compute node degrees from the adjacency matrix A.
Sort nodes in descending order of their degrees to obtain the sorted adjacency matrix A and sorted node features X.
Pad A to size N x and X to size N d, where d is the node feature dimension. Append the sorted and padded adjacency matrix A the list A.
Append the sorted and padded node features X to the list X.
Graphon Estimation using USVT:
Compute the mean adjacency matrix sum graph A E! Ai, where A; represents each sorted
adjacency matrix in the set
Perform Singular Value Decomposition (SVD) on sum graph: sum graph = UEVT.
Apply singular value thresholding: set a; 0 if or < x VN.
Reconstruct the graphon S UVVT using the modified singular values.
Clip the values of 5 to the range [0.
Sampling Graphs from the Estimated Graphon:
For each generation of a graph using the graphon, repeat the following steps to generate its node feature Xe:
For i (I,...,N)}
Collect the tth row from all matrices X in X into a pool.
Assign probabilistic weight each unique row in the pool based on frequency of occurrence, excluding rows that are all zeroes to avoid padding effects.
Randomly sample the ith row of the graph's node feature using the assigned probabilistic weights.
Combine the sampled rows to form the node feature X. of the generated graph. Sample S. using Equation ! to generate the adjacency matrix A of the sampled graph.
Return: Sc Estimated graphon for class c, Xe: Node feature of the sampled graph, Ag: Adjacency
Matrix of the sampled graph
19

---

Published in Transactions on Machine Learning Research (10/2024)
X)5²z. For any discrimánatice motif Me, Me and Mez E Mez the difference in the homomorphism density of Me, and Moz with the boundary graphon 8 compared to their respective graphons S and S is bounded above by:
; t(M,, 80) (1 A )lls - -
1 S0²) - 1(Meg S01 S X B(Mo))l S
Proof.
We use the following lemma from (Lváász & Szegedy 2006) for the proof of this theorem.
Lemma D.2. Let U, W be two graphons. Then, for every simple graph F -(F, [(F) W
Using D.2 we see that, 1t,M,, S0:) - t t(Me,, S (B(Ma)) 55
This can be rewritten as
[B(M))IlSS* -S - = -E((Ma A5 - (1 1)s = |E( Il[[( A)( -
Now, note that for any graphon S and positive constant E IR, the cut norm has the following property /xx [xxx S(x, v) = al|S||a
||aS||o = cc fxxx a5( v) = a suux,,rcoo. Using this property, we conclude,
EllM A)
(11 S In
The proof for motif Moz can be done similarly by mimicking this proof
Next, we present a theoretical guarantee that graphs generated from the graphon sh will inherit the motifs contained 5h.
Theorem D.3. Let 5h be the boundary graphon and assume that it contains discriminative motif M,. Then any random graph G on n nodes generated from Sh satisfies:
en
P(I((M, (((P, 5)
2 2exp
(8|V
Proof. We begin by stating a lemma from Lovász (2006) Lovássz & Szegedy our theorem will follow as a straightforward corollary.
2006,, from which the proof of
Lemma D.4. Let S be graphon and F be simple graph. Let G(n, S) be random graph on n modes sampled from the graphon S. Then, P(t(FF, G) t(F, < 2
en?
18[([1]
Setting, F My and S = Shh in Lemma D.4 the bound follows
Theorem 5.2 demonstrates that the topology random graph sampled from the boundary graphon closely resembles the topology of the boundary graphon itsolf, which includes discriminative motifs from both classes. Consequently, the theoretical analysis in this section confirms that graphs generated by the boundary graphon 5 will contain discrininative motifs from both target classes.
20

---

Published in Transactions on Machine Learning Research (10/2024)
D.2 Generation of Diverse Explanations
The estimated graphon S of target class is used to generate explanations for the target class When the graphon is approximated using step function over K partitions of the unit interval, then S E kk from which the adjacency matrix A is sampled as Aij BernStj. Consequently, the probability of generating two identical graphs using this sampling scheme II K ((S))) (1- S(j)))) which is extremely small since S E [0, 1] and K is a reasonably large number K is set be the median number of nodes of graphs contained the dataset D. This guarantees that Graphon-Explainer produces explanations that are distinct from each other.
x
D.3 Time Complexity Analysis
The estimation of a graphon using the USVT method on graphs, each with N nodes, and employing a step function with K partitions, has a time complexity of O(N3) (Xu et al. 2021). After optimization, generating a single explanation from & samples of the ostimated graphon requires O(sK) time for node feature generation and OssK²) for edge generation, resulting in total explanation generation complexity of Oss?²).
This is more efficient than GNNInterpreter, which has a time complexity of O(Tsk²) for single explanation, where T is the number of iterations needed for convergence, and represents the number of Monte Carlo samples. Notably, our method is non-iterative and does not require convergence for each explanation, as the graphon is estimated once for a target class and then sampled from, making our approach significantly faster than GNNInterprcter
Moreover, the XGNN method exhibits considerably higher time complexity of O(TRMS), where T is the number of episodes, R is the number of rollout operations per episode step, and M is the number of edges in the generated graph. Given that M typically exceeds K - 1, XGNN's time complexity is substantially worse.
E
Adjacency Score in Multi-Class Scenario
E.1 Algorithm for Computing Adjacency Score
A Algorithm Adjacency Score Computation
Input:
L: Last linear layer of the classifier that outputs class specific logits Embeddings Eey; Set of embeddings for class c₁-
Embeddings Eey: Set of embeddings for class C2-
Class indices Ct, 02: Indices of the larget classes for adjacency check.
Number of samples N: Number of samples to draw from each list for checking.
Number of interpolation steps T: Number of steps for linear interpolation.
Procedure:
Initialize adjacency score adjscore t- 0 for i = to N:
Initialize score S ++1
Randomly sample E Ec, and ec, E Ec,
for a
with step size 1:
Compute linear combination c c a ee, + (1- a) Eor Pass through the model: p coftmax(L(c)) if (p) & (21,
Set 5 <
break
adjscore adiscore + S
return adjscore/N
21

---

Published in Transactions on Machine Learning Research (10/2024)
In this section, detail in Algorithm the computation of adjacency scores for two target classes in multi-class scenario. If the score is above a certain threshold, we consider two classes to be adjacent. We use a threshold of 0.75 in this paper.
E.2 Adjacency Score on 4Shapes
Star
0.996
0.593
0.8
Windmill
0.6
Berbell
Wheel
0.593
0.0
Star
Windmill
Barbell
Wheel
Figure 5: Adjacency Scores between all classes in 4Shapes dataset
The confusion matrix depicting the scores which is used for choosing the boundary in 4Shapes dataset is shown in Figure
F XGNN Results
The visual results for XGNN shown in Figure
2Shapes
IMDB-B
Reddit-B
Classti
Classl (Discussion)
Lollipop
Tree
Class0 (Action)
Class (Romance)
(Question/ Answer)
4Shapes
Barbell
Windmill
Wheel
Star
MUTAG
B.AMottff
Mutagenic
Non- Mon-Mutagenic
Cycle
House
Figure 6: Visual Results from XGNN for real and synthetic dataset
22

---

Published in Transactions on Machine Learning Research (10/2024)
G
Code Usage for Comparison
We use the code for XGNN from the DIG repository
xgraph/XGNN We use the code for GNNInterproter from their official repository https://githur com/
yollamdblallaaa/ddMmmmeeerettrrr the code from this reproducibility study (Vasilcoiu el al. 2024) https:
H Analysis of Explanations Generated by Different Architectures
Table 4: Boundary Metrics for the GIN architecture
Boundary
Margin
Boundary Thickness
Boundary Complexity
Windmill
Wheel
Windmill
Wheel
4.1341
4.7986
3.1721
9.2698
0.21000
Barbell
Wheel
Barbell
Wheel
3.2143
2.9876
5.6178
5.2197
0.14100
Windmill
Barbell
Windmill
Barbell
1.2149
0.6987
1.4561
3.8654
0.09870
Star
Barbell
Star
Barbell
0.4129
3.9851
5.7614
3.2198
0.06842
Windmill
Star
Windmill
Star
2.1965
7.2684
2.4976
8.6514
0.10820
Table
Boundary metrics for the GCN architecture
Boundary Margin
Boundary Thickness
Boundary Complexity
Barbell
Wheel
Barbell
Wheel
0.7243
0.9656
5.64008
3.6377
0.02906
Barbell
Star
Barbell
Star
0.8292
2.807
7.3156
0.5376
0.07444
Windmill
Wheel
Windmill
Wheel
0.4993
0.4907
2.4315
4.6429
0.0855
Windmill
Star
Windmill 6.14
Star
0.4511
3.803
0.596
0.01686
Model
Confusion Matrix
Graphon
Generated Explanations
GCN-Model
GIN-Model
Figure Explanations the Barbell Class of the 4Shapes dataset for 2 different classifier architectures.
In this section, we demonstrate how different two classifiers are in their intrinsic interpretation of target class and their decision boundaries even in the scenario when they classify a dataset almost perfectly and even in the case when they have a 100% accuracy on some classes the dataset. For this, we use the previous GCN architocture detailed Appendix B and GIN of the same architecture on the 4Shapes dataset.
23

---

Published in Transactions on Machine Learning Research (10/2024)
Confusion Matrix
1.00
1,000
2,000
0.892
0.95
Windrill
0.90
7,000
2-996
1.400
0.85
Barbel
2,000
5,096
0.830
0.80
Wheell
0.692
1,000
0:420
Star
Wheel
Figure Adjacency Scores on 4Shapes for the GIN model
As seen in Figure 7 even though both the classifier have 100% accuracy on the Barbell class and the learned graphon generative model the same, sampling using equation / yields very different explanations. We find that the GCN model identifies Barbell using the branch like structures of the Barbell while the GIN model distinguishes the barbell class using the head like structure of the barbell. Also, shown in Figure8 and Figure 5 different pairs of classes share decision boundaries for both models as their inherent graph embeddings are different. Further, even when a pair of common classes share decision boundary, the boundary metrics are very different( demonstrated Table 4 and like the case Star and Barbell Barbell greater for
where the boundary thickness of Star greater for the GIN and the thickness the GCN architecture. However, they also share certain common grounds like the boundary complexity for the Windmill-Whne pair the highest for both the models indicating that separating these two classes is significantly harder. This is also validated by the confusion matrices of both the models in Figure
Smaller Explanations for IMDB-B and REDDIT-B
Class
Geuerated Examples
Class0 (Action)
Class (Romance)
Class0 (Question/
Answer)
Reddit-B
Class1 (Discussion)
Figure 9: Simpler Explanations of IMDB-Bine BonB-Binaryy and REDDIT-Binary
24

---

Published in Transactions on Machine Learning Research (10/2024)
Figure shows smaller sized explanations generated by Gtaphon-Explainee the IMDB-Binary and Reddit- Binary datasets.
J Comparative Results with other Metrics and Methods
In this section, compare the performance of Graphon-Explainer with GDM et al. 2024 on three datasets. We report class score averages for 10 explanations generated class on the dalasel whole as reported by the authors of GDM in Table 6 Further, we also adopt a metric proposed by the authors called model fidelity. Model fidelity demonstrates the usefulncss of the generated explanations by training a surrogate model on the explanations having a similar structure as the classifier and measuring the ratio where the surrogate model gives the same decision as the original classifier. As
of cases on the shown in Table 6 and Table (Graphon-Explannee performs better or equally on all except on one metric on the
one dataset. Figure 10 also shows the explanations and graphons generated by SHAPE dataset.
Table 6: Class Scores
Dataset
GDM
Graphon-Expae Explainer
MUTAG
0.8267 + 0.0470
1.0 + 0.00
BA2Motif
1.0 = 0.0000
1.0 + 0.00
SHAPE
1.0 + 0.0000
1.0 + 0.00
Table 7: Model Fidelity
Dataset
GDM
MUTAG
94.73 + 0.00
96.81 + 0.005
BA2Motif
98.00 + 0.00
96.19
0.034
SHAPE
84.00 + 8.00
92.41 + 0.143
Classes
Graphon
Example
Generated Explanations
Grid
Star
Wheel
Lollipop
Figure 10: Explanations on SHAPE Datasct generated by Graphon-Expie Explainer
25

---

Published in Transactions on Machine Learning Research (10/2024)
2Shapes
Reddit-B
4Shapes
Barbell Wheel
Barbell Star
Windmil Wheel
Windmill Star
MUTAG
BA2Motif
Figure 11: Visual Results of GNN Boundary
K Visual Results of GNNBoundary
Figure 11
demonstrates the motifs generated by GNNBoundary on all datasets for decision boundaries between different target classes.
L Sensitivity Analysis
L.1 Setting number of samples on the dataset
The following is the hyperparameter setting for Graphon-Explainee used on the corresponding datasets.
Table 8: Hyperparameter values of Dsamples on all datasets
Dataset
#samples for class explanation
#samples for boundary explanation
2Shapes
3
20
BA2Motif
10
50
4Shapes
800
IMDB-B
50
200
Reddit-B
10
150
MUTAG
20
100
L.2 Sensitivity Analysis on X and Number of Nodes
Figure 12 outlines the variation in class scores for the explanation graphs with the number of nodes for all the target classes and with varying mixup parameter on the boundary classes. On all the datasets for scores reported in Table / we choose K to be the median number of nodes in the datasct and X is varied from 0.4 to 0.6 for generation of the boundary motifs, and the yielding the best is chosen.
26

---

Published in Transactions on Machine Learning Research (10/2024)
Dataset
Lollipop
Tree
Decision Boundary
2Shapes
Cycle
House
Decision Boundary
BA2Motif
ClassO (Action)
Classl (Romance)
Decision Boundary
IMDB-B B
ClassO (Question/ Answer)
Class1 (Discussion)
Decision Boundary
Reddit-B
Mutagenic
Non- Non-Mutagenic
Decision Boundary
MUTAG
Barbell- Wheel Decision Boundary
Barbell-Star Decision Boundary
Barbell
Windmill
4Shapes
Windmill-Star Decision Boundary
Windmill- Wheel Decision Boundary
Wheel
Figure 12: Sensitivity Analysis on all Datasets
27


Reviews:
[{"review_id": "fTMybpEWwZ", "reviewer": "Reviewer_81Da", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\nS1: The method is intuitive and easy to follow.   \nS2: The results are promising.\n\nWeaknesses\n\nW1: The author suggests that \"multiple distinct $G^*$ may exist that maximize equation (2).\" However, they do not test this claim on publicly available multi-substructure datasets. (i.e. https://pytorch-geometric.readthedocs.io/en/latest/generated/torch_geometric.datasets.BAMultiShapesDataset.html#torch_geometric.datasets.BAMultiShapesDataset)\n\nW2: Other works in GNN global-explanability are not cited [1][2][3][4].\n\nW3: Upon reviewing the experimental results (Figure 3b IMDB and reddit), it remains unclear what specific insights or explanations can be derived from the generated subgraph.\n\nW4:It is unclear how node and edge features are handled by the graphon.\n\n\n\n[1]Nian, Yi, et al. \"Globally Interpretable Graph Learning via Distribution Matching.\" Proceedings of the ACM on Web Conference 2024. 2024. [2] Müller, Peter, et al. \"GraphChef: Decision-Tree Recipes to Explain Graph Neural Networks.\" The Twelfth International Conference on Learning Representations. 2024. [3] Azzolin, Steve, et al. \"Global Explainability of GNNs via Logic Combination of Learned Concepts.\" The Eleventh International Conference on Learning Representations. [4] Yu, Zhaoning, and Hongyang Gao. \"MAGE: Model-Level Graph Neural Networks Explanations via Motif-based Graph Generation.\" arXiv preprint arXiv:2405.12519 (2024).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors propose a novel method-level post-hoc explainer with a straightforward approach: learning a graphon for each target class. They conclude by evaluating their method on three real-world datasets and three synthetic ones.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The authors should compare their method's performance against other relevant works. Additionally, they should provide a more detailed discussion on the expected explanations in potential real-world applications. From an interpretability standpoint, it is difficult to discern meaningful insights from large graphs, such as those in Figure 3b (IMDB and Reddit).", "broader_impact_concerns": "Given the complexity of the extracted real-world explanations, I find it difficult to see the potential impact of the proposed method.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "qLH3fDKpUc", "reviewer": "Reviewer_1V5r", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n\n- [S1] The writing is very clearly presented, and I had no problem following the exposition.\n- [S2] Due to conceptual simplicity of the method, the authors are able to provide theoretical guarantees using graph-theoretical results.\n- [S3] The authors provide a thorough experimental comparison of the proposed and baseline methods.\n\n**Weaknesses**\n\n- [W1] There is a critical flaw in the proposed method, that in some scenarios, the method explains the data, not the model. For instance, suppose that there are two GNNs $f_1$ and $f_2$ which have perfect accuracy, but use different discriminative features to classify inputs. \n Graphon-Explainer, would yield identical class-conditional graphons for both $f_1$ and $f_2$ as they are learned using identical datasets. Thus, in scenarios where we would like to compare two or more highly accurate GNNs, Graphon-Explainer would not be really useful.\n- [W2] I found the description of the baselines to be slightly unclear. It would be beneficial to have a table comparing which methods need access to training data, GNN internal components, etc.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes Graphon-Explainer, which explains discriminative features of graph neural networks (GNNs) by learning a class-conditional generative model parametrized by graphons. Specifically, given a dataset of graphs and corresponding GNN classification labels, a graphon is learned to approximate the collection of graphs belonging to each class. Graphs near the GNN decision boundary can be generated by sampling from a linear interpolation of graphons of distinct classes. The authors verify that Graphon-Explainer outperform baselines GNNInterpreter and XGNN on a variety of datasets.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "**Critical**\n\n- [W1] : I am not sure what change would address [W1]. Unless my understanding of the method is wrong, the authors would have to modify the method such that it is able to explain discriminative features specific to individual GNNs. After making such a change to the method, the authors could demonstrate how the method is able to generate different explanations for different GNNs.\n\n**Not Critical**\n\n- [W2] : The authors could add a table, as mentioned in Strengths and Weaknesses.", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "zvazkLgoft", "reviewer": "Reviewer_wLSB", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "I find overall that the paper is well-written and most concepts are reasonably explained (see below for exceptions). The discussion of performed experiments is extensive, and the results are presented with many helpful illustrations.\n\nI also find that several important explanations regarding the graphon estimation are lacking. Besides that, it is not clear to me whether the obtained explanation actually entails interpretability of GNN. I expand on these topics in the next section of review.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Authors propose a method to study decision boundary and flat region of loss function minimum for a GNN-based classifier. Graph data is heterogeneous and inherently discrete, so classical gradient-based methods used in other domains (e.g., in CV see https://arxiv.org/abs/1412.1897 for discussion of flat region with small loss, see https://arxiv.org/abs/1312.6199 for the study of decision boundary) are not directly applicable. Authors overcome this issue by using techniques proposed in https://arxiv.org/abs/2202.07179. More specifically, they summarize statistics about graphs in each class with graphons and use them to study both regions with small loss and decision boundary (interpolation of learned features is also used in the later case). Authors perform a series of experiments with synthetic and real-world data. The results demonstrate that the developed method is superior to other recent interpretability techniques.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. What if the reader does not know what graphon is? Consider explaining this briefly in the abstract.\n2. GNNs can be used in many problems including, for example, the solution of PDEs https://arxiv.org/abs/2202.03376. The constructions discussed in the article are only applicable to the classification. On the other hand the title implies that the method can be used for general GNN. I kindly ask authors to resolve this issue by providing additional discussion about that in the abstrac or the main body of the paper.\n3. Graphons are further discussed in section 2 without a definition which appears in the next section. This makes it harder for the reader who did not know what graphon is to digest the content of the paper. I suggest authors explain graphons earlier.\n4. In section 4.1. the problem setup involves maximization over graph $G$. This is not a well-poised problem since authors do not provide the space of graphs $G$ belongs to. From later text, I can guess authors assume there is some unknown underlying distribution $p(G, C)$. A related conditional distribution $p(G|C)$ is approximated from data using graphon estimation (+ extension for feature estimation). Since authors optimize over samples from this estimated distribution, I assume in section 4.1 one should also specify that $G$ are drawn from $p(G|C)$.\n5. It is not clear how features are mixed and how graphons are estimated. I also consulted the G-Mixup paper https://arxiv.org/abs/2202.07179 which contains a similar vague description. Can the authors please provide a more detailed explanation on graphons and feature mixing? In particular, I have the following question:\n   1. Authors write “Algorithm 4 describes the method we use to approximate graphons.” Unfortunately, Algorithm 4 does not help since it contains “graphon estimator g” as an input. In essence Algorithm 4 is a suggestion to use “graphon estimator” to estimate graphon. Can the authors please explain clearly, preferably in the form of pseudocode or concrete mathematical expression, what one should do with sorted adjacency matrices?\n   2. Related to the previous question, the description in Section 4.3. is not helpful. Authors write “We use the USVT method (Chatterjee, 2015) to estimate the step function. This method aligns the nodes of the graph in each dataset based on certain node properties and then proceeds to estimate the step function from the aligned adjacency matrices.” From this sentence I can infer that to estimate step function I should estimate it from aligned adjacency matrices. How to do this is not detailed. Since graphons are central to the method advocated in the paper I think it is instructive to include this algorithm in the main text or appendix.\n   3. In a similar way it is not clear how features are generated. Authors provide a reference to https://arxiv.org/abs/2202.07179 where this part is explained in precisely the same language without clear mathematical or algorithmic description. Suppose I have a set of graphs $G_1, \\dots, G_N$ with node features $f_1, \\dots, f_N$. What should I do with node features to obtain features of graphs, generated from estimated graphon?\n   4. Graphon is a function [0, 1]^2 -> [0, 1] with certain properties. What do authors mean by “graphon nodes”? Functions do not have nodes. Perhaps, it is related to interpretation of graphon as a “graph with infinite number of nodes”. Can the authors please clarify this part?\n6. Section 4.5. that describes how to generate graphs near the decision boundary. For that authors first propose (for two selected classes) to extract learned features from the layer preceding the softmax function, mix these features and track predicted classes based on the mixing parameter. Based on that one can conclude whether two classes have a shared decision boundary. I have several question:\n   1. In this procedure one uses synthetic data obtained from graphons or the original data?\n   2. Authors write in the introduction “... it [Graphon-Explainer] does not need access to training data embeddings to generate explanations.”  Regardless of the answer to the previous question, Graphon-Explainer directly uses embeddings, so I suggest rephrasing or clarifying the sentence from the introduction.\n7. How the number of nodes is selected when synthetic graphs are generated with estimated graphon ($N$ in equation (4))?\n8. In the theoretical analysis authors focus on graph motifs. As I understand this section ignores the part that nodes of the graph have features. Can the features be included in the theoretical analysis?\n9. Authors claim generated graphs include discriminative motifs. This motif is available only indirectly since we know it is included in generated graphs with high classification score. Is there a way to directly find this motif?\n10. Is the generation of new samples necessary for finding a model-level explanation? If one selects samples from the dataset with a largest classification score they also should contain discriminative motifs. Can the authors report the difference between classification scores and densities for best samples from train/test sets and samples generated by their method? \n11. In the definition of boundary margins authors need to clarify what “graph belongs to class $c_1$” means. Since synthetic data is used is it not clear how this class is defined. If it is defined with the classifier itself, authors need to specify the decision rule used, e.g., the usual one is the class with largest predicted probability.\n12. In boundary thickness $G_b$ should probably be replaced with $G_{c_2}$.\n\nIn addition to all technical problems highlighted above, I would like to ask authors to provide general comments on the goals of the method. More specifically, it is not clear to me whether the search for “discriminative motif” helps in interpretability. One may argue that in all cases authors studied (table 3) the graph found for each class is not easier to analyze than any graph from the same class from the training set. How precisely generated graphs can help the user in the analysis of the classifier? This question is more or less clear for the decision boundary since there are quantitative measures of how complex the boundary is and how well classes are separated. For the case of individual classes. class score does not seem to be very informative and the density only refers to the complexity of the generated graph, so the quality of the method, not the classifier or the problem. Can the authors expand discussion on why the generated “discriminative motif” is useful?", "broader_impact_concerns": "na", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: All the reviewers agree that the paper is well written and includes extensive experiments with helpful illustrations. There are two concerns that were not addressed according to the reviewers:
1. Explanations provided for the larger graphs is unclear. It is helpful if authors can improve this part in the camera-ready version.
2.  Whether the explanations generated with the method is informative or useful for the user.  However, the reviewer also points out that this is a subjective point.
Decision: Accept (poster)