Title: Interpretable Prototype-based Graph Information Bottleneck
Venue: NeurIPS 2023
Accepted: yes
Keywords: Graph neural network

Explainable AI

Interpretability
Abstract: The success of Graph Neural Networks (GNNs) has led to a need for understanding their decision-making process and providing explanations for their predictions, which has given rise to explainable AI (XAI) that offers transparent explanations for black-box models. Recently, the use of prototypes has successfully improved the explainability of models by learning prototypes to imply training graphs that affect the prediction. However, these approaches tend to provide prototypes with excessive information from the entire graph, leading to the exclusion of key substructures or the inclusion of irrelevant substructures, which can limit both the interpretability and the performance of the model in downstream tasks. In this work, we propose a novel framework of explainable GNNs, called interpretable Prototype-based Graph Information Bottleneck (PGIB) that incorporates prototype learning within the information bottleneck framework to provide prototypes with the key subgraph from the input graph that is important for the model prediction. This is the first work that incorporates prototype learning into the process of identifying the key subgraphs that have a critical impact on the prediction performance. Extensive experiments, including qualitative analysis, demonstrate that PGIB outperforms state-of-the-art methods in terms of both prediction performance and explainability.

Interpretable Prototype-based Graph Information Bottleneck

Abstract

The success of Graph Neural Networks (GNNs) has led to a need for understanding their decision-making process and providing explanations for their predictions, which has given rise to explainable AI (XAI) that offers transparent explanations for black-box models. Recently, the use of prototypes has successfully improved the explainability of models by learning prototypes to imply training graphs that affect the prediction. However, these approaches tend to provide prototypes with excessive information from the entire graph, leading to the exclusion of key substructures or the inclusion of irrelevant substructures, which can limit both the interpretability and the performance of the model in downstream tasks. In this work, we propose a novel framework of explainable GNNs, called interpretable P rototype-based G raph I nformation B ottleneck (PGIB), that incorporates prototype learning within the information bottleneck framework to provide prototypes with the key subgraph from the input graph that is important for the model prediction. This is the first work that incorporates prototype learning into the process of identifying the key subgraphs that have a critical impact on the prediction performance. Extensive experiments, including qualitative analysis, demonstrate that PGIB outperforms state-of-the-art methods in terms of both prediction performance and explainability. The source code of PGIB is available at https://github.com/sang-woo-seo/PGIB .

1 Introduction

With the success of Graph Neural Networks (GNNs) in a wide range of deep learning tasks, there has been an increasing demand for exploring the decision-making process of these models and providing explanations for their predictions. To address this demand, explainable AI (XAI) has emerged as a way to understand black-box models by providing transparent explanations for their predictions. This approach can improve the credibility of models and ensure transparency in their decision-making processes. As a result, XAI is actively being used in various applications, such as medical, finance, security, and chemistry [3, 26].

In general, explainability can be viewed from two perspectives: 1) improving the interpretability by providing explanations for the model's predictions , and 2) providing the reasoning process behind the model prediction by giving explanations for the model's training process . Improving the interpretability in GNNs involves detecting important substructures during the inference phase, which is useful for tasks such as identifying functional groups (i.e., important substructures) in molecular chemistry [29, 33, 23, 12]. On the other hand, it is also important to provide the reasoning process for why the model predicts in a certain way, which requires an in-depth analysis of the training phase, so as to understand the model in a more fundamental level. Through this reasoning process, we can visualize and analyze how the model makes correct or incorrect decisions, thus obtaining crucial information for improving its performance.

(a) ProtGNN

(b) PGIB

𝓖

𝒑

𝟏

𝓖

𝒑

𝟐

𝓖

𝒔𝒖𝒃

Prototype graphs,

𝒢

#

Input graph,

𝒢

𝓖

𝒑

𝟏

𝓖

𝒑

𝟐

𝑁𝑂

$

Prototype graphs,

𝒢

#

Input graph,

𝒢

In recent years, there has been a growing interest in exploring the reasoning process to provide greater transparency and explainability in deep learning models. These approaches can be generally classified into two main categories: 1) post-hoc approaches, and 2) built-in approaches. Post-hoc approaches focus on exploring the model outputs by visualizing the degree of activation of neurons through measuring their contribution based on gradients of the model predictions. For instance, recent works [14, 9, 34] use techniques such as saliency maps and class activation maps to visualize the activated areas during the model's prediction process. However, these approaches require a separate explanatory model for each trained model, resulting in the need for a new explanatory model for additional training data and different models [2, 15]. In order to address the aforementioned challenges, built-in approaches aim to integrate the generation of explanations into the model training process. One such approach is prototype learning, which involves learning prototypes that represent each class of the input data, which are then compared with new instances to make predictions. ProtGNN [35], for instance, measures the similarity between the embedding of an input graph and each prototype, providing explanations through the similarity calculation and making predictions for the input graph based on its similarity with the learned prototypes. More precisely, ProtGNN projects each learned prototype onto the closest training graph, enabling it to provide explanations of primary structures for its prediction.

However, since ProtGNN compares the graph-level embedding of an input graph with the learned prototypes, the model overlooks the key substructures in the input graph while also potentially including uninformative substructures. This not only results in a degradation of the interpretability of the reasoning process, but also limits the performance on the downstream tasks. Figure 1(a) shows the prototype graphs in the training set (i.e., G p , denoted as bold edges) detected by ProtGNN for an input molecule (i.e., G ) that belongs to the 'mutagenic' class. Despite the NO 2 structure being the key functional group for classifying a given molecule as 'mutagenic,' G p detected by ProtGNN tends to include numerous ring structures (i.e., uninformative substructures) that are commonly found throughout the input graph, and exclude NO 2 structures (i.e., key substructures) in learned prototype graphs, which is mainly due to the fact that the input graph G is considered in the whole graph-level. As a result, it is crucial to identify a key subgraph within the input graph that holds essential information for the learning of prototypes, which in turn enhances both the explanation of the reasoning process and the performance on the downstream tasks. Among the various solutions for detecting important subgraphs, the Information Bottleneck (IB) has emerged as one of the most effective methods [27, 31], and it has been demonstrated that key subgraphs detected based on IB can contribute to performance improvement in various tasks such as relational learning [11] and structure learning [19]. We aim to approach the IB principle from the perspective of prototypes to convey important substructure information to the prototypes.

To this end, we propose a novel framework of explainable GNNs, called Interpretable P rototype-based G raph I nformation B ottleneck (PGIB). The main idea is to incorporate prototype learning within the Information Bottleneck (IB) framework, which enables the prototypes to capture the essential key subgraph of the input graph detected by the IB framework. Specifically, PGIB involves prototypes (i.e., G p ) in a process that maximizes the mutual information between the learnable key subgraph (i.e., G sub ) of the input graph (i.e., G ) and target information (i.e., Y ), which allows the prototypes to interact with the subgraph. This enables the learning of prototypes G p based on the key subgraph G sub within the input graph G , leading to a more precise explanation of the reasoning process and improvement in the performance on the downstream tasks. To the best of our knowledge, this is the first work that combines the process of optimizing the reasoning process and interpretability by identifying the key subgraphs that have a critical impact on the prediction performance. In Figure 1(b), PGIB is shown to successfully detect the key subgraph G sub that includes NO 2 from the input graph G , even when the ring structures are dominant in G . It is important to note that PGIB is highly efficient in detecting G sub from G since PGIB adopts a learnable masking technique, effectively resolving the time complexity issue. Last but not least, since the number of prototypes for each

class is determined before training, some of the learned prototypes may share similar semantics, which negatively affects the model interpretability for which the small size and low complexity are desirable [6, 25, 17]. Hence, we propose a method for effectively merging the prototypes, which in turn contributes to enhancing both the explanation of the reasoning process and the performance on the downstream tasks.

We conducted extensive experiments to evaluate the effectiveness and interpretability of the reasoning process of PGIB in graph classification tasks. Our results show that PGIB outperforms recent stateof-the-art methods, including existing prototype learning-based and IB-based methods. Moreover, we evaluated the ability of PGIB in capturing the label information by evaluating the classification performance using only the detected subgraph G sub . We also conducted a qualitative analysis that visualizes the subgraph G sub and prototype graph G p , suggesting the ability of PGIB in detecting the key subgraph. Overall, our results show that PGIB significantly improves the interpretability of both G sub and G p in the reasoning process, while simultaneously improving the performance in downstream tasks.

In summary, our main contributions can be summarized as follows: 1) We propose an effective approach, PGIB, that not only improves the interpretability of the reasoning process, but also the overall performance in downstream tasks by incorporating the prototype learning in a process of detecting key subgraphs based on the IB framework. 2) We provide theoretical background with our method that utilizes interpretable prototypes in the process of optimizing G sub . 3) Extensive experiments, including qualitative analysis, demonstrate that PGIB outperforms state-of-the-art methods in terms of both prediction performance and explainability.

2 Preliminaries

In this section, we introduce notations used throughout the paper followed by the definitions of IB and IB-Graph.

Notations. We use G = ( V , E , A , X ) to denote a graph, where V , E , A and X denote the set of nodes and edges, the adjacency matrix and node features, respectively. We assume that each node v i ∈ V is associated with a feature vector x i , which is the i -th row of X . We use { ( G 1 , y 1 ) , ( G 2 , y 2 ) , · · · , ( G N , y N ) } to denote the set of N graphs with its corresponding labels. The graph labels are given by a set of K classes C = { 1 , 2 , . . . , K } , and the ground truth label of a graph G i is denoted by y i ∈ C . We use G sub to denote a subgraph of G , and use ¯ G sub to denote the complementary structure of G sub in G . We also introduce the prototype layer, which consists of a set of prototypes Z p = { z 1 G p , z 2 G p , · · · , z M G p } , where M is the total number of prototypes, and each prototype z m G p is a learnable parameter vector that serves as the latent representation of the prototypical part (i.e., G p ) of graph G . We allocate J prototypes for each class, i.e., M = K × J .

Graph Information Bottleneck. The mutual information between two random variables X and Y , i.e., I ( X ; Y ) , is defined as follows:

Given the input X and its associated label Y , the Information Bottleneck (IB) [21] aims to obtain a bottleneck variable Z by optimizing the following objective:

where β is the Lagrange multiplier used to control the trade-off between the two terms. IB principle has recently been applied to learning a bottleneck graph, named IB-Graph, for a given graph G , which retains the minimal sufficient information in terms of G 's properties [31]. This approach is motivated by the Graph Information Bottleneck (GIB) principle, which seeks to identify an informative yet compressed subgraph G sub from the original graph G by optimizing the following objective:

where Y is the label of G . The first term maximizes the mutual information between the graph label and the compressed subgraph, which ensures that the compressed subgraph contains as much information as possible about the graph label. The second term minimizes the mutual information between the input graph and the compressed subgraph, which ensures that the compressed subgraph contains minimal information about the input graph.

⋮

FC-Layer

Linear

Softmax

Output

Readout

𝒁𝒢𝑠𝑢𝑏

𝑝1, 𝑝

2 , … , 𝑝𝑛

𝑧𝑖

= 𝜆𝑖 ℎ

𝑖

+(1 - 𝜆𝑖)𝜖

𝜆1, 𝜆

2 , … , 𝜆𝑛

GNN

𝑓 𝑔

MLP

MLP

Sigmoid

Selected

Noised

Input graph,

𝒢

Prototype Layer

⋮

𝒁𝒢𝑝

1

𝒁𝒢𝑝

2

𝒁𝒢𝑝

3

𝒁𝒢𝑝

4

𝒁𝒢𝑠𝑢𝑏

𝒁𝒢𝑠𝑢𝑏

𝒁𝒢𝑠𝑢𝑏

𝒁𝒢𝑠𝑢𝑏

𝒁𝒢𝑠𝑢𝑏

𝒁𝒢𝑝

𝑀

𝑔

⋮

(

)

,

(

)

,

(

)

,

(

)

,

(

)

,

Prototypes

Merge

𝒁𝒢𝑠𝑢𝑏

⊕

3 Methodology

In this section, we present our proposed method, called PGIB. We introduce Prototype-based Graph Information Bottleneck (Section 3.1), each layer in the architecture (Section 3.2 - 3.4), and the interpretability stabilization process (Section 3.5), which enhances the interpretability and the tracking capabilities of the reasoning process during the model training.

Model Architecture. Figure 2 presents an overview of PGIB. We first generate the representations of nodes in the input graph G using a GNN encoder. Then, the node representations are passed to the subgraph extraction layer that assigns each node in G to either G sub or ¯ G sub . Next, we compute the similarities between the embedding z G sub and the set of prototypes Z p = { z 1 G p , z 2 G p , · · · , z M G p } in the prototype layer. Finally, we merge the prototypes that are semantically similar, which are then used to generate the final prediction.

3.1 Prototype-based Graph Information Bottleneck

PGIB is a novel explainable GNN framework that incorporates the prototype learning within the IB framework, thereby enabling the prototypes to capture the essential key subgraph of the input graph detected by the IB framework. More precisely, we reformulate the GIB objective shown in Equation 3 by decomposing the first term, i.e., I ( Y ; G sub ) , with respect to the prototype G p using the chain rule of mutual information in order to examine the impact of the joint information between G sub and G p on Y as follows:

Please refer to Appendix A.1 for a detailed proof of Equation 4. In the following sections, we describe how each term is optimized during training.

3.2 Subgraph Extraction Layer (Minimizing I ( G ; G sub ) )

The goal of the subgraph extraction layer is to extract an informative subgraph G sub from G that contains minimal information about G . We minimize I ( G ; G sub ) by training the model to inject noise into insignificant subgraphs ¯ G sub , while injecting less noise into more informative ones G sub [32]. Specifically, given the representation of node v i , i.e., h i , we compute the probability p i with an MLP followed by a sigmoid function, which is then used to replace the representation h i to obtain the final representation z i as follows:

That is, the learned probability p i enables selective preservation of information in G sub , and based on this probability, the quantity of information transmitted from h i to z i can be flexibly adjusted to compress the information from G to G sub . This approach not only retains interpretability within the

subgraph itself, but also potentially facilitates the learning of prototypes that are introduced in the next step. Following [32], we minimize the upper bound of I ( G ; G sub ) as follows:

where A = ∑ |V G | i =1 (1 -λ i ) 2 and B = ∑ |V G | i =1 λ i ( h i -µ h i ) σ h i . Thus, minimizing L 1 MI allows us to minimize the upper bound of I ( G ; G sub ) . After noise injection, we compute the embedding z G sub through a graph readout function such as max pooling and sum pooling. For further details and analysis of the different graph readout functions, please refer to Appendix A.4.2.

3.3 Prototype Layer (Minimizing -I ( Y ; G sub , G p ) + I ( Y ; G p |G sub ) )

The prototype layer involves allocation of a fixed number of prototypes for each class. The prototypes are required to capture the most significant graph patterns that can aid in the identification of the graphs within each class. To begin with, we define the similarity score between the prototype z G p and the embedding z G sub obtained from noise injection as follows:

where z G p is the prototype and shares the same dimension as z G sub .

3.3.1 Minimizing -I ( Y ; G sub , G p )

We derive the lower bound of I ( Y ; G sub , G p ) as follows:

Proposition 1 . (Lower bound of I ( Y ; G sub , G p ) ) Given significant subgraph G sub for a graph G , its label information Y , prototype graph G p and similarity function γ , we have

where q θ ( Y | γ ( G sub , G p )) is the variational approximation to the true posterior p ( Y | γ ( G sub , G p )) .

Equation 8 demonstrates that the maximization of the mutual information I ( Y ; G sub , G p ) can be attained by minimizing the classification loss, denoted as L cls . This maximization of mutual information between the label Y and the similarity information γ ( G sub , G p ) promotes the subgraph G sub and prototype G p to possess predictive capabilities concerning the graph label Y . In practical applications, the cross-entropy loss is chosen for a categorical Y . For a comprehensive understanding of the derivation process of Equation 8, refer to the Appendix A.2. Note that the similarity between G sub and G p , i.e., γ ( G sub , G p ) , is computed by the similarity score defined in Equation 7, i.e., g ( z G sub , z G p ) .

3.3.2 Minimizing I ( Y ; G p |G sub )

We investigate the mutual information, denoted as I ( Y ; G p |G sub ) , from the perspective of the interaction between G sub and G p . We decompose I ( Y ; G p |G sub ) into the sum of two terms based on the chain rule of mutual information as follows:

It is important to note that the first term, i.e., I ( G p ; Y, G sub ) , minimizes the mutual information between G p and the joint variables ( Y , G sub ) , which eliminates the information about Y related to G sub from G p . However, since our goal is not to solely minimize I ( G p ; Y, G sub ) but to ensure the interpretability of the prototype G p , including this term leads to diminished interpretability of G p . Consequently, we excluded the first term during training, and only consider the second term, i.e., -I ( G sub ; G p ) , to simultaneously guarantee the interpretability of both G sub and G p . A detailed derivation for Equation 9 is given in Appendix A.3. From now on, we describe approaches for minimizing the second term. Inspired by [11], we introduce two different approaches for minimizing -I ( G sub ; G p ) .

1) Variational IB-based approach. Weobtain the upper bound of -I ( G sub ; G p ) using the variational IB-based approach as follows:

where q ϕ ( G p |G sub ) is the variation approximation of p ( G p |G sub ) . Equation 10 shows that the maximization of the mutual information I ( G sub ; G p ) can be attained by minimizing L 2 MI ( G sub , G p ) . We select a single-layer linear transformation as a modeling option for q ϕ to minimize the information loss of G sub when predicting G p .

2) Contrastive learning-based approach. Recent studies on contrastive learning [20, 30, 7] have proven that minimizing contrastive loss is equivalent to maximizing the mutual information between two variables. Hence, we additionally propose a variant of PGIB, i.e., PGIB cont , that minimizes the contrastive loss instead of minimizing the lower bound as defined in Equation 10. More precisely, we consider G p and G sub with the same label as a positive pair, and the contrastive loss is defined as follows:

where τ is the temperature hyperparameter, n denotes the number of graphs in a batch, and j and k indicate indices of positive and negative samples, respectively. P y i is the set of prototypes that belong to class y i . We effectively confer interpretability to G p by increasing its similarity with each G sub .

3.4 Prediction Layer

We obtain the set of similarity scores r ∈ R M , whose m -th element r m = g ( z G sub , z m G p ) denotes the similarity score between z G sub and z G p as defined in Equation 7. Then, we compute the final predicted probability π ∈ R K by passing r and z G sub through a linear layer with weights ω , followed by the softmax function. Specifically, ω [ m, :] denotes weight assigned to r m , i.e., ω ( z G p m ) . Finally, we calculate the cross-entropy classification loss, as follows:

3.5 Interpretability Stabilization

Merging Prototypes . Since the number of prototypes for each class is determined before training, some of the learned prototypes may share similar semantics, which negatively affects the model interpretability for which the small size and low complexity are desirable [6, 25]. Inspired by [17], we propose a method to effectively merge the prototypes for graph-structured data, which, in turn, enhances the explanation of the reasoning process and improves performance on downstream tasks while reducing model complexity. The main idea is to merge prototypes based on the similarity between prototype pairs using the embeddings z G sub . This similarity utilizes all training subgraphs (i.e., ⋃ G∈X G sub , where X is the training set) by measuring the disparity between g ( z G sub , z G i p ) and

Then, for every pair ( z G i p , z G j p ) that falls within the highest ξ percent of similar pairs, the prototype z G j p and its corresponding weights ω ( z G j p ) are removed, and the weights ω ( z G i p ) are updated to the sum of ω ( z G i p ) and ω ( z G j p ) . We combine the ξ percentage of the most similar prototype pairs based on the calculated similarity scores.

Prototype Projection . Since the learned prototypes are embedding vectors that cannot be directly interpreted, we project each prototype z G p onto the nearest latent training subgraph from the same class. This process establishes a conceptual equivalence between each prototype and a training subgraph, thereby enhancing interpretability of the prototypes. Specifically, we update prototype z G p of class k ( i.e., z G p ∈ P k ) by performing the following operation:

In the Equation 14, we use Monte Carlo Tree Search (MCTS) [18] to explore training subgraphs ˜ G during prototype projection.

Connectivity Loss. For an input graph G , we construct a node assignment S G ∈ R |V G |× 2 based on the probability values that are computed by Equation 5. Specifically, S G [ j, 0] and S G [ j, 1] denote the probability of node v j ∈ V G belonging to G sub and ¯ G sub , respectively. Following [31], poor initialization of the matrix S may result in the proximity of its elements S [ j, 0] and S [ j, 1] for ∀ v j ∈ V G i , leading to an unstable connectivity of G sub . This instability can have adverse effects on the subgraph generation process. To enhance the interpretability of G sub by inducing a compact topology, we utilize a batch-wise loss function as follows:

where S B ∈ R n ∑ i =1 |V G i |× 2 and A B ∈ R n ∑ i =1 |V G i |× n ∑ i =1 |V G i | are the node assignment and the adjacency matrix at the batch level, respectively. I 2 is 2 -by2 identity matrix, ∥ · ∥ F is the Frobenius norm and Norm( · ) is the row normalization. Minimizing L con indicates that if v j is in G sub its neighbors also have a high probability to be in G sub , while if v i is in G sub , its neighbors have a low probability to be in ¯ G sub .

Final Objectives . Finally, we define the objective of our model as the sum of the losses as follows:

where α 1 , α 2 and α 3 are hyper-parameters that adjust the weights of the losses. A detailed ablation study for each loss term is provided in Appendix A.4.1 for further analysis.

4 Experiments

4.1 Experimental Settings

Each dataset is split into training, validation, and test sets with a ratio of 80% , 10% , and 10% , respectively. All models are trained for 300 epochs using the Adam optimizer with a learning rate of 0.005. GIN [28] is used as the encoder for all models used in the experiment. We evaluate the performance based on accuracy, which is averaged over 10 independent runs with different random seeds. For simplicity, the hyperparameters α 1 , α 2 , and α 3 in Equation 16 are set to 0 . 0001 , 0 . 01 to 0 . 1 and 5 , respectively. The prototype merge operation starts at epoch 100 and is performed every 50 epochs thereafter. We set the number of prototypes per class to 7 and combine 30% of the most similar prototype pairs.

4.2 Graph Classification

Datasets and Baselines . We use the MUTAG [16], PROTEINS [1], NCI1 [24], and DD[5] datasets. These are datasets related to molecules or bioinformatics, and are widely used for evaluations on graph classification. We consider three GNN baselines, including GCN [10], GIN [28], GAT [22]. In addition, we compare PGIB with several state-of-the-art built-in models that integrate explanation functionality internally, including a prototype-based method ProtGNN [35], and IB-based models such as GIB [31], VGIB [32], and GSAT [13]. Further details about the baselines and datasets are provided in Appendix A.5 and A.6, respectively.

Experiment Results . Experimental results for graph classification are presented in the Table 1. In the table, PGIB and PGIB cont represent our proposed methods. PGIB utilizes a Variational IB-based approach, while PGIB cont employs a Contrastive learning-based approach to maximize I ( G sub ; G p ) (Section 3.3.2). We have the following observations: 1) All variants of PGIB outperform the baselines including both the prototype-based and IB-based methods on all datasets. Notably, PGIBs incorporate the crucial information of the key subgraph, which significantly contributes to enhancing the classification performance. PGIB cont achieves a significant improvement of up to 5.6% compared to the runner-up baseline. 2) We observe that PGIB cont performs relatively better than PGIB. We attribute this to the nature of the contrastive loss, which is generally shown to be effective in classifying instances between different classes, allowing the prototypes learned based on the contrastive loss to be more distinguishable from one another.

4.3 Graph Interpretation

In this section, we evaluate the process of extracting subgraphs that possess the most similar properties of the original graph. We present qualitative results including subgraph visualizations, and conduct quantitative experiments to measure how accurately explanations capture the important components that contribute to the model's predictions.

Datasets and Baselines . We use four molecular properties from the ZINC [8] dataset, which consists of 250,000 molecules, for graph interpretation. QED measures the likelihood of a molecule being a drug and DRD2 indicates the probability of a molecule being active on dopamine type 2 receptors. HLM-CLint and RLM represent estimates of in vitro human and rat liver microsomal metabolic stability (mL/min/g as base 10 logarithm). We compare PGIBcont with several representative interpretation models, including GNNexplainer [29], PGexplainer [12], GIB [31], and VGIB [32]. Note that as PGIB shows similar results with PGIBcont, we only report the results of PGIBcont for simplicity. Further details on baselines and datasets are described in Appendix A.5 and A.6, respectively.

Qualitative Analysis . Figure 3(a)-(d) present the visualization of subgraphs in Mutag dataset. According to [29, 4], the NO 2 functional group is known to be a cause of mutagenicity, while the carbon ring is a substructure that is not related to mutagenicity. In the figure, the bold edges connect the nodes that the models consider important. The NO 2 group in Mutag is correctly identified by PGIB, while VGIB, ProtGNN, and GNNexplainer fail to recognize all NO 2 groups or include other unnecessary substructures. Figure 3(e)-(h) present the visualization of subgraphs in BA-2Motifs dataset. We observe that PGIB accurately recognizes motif graphs containing the label information such as a house or a five-node cycle, but other models have difficulty in detecting the complete motifs.

Quantitative Analysis . Although visualizing the explanations generated by models plays a crucial role in assessing various explanation models, relying solely on qualitative evaluations may not always be reliable due to their subjective nature. Therefore, we also perform quantitative experiments using the Fidelity metric [14, 34].

The Fidelity metric quantifies the extent to which explanations accurately capture the important components that contribute to the model's predictions. Specifically, let y i and ˆ y i denote the groundtruth and predicted values for the i -th input graph, respectively. Moreover, k denotes the sparsity score of the selected subgraph in which nodes whose importance scores obtained by Equation 5 are among the top-( k × 100) %within the original graph, and its prediction is denoted by ˆ y k i . Additionally, ˆ y 1 -k i denotes the prediction based on the remaining subgraph. The Fidelity scores are computed as follows:

where I ( y i = ˆ y i ) is the binary indicator which returns 1 if y i = ˆ y i , and 0 otherwise. In other words, they measure how well the predictions made solely based on the extracted subgraph (i.e., F -) and the remaining subgraph (i.e., F + ) mimic the predictions made based on the entire graph, respectively. Hence, a low value of F -and a high value of F -indicate better explainability of the model.

Table 2 shows the fidelity scores on four datasets at the sparsity score of k = 0 . 5 . Our proposed model outperforms both post-hoc and built-in state-of-theart explanation models in all datasets. Furthermore,

merging prototypes achieves significant improvements in terms of interpretability. This implies

(a) PGIB

cont

(b)VGIB

(c)ProtGNN

(d)GNNexplainer

(e)PGIB

cont

(f)VGIB

(g)ProtGNN

(h)GNNexplainer

PGIB cont

ℱ

!

ℱ

"

Spar

s

ity

(

k

)

Spar

s

ity

(

k

)

Spar

s

ity

(

k

)

Spar

s

ity

(

k

)

(a)

𝛼

!

= 0.0001

(b)

𝛼

!

= 1

𝑁𝑂

!

𝒢

"

!

𝒢

"

"

𝒢

"

#

𝒢

"

!

𝒢

"

"

𝒢

"

#

that decreasing the number of prototypes can eliminate uninformative substructures and emphasize key substructures, which increases the interpretability of the extracted subgraphs. Figure 4 visualizes the comparison of fidelity scores over various sparsity scores of subgraphs. To ensure a fair comparison, the fidelity scores are compared under the same subgraph sparsity, as the difference between the predictions of the original graph and subgraph strongly depends on the level of sparsity. We observe that PGIBcont achieves the best performance in most sparsity environments on the four datasets.

4.4 Hyperparameter Analysis

In Figure 6, we conduct a sensitivity analysis on the hyperparameters α 1 and α 2 of the final loss (Equation 16) relevant to mutual information. Note that α 1 and α 2 are related to minimizing I ( G ; G sub ) and maximizing I ( G sub ; G p ) , respectively. 1) Figure 6(a) shows a significant decrease in performance when α 1 becomes large, i.e., when the model focuses on compressing the subgraphs. This is because too much compression of

a

𝛼

!

b

𝛼

"

subgraphs results in the loss of important information, ultimately having a negative impact on the downstream performance. However, when α 1 = 0 (i.e., G = G sub ; no compression at all), uninformative information would be included in G sub , which incurs a performance degradation. 2) Figure 6(b) visualizes the change in performance depending on α 2 . A small value of α 2 prevents sufficient transmission of information from G sub to G p , whereas excessive value of α 2 allows the influence of G sub to dominate the prototypes G p , both of which lead to a performance deterioration. For example, in Mutag dataset, a low value of α 2 ultimately results in G p not obtaining label-relevant information (i.e., NO 2 ) that is captured by G sub (See Figure 5(a)). On the other hand, a high value of α 2 hinders the formation of diverse prototypes (See Figure 5(b)).

5 Conclusion and Future work

We propose a novel framework of explainable GNNs, called interpretable Prototype-based Graph Information Bottleneck (PGIB), that integrates prototype learning into the information bottleneck framework. The main idea of PGIB is to learn prototypes that capture subgraphs containing key structures relevant to the label information, and to merge the semantically similar prototypes for better model interpretability and model complexity. Experimental results show that PGIB achieves improvements not only in the performance on downstream tasks, but also provides more precise

explanation of the reasoning process. For future work, we plan to further extend the applicability of PGIB by integrating domain knowledge into prototype learning by imposing constraints on subgraphs. We expect that this approach would enable the learning of prototypes that align with the domain knowledge as they obtain domain-specific information from the subgraphs.

Acknowledgments and Disclosure of Funding

This work was supported by Institute of Information & Communications Technology Planning & Evaluation(IITP) grant funded by the Korea government(MSIT) (RS-2023-00216011, No.2022-000077), and the National Research Foundation of Korea(NRF) funded by Ministry of Science and ICT (NRF-2022M3J6A1063021).

References

K. M. Borgwardt, C. S. Ong, S. Schönauer, S. Vishwanathan, A. J. Smola, and H.-P. Kriegel. Protein function prediction via graph kernels. Bioinformatics , 21(suppl_1):i47-i56, 2005.

C. Chen, O. Li, D. Tao, A. Barnett, C. Rudin, and J. K. Su. This looks like that: deep learning for interpretable image recognition. Advances in neural information processing systems , 32, 2019.

M. Cranmer, A. Sanchez Gonzalez, P. Battaglia, R. Xu, K. Cranmer, D. Spergel, and S. Ho. Discovering symbolic models from deep learning with inductive biases. In Advances in Neural Information Processing Systems , pages 17429-17442, 2020.

A. K. Debnath, R. L. Lopez de Compadre, G. Debnath, A. J. Shusterman, and C. Hansch. Structure-activity relationship of mutagenic aromatic and heteroaromatic nitro compounds. correlation with molecular orbital energies and hydrophobicity. Journal of medicinal chemistry , 34(2):786-797, 1991.

P. D. Dobson and A. J. Doig. Distinguishing enzyme structures from non-enzymes without alignments. Journal of molecular biology , 330(4):771-783, 2003.

F. Doshi-Velez and B. Kim. A roadmap for a rigorous science of interpretability. arXiv preprint arXiv:1702.08608 , 2(1), 2017.

R. D. Hjelm, A. Fedorov, S. Lavoie-Marchildon, K. Grewal, P. Bachman, A. Trischler, and Y. Bengio. Learning deep representations by mutual information estimation and maximization. arXiv preprint arXiv:1808.06670 , 2018.

J. J. Irwin and B. K. Shoichet. Zinc- a free database of commercially available compounds for virtual screening. Journal of chemical information and modeling , 45(1):177-182, 2005.

G. Jaume, P. Pati, B. Bozorgtabar, A. Foncubierta, A. M. Anniciello, F. Feroce, T. Rau, J.P. Thiran, M. Gabrani, and O. Goksel. Quantifying explainers of graph neural networks in computational pathology. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 8106-8116, 2021.

T. N. Kipf and M. Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 , 2016.

N. Lee, D. Hyun, G. S. Na, S. Kim, J. Lee, and C. Park. Conditional graph information bottleneck for molecular relational learning. arXiv preprint arXiv:2305.01520 , 2023.

D. Luo, W. Cheng, D. Xu, W. Yu, B. Zong, H. Chen, and X. Zhang. Parameterized explainer for graph neural network. In Advances in Neural Information Processing Systems , pages 19620-19631, 2020.

S. Miao, M. Liu, and P. Li. Interpretable and generalizable graph learning via stochastic attention mechanism. In International Conference on Machine Learning , pages 15524-15543. PMLR, 2022.

P. E. Pope, S. Kolouri, M. Rostami, C. E. Martin, and H. Hoffmann. Explainability methods for graph convolutional neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 10772-10781, 2019.

C. Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature machine intelligence , 1(5):206-215, 2019.

M. Rupp, A. Tkatchenko, K.-R. Müller, and O. A. Von Lilienfeld. Fast and accurate modeling of molecular atomization energies with machine learning. Physical review letters , 108(5):058301, 2012.

D. Rymarczyk, Ł. Struski, J. Tabor, and B. Zieli´ nski. Protopshare: Prototypical parts sharing for similarity discovery in interpretable image classification. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining , pages 1420-1430, 2021.

D. Silver, J. Schrittwieser, K. Simonyan, I. Antonoglou, A. Huang, A. Guez, T. Hubert, L. Baker, M. Lai, A. Bolton, et al. Mastering the game of go without human knowledge. nature , 550 (7676):354-359, 2017.

Q. Sun, J. Li, H. Peng, J. Wu, X. Fu, C. Ji, and S. Y. Philip. Graph structure learning with variational information bottleneck. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 36, pages 4165-4174, 2022.

Y. Tian, D. Krishnan, and P. Isola. Contrastive multiview coding. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XI 16 , pages 776-794. Springer, 2020.

N. Tishby, F. C. Pereira, and W. Bialek. The information bottleneck method. arXiv preprint physics/0004057 , 2000.

P. Veliˇ ckovi´ c, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y. Bengio. Graph attention networks. arXiv preprint arXiv:1710.10903 , 2017.

M. Vu and M. T. Thai. Pgm-explainer: Probabilistic graphical model explanations for graph neural networks. In Advances in Neural Information Processing Systems , pages 12225-12235, 2020.

N. Wale, I. A. Watson, and G. Karypis. Comparison of descriptor spaces for chemical compound retrieval and classification. Knowledge and Information Systems , 14:347-375, 2008.

T. Wang. Gaining free or low-cost interpretability with interpretable partial substitute. In International Conference on Machine Learning , pages 6505-6514. PMLR, 2019.

J. Wencel-Delord and F. Glorius. C-h bond activation enables the rapid construction and late-stage diversification of functional molecules. Nature chemistry , 5(5):369-375, 2013.

T. Wu, H. Ren, P. Li, and J. Leskovec. Graph information bottleneck. Advances in Neural Information Processing Systems , 33:20437-20448, 2020.

K. Xu, W. Hu, J. Leskovec, and S. Jegelka. How powerful are graph neural networks? arXiv preprint arXiv:1810.00826 , 2018.

Z. Ying, D. Bourgeois, J. You, M. Zitnik, and J. Leskovec. Gnnexplainer: Generating explanations for graph neural networks. In Advances in Neural Information Processing Systems , pages

9240-9251, 2019.

Y. You, T. Chen, Y. Sui, T. Chen, Z. Wang, and Y. Shen. Graph contrastive learning with augmentations. Advances in neural information processing systems , 33:5812-5823, 2020.

J. Yu, T. Xu, Y . Rong, Y . Bian, J. Huang, and R. He. Graph information bottleneck for subgraph recognition. In International Conference on Learning Representations , 2021.

J. Yu, J. Cao, and R. He. Improving subgraph recognition with variational graph information bottleneck. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 19396-19405, 2022.

H. Yuan, J. Tang, X. Hu, and S. Ji. Xgnn: Towards model-level explanations of graph neural networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , pages 430-438, 2020.

H. Yuan, H. Yu, S. Gui, and S. Ji. Explainability in graph neural networks: A taxonomic survey. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2022.

Z. Zhang, Q. Liu, H. Wang, C. Lu, and C. Lee. Protgnn: Towards self-explaining graph neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 36, pages 9127-9135, 2022.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a new GNN explanation framework, which combines prototype learning and the information bottleneck. They define the Prototype-based Graph Information Bottleneck framework in detail, and experiments have demonstrated the effectiveness of their proposed framework.\n\nSTRENGTHS:\n1.The authors introduce the Graph Information Bottleneck based on prototype learning, which makes a significant improvement over the baseline methods.\n\n2.The experiments are comprehensive and there are many additional experiments in the appendix.\n\n3.The paper is well written.\n\nWEAKNESSES:\n1.I argue that the novelty of the paper is limited by the fact that both the IB approach and the prototype approach have been well discussed before, and the authors seem to have simply combined the two methods together.\n\n2.The number of prototypes per class is an important hyperparameter and the author should add a hyperparameter analysis of it.\n\n3.I am not quite sure about the differences between PGIB and ProtGNN. If I use the GIB strategy in ProtGNN, what effect will it have?\n\n4.In this paper, authors use GIN as a backbone. However, PGIB as a model-agnostic approach should be validated on more GNN structures, e.g. GCN, GraphSage.\n\n\nOthers:\n\n1.Line 143 \"PGIB s a novel \" ->\"PGIB is a novel \"\n\n2.Line 275 \"This is because When\" -> \"This is because when\"\n\nQUESTIONS:\n1.I wonder if the authors' proposed approach can alleviate the OOD problem which is widely discussed in the explanation GNNs[1,2].\n\n2.I notice that the authors obtain two types of subgraphs, one in the Subgraph Extraction Layer and one in the Prototype Projection. In practice, which type of subgraph do we use as the final interpretation? What if the two types of subgraphs conflict? \n\n3.If the interpretability of the prototypes is not considered,  can the Prototype Projection method  be removed?\n\nReferences:\n\n[1] Debiasing graph neural networks via learning disentangled causal substructure.\n\n[2] Causal attention for interpretable and generalizable graph classiﬁcation.\n\nLIMITATIONS:\nSee Weaknesses for details.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-14"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nInterpretable graph learning can promote the use of graph-based scientific applications by providing model explanations. This paper focuses on extracting key subgraphs and employing a case-based reasoning process (also known as prototype learning) for model prediction. To make the extracted substructures more informative, the information bottleneck theory is incorporated into the subgraph extraction. Several strategies, such as prototype merging and connectivity constraints, are used to improve the model's interpretability and accuracy. Finally, this paper provides both qualitative and quantitative analyses of the model's interpretability and performance.\n\nSTRENGTHS:\n1.\tThe idea of using prototype learning to explain key substructures of the input graph is intuitive and reasonable.\n2.\tThe extension of information bottleneck theory to prototype learning is inspiring and has practical utility in a wide range of application scenarios. In particular, it can help compress excessive information from the input graph and provide more fine-grained explanations, including key subgraphs and representative examples.\n3.\tThe paper is well-organized and easy to follow.\n\nWEAKNESSES:\n1.\tThe related works are not properly cited. For example, the prototype merging method is quite similar to the one used in ProtoPShare [1], yet the author simply claims that \"We propose a method to effectively merge the prototypes\" (Line 224), without mentioning any existing works. This can confuse readers about the technical contribution.\n[1] Rymarczyk, Dawid, et al. \"Protopshare: Prototypical parts sharing for similarity discovery in interpretable image classification.\" Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2021.\n2.\tThe paper's experiments are not sound enough. Firstly, the paper should include at least one black-box GNN model for performance prediction. Secondly, when evaluating the fidelity scores, ProtGNN should not be excluded (Section 4.3). Third, the paper is missing ablation studies for some proposed components, such as Connectivity loss.\n3.\tThe design of dropping the first term in equation 9 (in Section 3.3.2) is not well-motivated. The goal is to minimize the whole $I(Y;\\mathcal{G}p|\\mathcal{G}{sub})$ instead of the single term $I(\\mathcal{G}p ; \\mathcal{G}{sub})$. The paper should provide essential theoretical analysis to better illustrate the effectiveness and rationality of this optimization goal.\n\nQUESTIONS:\n1.\tWhat is the physical meaning of $p_i$ in Eq. 5? It seems to represent the extent of interference on the node representation. It is unclear how this practice can be considered as selecting a subgraph, making the physical meaning of connectivity loss also unclear.\n2.\tWhy adding interference to the nodes according to the probability $p_i$ yields the representation of the subgraph? Is the subgraph representation obtained in this way equal to $f_g(G_{sub})$?\n3.\tIn Eq. 8, how is the first inequality derived?\n\nLIMITATIONS:\nThe authors have discussed the limitations and societal impacts.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-06-24"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a new design of interpretable graph neural networks (GNNs) that combines ideas from information bottleneck approaches and prototype based approaches for by-design interpertation. The system titled PGIB extracts a subgraph from a given input graph and compares the embedding of the extracted subgraph in latent space to a set of learnt prototype subgraphs for final prediction. The authors evaluate their method on multiple datasets and baselines for classification performance and fidelity and demonstrate their efficacy for both.\n\nSTRENGTHS:\n1. The presented idea is novel. I have not seen any previous approach combining information bottleneck principles with prototype based classification.\n\n2. The choice of baselines for comparison and overall experimental setup is solid. The main results in the paper are also convincing and positively reflect for the method.\n\n3. The paper is generally well written with clear motivations.\n\nWEAKNESSES:\n1. The paper while reading gives sense of having too many moving parts. \n\n2. Hyperparameter selection is understudied and feels arbitrary. The ablation studies should be strengthened.\n\n3. Interpretation analysis --  Various choices in the system (loss weights or otherwise) are not well studied from the lens of interpretability. There are comments made around $\\alpha_1, \\alpha_2$ in Sec 4.4 but are not very well supported, with sparing analysis in appendix. At the very least for multiple choices you should include qualitative results specially for the ones for which you have some insights.\n\nOverall it's a decent paper but various choices the authors make need to be better supported experimentally.\n\nQUESTIONS:\n1. **Hyperparameter selection:** -- I am a little puzzled by the hyperparameter choices selection. For eg. is the performance drop in Fig. 6(a) \"drastic\". It is just 2-3% drop in mean performance throughout from $\\alpha_1=0$ to $1$. Am I understanding the graph incorrectly? The behaviour you highlight in A.4.3 is expected but I am rather thinking how is the performance still so high if the subgraphs are too compressed for $\\alpha_1=1$? Similarly $\\alpha_2$ choice also seems arbitrary based on the Fig. 6(b) with even less performance variation. \n\n2. **Ablation** -- The number of prototypes and parameters for merge operation. How were they determined? How do they affect the interpretability and performance?\n\n3. Typos in line 126 (\"is maximizes\"), line 312 (\"Fidlity\"), Fig 4 ('sparcity')\n\nLIMITATIONS:\nAuthors discuss these in supplementary. I would prefer to have the limitations in the main paper.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-06"}

comments: SUMMARY:
This paper investigates the usage of prototype learning for GNN explainability, focusing in particular on identifying key subgraphs through the graph information bottleneck principle.

Extensive experiments are conducted, which consider several baselines and different molecular datasets.

STRENGTHS:
- The scope of the paper is well-defined, and the goal of the novel method is relevant.
- The method is well described and motivated, and the code has been made available to improve reproducibility.
- The baselines include diverse, recent SOTA methods. Different real-world datasets have been used. Overall, the experiments appear robust.

WEAKNESSES:
- The main weakness in my opinion is that it is hard to conclude that the proposed method consistently outperforms all the baselines across all the metrics, given the reported results. For example, are the results in Table 1 statistically significant, given how large the confidence interval is? Similarly, in Table 2 for F+, VGIB reports the same results (why they are not bold?). 
I think that the proposed method clearly has an advantage for some properties/metrics, but the complexity of the results should be better discussed n the text. I suggest (1) extending the discussion about the limitations of the model and (2) extending the analysis to investigate in what contexts (and why) the model significantly outperforms the baseline.

QUESTIONS:
See previous point.

LIMITATIONS:
See previous point on better discussing the limitations of the results.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-10