FlowX: Towards Explainable Graph Neural Networks via Message Flows

1 INTRODUCTION . With the advances of deep learning , graph neural networks ( GNNs ) are achieving promising performance on many graph tasks , including graph classification ( Xu et al. , 2019 ; Gao & Ji , 2019 ; Chen et al. , 2020 ) , node classification ( Kipf & Welling , 2017 ; Veličković et al. , 2018 ; Wu et al. , 2019 ) , and graph generation ( Luo et al. , 2021 ; You et al. , 2018 ) . Many research efforts have been made to develop advanced graph operations , such as graph message passing ( Kipf & Welling , 2017 ; Veličković et al. , 2018 ; Li et al. , 2019 ) , graph pooling ( Yuan & Ji , 2020 ; Zhang et al. , 2018 ; Ying et al. , 2018 ) , and 3D graph operations ( Schütt et al. , 2017 ; Klicpera et al. , 2020 ) . Deep graph models usually consist of many layers of these operations stacked on top of each other interspersed with nonlinear functions . The resulting deep models are usually deep and highly nonlinear and complex . While these complex systems allow for accurate modeling , their decision mechanisms are highly elusive and not human-intelligible . Given the increasing importance and demand for trustworthy and fair artificial intelligence , it is imperative to develop methods to open the black-box and explain these highly complex deep models . Driven by these needs , significant efforts have been made to investigate the explainability of deep models on images and texts . These methods are developed from different perspectives , including studying the gradients of models ( Simonyan et al. , 2013 ; Smilkov et al. , 2017 ; Yang et al. , 2019 ) , mapping hidden features to input space ( Zhou et al. , 2016 ; Selvaraju et al. , 2017 ) , occluding different input features ( Yuan et al. , 2020a ; Dabkowski & Gal , 2017 ; Chen et al. , 2018 ) , and studying the meaning of hidden layers ( Yuan et al. , 2019 ; Olah et al. , 2018 ; Du et al. , 2018 ) , etc . In contrast , the explainability of deep graph models is still less explored . Since graph data contain limited locality information but have important structural information , it is usually not natural to directly extend image or text based methods to graphs . Recently , several techniques have been proposed to explain GNNs , such as XGNN ( Yuan et al. , 2020b ) , GNNExplainer ( Ying et al. , 2019 ) , PGExplainer ( Luo et al. , 2020 ) , and SubgraphX ( Yuan et al. , 2021 ) , etc . These methods mainly focus on explaining graph nodes , edges , features , or subgraphs . In this work , we observe and argue that message flows are the inherent functional mechanism of GNNs and thus are more natural and intuitive for studying the explainability of GNNs . To this end , we propose a message flow based explanation method , known as FlowX , to explain GNNs . FlowX attributes GNN predictions to message flows and studies the importance of different message flows . We first develop a systematic framework that lays the foundation on message flows for naturally explaining the message passing in GNNs . With our framework , the FlowX first quantifies the importance of flows by following the phylosophy of Shapley values . Since message flows can not be directly quantified to calculate marginal contributions , we propose an approximation scheme as the initial assessments of different flows . We then propose a learning-based algorithm taking advantages from the initial assessments to capture important flows in predictions . We conduct extensive experiments on both synthetic and real-world datasets . Experimental results show that our proposed FlowX outperforms existing methods significantly and consistently . Both quantitative and qualitative studies demonstrate that our proposed FlowX leads to improved explainability of GNNs . 2 RELATED WORK . 2.1 GRAPH NEURAL NETWORKS . With the advances of deep learning , several graph neural network approaches have been proposed to solve graph tasks , including graph convolutional networks ( GCNs ) ( Kipf & Welling , 2017 ) , graph attention networks ( GATs ) ( Veličković et al. , 2018 ) , and graph isomorphism networks ( GINs ) ( Xu et al. , 2019 ) , etc . They generally follow a message-passing framework to learn graph node features . Specifically , the new features of a target node are learned by aggregating messages flows passed from its neighboring nodes . Without loss of generality , we consider the input graph as a directed graph with n nodes and m edges . The graph is denoted as G = ( V , E ) , where V = { v1 , . . . , vn } denotes nodes , and E = { eij } represents edges in which eij is the directed edge vi → vj . Then it can be represented by a feature matrix X ∈ Rd×n and an adjacency matrix A ∈ Rn×n . Each node vi is associated with a d-dimensional feature vector xi corresponding to the i-th column of X . The element aij in A represents the weight of eij , and aij = 0 indicates eij does not exist . For the t-th layer in GNNs , the message aggregation procedures can be mathematically written as a two-step computation as Aggregate : St = Xt−1Ât , ( 1 ) Combine : Xt = M t ( St ) , ( 2 ) where Xt ∈ Rdt×n denotes the node feature matrix computed by the t-th GNN layer and X0 = X . Here M t ( · ) denotes the node feature transformation function at layer t and Ât is the connectivity matrix at layer t. Note that we name the elements in Ât as layer edges and âtjk indicates the layer edge connecting node j and k in layer t. For example , in GCNs , the transformations are defined as σ ( W tSt ) and Ât = D− 1 2 ( A+ I ) D− 1 2 where W t ∈ Rdt×dt−1 is a trainable weight matrix , σ ( · ) denotes the activation function , I is an identity matrix to add self-loops to the adjacency matrix , and D denotes the diagonal node degree matrix . We can stack T GNN layers on top of each other to form a T -layer network , and the network function can be expressed as f ( G ) = g ( MT ( MT−1 ( · · ·M1 ( X0Â1 ) · · · ) ÂT−1 ) ÂT ) . When f ( G ) is a graph classification model , g ( · ) generally consists of a readout function , such as global mean pooling , and a multi-layer perceptron ( MLP ) graph classifier . Meanwhile , when f ( G ) is a node classification model , g ( · ) represents a MLP node classifier . 2.2 EXPLAINABILITY OF GRAPH NEURAL NETWORKS . A major limitation of GNNs is their lack of explainability . Thus , different methods have been proposed to explain the predictions of GNNs , such as GraphLime ( Huang et al. , 2020 ) , GNNExplainer ( Ying et al. , 2019 ) , PGExplainer ( Luo et al. , 2020 ) , PGMExplainer ( Vu & Thai , 2020 ) , SubgraphX ( Yuan et al. , 2021 ) , XGNN ( Yuan et al. , 2020b ) , and GraphSVX ( Duval & Malliaros , 2021 ) . These methods can be mainly grouped into four categories based on the views of their explanations . First , several techniques provide explanations by identifying important nodes in the input graph . For example , GradCAM ( Pope et al. , 2019 ) measures node importance by combining the hidden features and gradients ; LRP ( Baldassarre & Azizpour , 2019 ) and Excitation BP ( Pope et al. , 2019 ) decompose the predictions into several terms and assign these terms to different nodes ; PGM-Explainer ( Vu & Thai , 2020 ) builds a probabilistic graphical model by randomly perturbing the node features and employs an interpretable Bayesian network to generate explanations . Second , several existing methods , such as GNNExplainer ( Ying et al. , 2019 ) , PGExplainer ( Luo et al. , 2020 ) , and GraphMask ( Schlichtkrull et al. , 2021 ) , explain GNNs by studying the importance of different graph edges . These methods follow a similar high-level idea that learns masks to identify important edges while maximizing the mutual information . Next , the recent study SubgraphX ( Yuan et al. , 2021 ) proposes to explain GNNs via subgraphs . It incorporates the Monte Carlo tree search algorithm to explore subgraphs and employs Shapley values to measure the importance . Finally , XGNN ( Yuan et al. , 2020b ) focuses on model-level explanations , which can provide high-level insights and general understanding . It proposes to generate graph patterns that can maximize a certain model prediction . While those methods explain GNNs from different views , none of them can provide explanations in terms of message flows and we believe message flows are more natural for performing explainability . To the best of our knowledge , GNN-LRP ( Schnake et al. , 2020 ) is the only algorithm that explains GNNs by relevant walks . The relevant walk is defined as a T -length ordered edge sequence that corresponds to a T -step directed path on the input graph . To study walk explainability , GNN-LRP considers the GNN prediction as a function and decomposes it using higher-order Taylor expansions to distribute prediction scores to relevant walks . Specifically , by using T -order Taylor expansion with a proper root , each term in the Taylor expansion corresponds to a relevant walk and is regarded as the importance score . While our proposed FlowX shares a similar explanation target , i.e. , flow/walk , with GNN-LRP , our method is fundamentally different . The GNN-LRP is developed based on score decomposition while our method follows the phylosopy of Shapley values from cooperative game theory as initial assessments and proposes a learning-based algorithm for the score generation . In addition , GNN-LRP has several constraints on the activation function and bias term used in GNNs while our method can be applied to general GNN models . Furthermore , as the GNN-LRP follows the Gradient× Input scheme , it may not pass the model parameter randomization test and may not be sensitive to model parameters ( Adebayo et al. , 2018 ) . Differences with Other Methods Using Shapley Values : Shapley values are commonly used in explaining machine learning methods . In particular , a recent study proposes a surrogate method , known as GraphSVX ( Duval & Malliaros , 2021 ) , to explain GNNs with both node and feature masks . Another recent study proposes SubgraphX ( Yuan et al. , 2021 ) , which employs a search algorithm to explore and identify subgraphs with high Shapley scores . While these methods use Shapley values , there are several fundamental differences . First , our proposed method focuses on explaining message flows , which are the most basic and natural units for explanations as GNNs are based on message passing schemes . Second , we only use Shapley-like values as initial approximations to facilitating further training . Experiments show that the learning step is very important . Furthermore , the fundamental difference with GraphSVX is reflected in the fact that , while our method is a perturbation-based method , GraphSVX is a surrogate method ( Yuan et al. , 2020c ) . Due to these differences , we show in experiments that our FlowX achieves more natural and improved performance as compared with other methods .

Summary: This paper explains GNNs by identifying important message flows. The authors use the concept of Shapley Value in Cooperative Game Theory and calculate Shapley Value as the initial assessments of flow importance score. A learning-based algorithm enables the refinement of important scores. Experiments are evaluated on both synthetic and real-world datasets.