FlowX: Towards Explainable Graph Neural Networks via Message Flows

1 INTRODUCTION . With the advances of deep learning , graph neural networks ( GNNs ) are achieving promising performance on many graph tasks , including graph classification ( Xu et al. , 2019 ; Gao & Ji , 2019 ; Chen et al. , 2020 ) , node classification ( Kipf & Welling , 2017 ; Veličković et al. , 2018 ; Wu et al. , 2019 ) , and graph generation ( Luo et al. , 2021 ; You et al. , 2018 ) . Many research efforts have been made to develop advanced graph operations , such as graph message passing ( Kipf & Welling , 2017 ; Veličković et al. , 2018 ; Li et al. , 2019 ) , graph pooling ( Yuan & Ji , 2020 ; Zhang et al. , 2018 ; Ying et al. , 2018 ) , and 3D graph operations ( Schütt et al. , 2017 ; Klicpera et al. , 2020 ) . Deep graph models usually consist of many layers of these operations stacked on top of each other interspersed with nonlinear functions . The resulting deep models are usually deep and highly nonlinear and complex . While these complex systems allow for accurate modeling , their decision mechanisms are highly elusive and not human-intelligible . Given the increasing importance and demand for trustworthy and fair artificial intelligence , it is imperative to develop methods to open the black-box and explain these highly complex deep models . Driven by these needs , significant efforts have been made to investigate the explainability of deep models on images and texts . These methods are developed from different perspectives , including studying the gradients of models ( Simonyan et al. , 2013 ; Smilkov et al. , 2017 ; Yang et al. , 2019 ) , mapping hidden features to input space ( Zhou et al. , 2016 ; Selvaraju et al. , 2017 ) , occluding different input features ( Yuan et al. , 2020a ; Dabkowski & Gal , 2017 ; Chen et al. , 2018 ) , and studying the meaning of hidden layers ( Yuan et al. , 2019 ; Olah et al. , 2018 ; Du et al. , 2018 ) , etc . In contrast , the explainability of deep graph models is still less explored . Since graph data contain limited locality information but have important structural information , it is usually not natural to directly extend image or text based methods to graphs . Recently , several techniques have been proposed to explain GNNs , such as XGNN ( Yuan et al. , 2020b ) , GNNExplainer ( Ying et al. , 2019 ) , PGExplainer ( Luo et al. , 2020 ) , and SubgraphX ( Yuan et al. , 2021 ) , etc . These methods mainly focus on explaining graph nodes , edges , features , or subgraphs . In this work , we observe and argue that message flows are the inherent functional mechanism of GNNs and thus are more natural and intuitive for studying the explainability of GNNs . To this end , we propose a message flow based explanation method , known as FlowX , to explain GNNs . FlowX attributes GNN predictions to message flows and studies the importance of different message flows . We first develop a systematic framework that lays the foundation on message flows for naturally explaining the message passing in GNNs . With our framework , the FlowX first quantifies the importance of flows by following the phylosophy of Shapley values . Since message flows can not be directly quantified to calculate marginal contributions , we propose an approximation scheme as the initial assessments of different flows . We then propose a learning-based algorithm taking advantages from the initial assessments to capture important flows in predictions . We conduct extensive experiments on both synthetic and real-world datasets . Experimental results show that our proposed FlowX outperforms existing methods significantly and consistently . Both quantitative and qualitative studies demonstrate that our proposed FlowX leads to improved explainability of GNNs . 2 RELATED WORK . 2.1 GRAPH NEURAL NETWORKS . With the advances of deep learning , several graph neural network approaches have been proposed to solve graph tasks , including graph convolutional networks ( GCNs ) ( Kipf & Welling , 2017 ) , graph attention networks ( GATs ) ( Veličković et al. , 2018 ) , and graph isomorphism networks ( GINs ) ( Xu et al. , 2019 ) , etc . They generally follow a message-passing framework to learn graph node features . Specifically , the new features of a target node are learned by aggregating messages flows passed from its neighboring nodes . Without loss of generality , we consider the input graph as a directed graph with n nodes and m edges . The graph is denoted as G = ( V , E ) , where V = { v1 , . . . , vn } denotes nodes , and E = { eij } represents edges in which eij is the directed edge vi → vj . Then it can be represented by a feature matrix X ∈ Rd×n and an adjacency matrix A ∈ Rn×n . Each node vi is associated with a d-dimensional feature vector xi corresponding to the i-th column of X . The element aij in A represents the weight of eij , and aij = 0 indicates eij does not exist . For the t-th layer in GNNs , the message aggregation procedures can be mathematically written as a two-step computation as Aggregate : St = Xt−1Ât , ( 1 ) Combine : Xt = M t ( St ) , ( 2 ) where Xt ∈ Rdt×n denotes the node feature matrix computed by the t-th GNN layer and X0 = X . Here M t ( · ) denotes the node feature transformation function at layer t and Ât is the connectivity matrix at layer t. Note that we name the elements in Ât as layer edges and âtjk indicates the layer edge connecting node j and k in layer t. For example , in GCNs , the transformations are defined as σ ( W tSt ) and Ât = D− 1 2 ( A+ I ) D− 1 2 where W t ∈ Rdt×dt−1 is a trainable weight matrix , σ ( · ) denotes the activation function , I is an identity matrix to add self-loops to the adjacency matrix , and D denotes the diagonal node degree matrix . We can stack T GNN layers on top of each other to form a T -layer network , and the network function can be expressed as f ( G ) = g ( MT ( MT−1 ( · · ·M1 ( X0Â1 ) · · · ) ÂT−1 ) ÂT ) . When f ( G ) is a graph classification model , g ( · ) generally consists of a readout function , such as global mean pooling , and a multi-layer perceptron ( MLP ) graph classifier . Meanwhile , when f ( G ) is a node classification model , g ( · ) represents a MLP node classifier . 2.2 EXPLAINABILITY OF GRAPH NEURAL NETWORKS . A major limitation of GNNs is their lack of explainability . Thus , different methods have been proposed to explain the predictions of GNNs , such as GraphLime ( Huang et al. , 2020 ) , GNNExplainer ( Ying et al. , 2019 ) , PGExplainer ( Luo et al. , 2020 ) , PGMExplainer ( Vu & Thai , 2020 ) , SubgraphX ( Yuan et al. , 2021 ) , XGNN ( Yuan et al. , 2020b ) , and GraphSVX ( Duval & Malliaros , 2021 ) . These methods can be mainly grouped into four categories based on the views of their explanations . First , several techniques provide explanations by identifying important nodes in the input graph . For example , GradCAM ( Pope et al. , 2019 ) measures node importance by combining the hidden features and gradients ; LRP ( Baldassarre & Azizpour , 2019 ) and Excitation BP ( Pope et al. , 2019 ) decompose the predictions into several terms and assign these terms to different nodes ; PGM-Explainer ( Vu & Thai , 2020 ) builds a probabilistic graphical model by randomly perturbing the node features and employs an interpretable Bayesian network to generate explanations . Second , several existing methods , such as GNNExplainer ( Ying et al. , 2019 ) , PGExplainer ( Luo et al. , 2020 ) , and GraphMask ( Schlichtkrull et al. , 2021 ) , explain GNNs by studying the importance of different graph edges . These methods follow a similar high-level idea that learns masks to identify important edges while maximizing the mutual information . Next , the recent study SubgraphX ( Yuan et al. , 2021 ) proposes to explain GNNs via subgraphs . It incorporates the Monte Carlo tree search algorithm to explore subgraphs and employs Shapley values to measure the importance . Finally , XGNN ( Yuan et al. , 2020b ) focuses on model-level explanations , which can provide high-level insights and general understanding . It proposes to generate graph patterns that can maximize a certain model prediction . While those methods explain GNNs from different views , none of them can provide explanations in terms of message flows and we believe message flows are more natural for performing explainability . To the best of our knowledge , GNN-LRP ( Schnake et al. , 2020 ) is the only algorithm that explains GNNs by relevant walks . The relevant walk is defined as a T -length ordered edge sequence that corresponds to a T -step directed path on the input graph . To study walk explainability , GNN-LRP considers the GNN prediction as a function and decomposes it using higher-order Taylor expansions to distribute prediction scores to relevant walks . Specifically , by using T -order Taylor expansion with a proper root , each term in the Taylor expansion corresponds to a relevant walk and is regarded as the importance score . While our proposed FlowX shares a similar explanation target , i.e. , flow/walk , with GNN-LRP , our method is fundamentally different . The GNN-LRP is developed based on score decomposition while our method follows the phylosopy of Shapley values from cooperative game theory as initial assessments and proposes a learning-based algorithm for the score generation . In addition , GNN-LRP has several constraints on the activation function and bias term used in GNNs while our method can be applied to general GNN models . Furthermore , as the GNN-LRP follows the Gradient× Input scheme , it may not pass the model parameter randomization test and may not be sensitive to model parameters ( Adebayo et al. , 2018 ) . Differences with Other Methods Using Shapley Values : Shapley values are commonly used in explaining machine learning methods . In particular , a recent study proposes a surrogate method , known as GraphSVX ( Duval & Malliaros , 2021 ) , to explain GNNs with both node and feature masks . Another recent study proposes SubgraphX ( Yuan et al. , 2021 ) , which employs a search algorithm to explore and identify subgraphs with high Shapley scores . While these methods use Shapley values , there are several fundamental differences . First , our proposed method focuses on explaining message flows , which are the most basic and natural units for explanations as GNNs are based on message passing schemes . Second , we only use Shapley-like values as initial approximations to facilitating further training . Experiments show that the learning step is very important . Furthermore , the fundamental difference with GraphSVX is reflected in the fact that , while our method is a perturbation-based method , GraphSVX is a surrogate method ( Yuan et al. , 2020c ) . Due to these differences , we show in experiments that our FlowX achieves more natural and improved performance as compared with other methods .

Summary: FlowX is an explanation method for Graph Neural Networks. It derives importance measures for message flows, which are inherent function mechanisms of GNNs. These measures are initialized with a Monte Carlo approximation of the Shapley Values from Game Theory, applied to message flows. They are refined by learning a linear transformation, before being converted back to layer edge importance score ultimately provided as an explanation. Various experiments are performed to assess the relevance of the proposed explainer.