Causal Screening to Interpret Graph Neural Networks

1 INTRODUCTION . Graph neural networks ( GNNs ) ( Gilmer et al. , 2017 ; Hamilton et al. , 2017 ; Velickovic et al. , 2018 ; Dwivedi et al. , 2020 ) have exhibited impressive performance in a wide range of tasks . Such a success comes from the powerful representation learning , which incorporates the graph structure with node and edge features in an end-to-end fashion . With the growing interest in GNNs , the explainability of GNN is attracting considerable attention . A prevalent technique is to offer post-hoc explanations via feature attribution . The attributions in GNNs are typically defined as the contributions of input features ( e.g. , nodes and edges ) to the model ’ s outcome ; thereafter , by selecting the most important features with top attributions , an explanatory subgraph is constructed to answer “ Why this GNN model makes such predictions ? ” . In this line , current works roughly fall into two categories : ( 1 ) decomposing the outcome prediction to graph structures via backpropagating the gradient-like signals ( Pope et al. , 2019 ; Baldassarre & Azizpour , 2019 ) ; and ( 2 ) approximating the decision boundary via structure perturbations ( Huang et al. , 2020 ) or structure masking ( Ying et al. , 2019 ) . However , these works mostly focus on the statistical interpretability ( Pearl , 2018 ; Moraffah et al. , 2020 ) , which could fail to uncover the causation of model predictions reliably . The key reason is that , they approach the input-outcome relationships from an associational standpoint , without distinguishing between causal and noncausal effects . Using correlation as causation to interpret feature importance will result in unfaithful explanations , as shown in the running example . Running Example . Consider the example in Figure 1 , where SA ( Baldassarre & Azizpour , 2019 ) and GNNExplainer ( Ying et al. , 2019 ) use gradients and masks as attributions respectively , to explain why the scene type of a scene graph is predicted as Surfing by APPNP ( Klicpera et al. , 2019 ) . Two limitations in the statistical interpretability are : ( 1 ) Confounding association . The edges with large gradient ( e.g. , ( shorts , on , man ) ) or masks ( e.g. , ( man , has , hand ) ) are highly correlated with the prediction , rather than causing it ( Moraffah et al. , 2020 ) . Such confounding associations distort the estimation of the causation ( e.g. , ( standing , on , surfboard ) ) ; ( 2 ) Redundancy . As the graph structure is highly entangled with GNNs , the gradient-like signals of edges are influenced , even scaled , by the connected edges . This makes redundant edges ( e.g. , ( man , on ocean ) and ( man , riding , waves ) ) involved in the top explanations , while forgoing other edges with unique information . In this work , we focus on the causal interpretability ( Pearl , 2018 ; Moraffah et al. , 2020 ) in GNNs . From a cause-effect standpoint , we need to answer causality-related questions like “ Was it a specific edge that caused the GNN ’ s prediction ? ” . Technically , explanatory subgraphs of interpretable features should account for two desirable characteristics : ( 1 ) Causality association . We need to identify important features , which may plausibly be causal determinants of the model outcome , away from these associated with the outcome due to confounding . As Figure 1 shows , ( standing , on , surfboard ) should be ranked at the top , instead of ( shorts , on , man ) . ( 2 ) Briefness . We need brief explanations to avoid redundancy . For example , ( man , on , surfboard ) is informative but redundant since it can be replaced by the top selection ( standing , on , surfboard ) . Hence , it is important to consider the dependency among edges . Although causality has been considered in very recent works of other domains ( Moraffah et al. , 2020 ) , they have yet to be applied in GNNs and fall short in the ability to explain GNNs . See Section 2 for an exhaustive list and full discussion . To the best of our knowledge , causal interpretability is yet unexplored in GNNs . Here we propose a novel method , Causal Screening , to take a cause-effect look at feature attribution . It takes a graph of interest , along with the prediction made by a trained GNN model , and aims to return an explanatory subgraph that plausibly has the largest causal attribution on the prediction . Towards this end , Causal Screening starts from an empty set as the explanatory subgraph , and adopts the screening strategy to incrementally select edges into the subgraph , one edge at a time step . At its core is to quantify the causal attribution of an edge candidate as its individual causal effect ( ICE ) ( Pearl , 2009 ) , answering “ What would happen to the prediction , if we add this edge into the GNN ’ s input ? ” . It coincides with maximizing the information-theoretic measure at each step — conditional mutual information ( Ay & Polani , 2008 ; Janzing et al. , 2013 ; O ’ Shaughnessy et al. , 2020 ) between the edge and the prediction , conditioned on the previously selected edges . Specifically , at each step , ICE is formulated as the difference between two outcomes , where the edge had received treatment ( i.e. , combining it with the previous selection as the GNN input ) or control ( i.e. , feeding the previous selection alone into the GNN ) . Last but not least , we propose an efficient version which considers the cause-effect of edge groups , rather than single edges , in order to speedup the exhaustive search . We apply Causal Screening to multiple graph classification datasets , generating qualitative results showcasing the effectiveness of our explanation subgraphs , which are more consistent , concise , and faithful to the predictions compared with existing methods . Contributions of this study can be summarized as : • We propose a model-agnostic method , Causal Screening , to provide a cause-effect perspective for explaining GNN models , i.e. , uncovering causal relationships between graph features and model predictions . • We conduct extensive experiments on three datasets , showcasing the effectiveness of our method w.r.t . predictive accuracy , contrastivity , and sanity checks . 2 RELATED WORK . Interpretability in Non-Graph Neural Networks . We focus preliminarily on feature attribution methods that generate post-hoc explanations for neural networks , especially convolutional neural networks ( CNNs ) . We roughly categorize current works into two groups : ( 1 ) Studies decompose the model prediction to the input features via backpropagating the gradient-like signals . Early works like Gradient ( Simonyan et al. , 2014 ) and Gradient * Input ( Shrikumar et al. , 2016 ) directly use gradients w.r.t . inputs as feature importance . Some follow-on studies , such as LRP ( Bach et al. , 2015 ) and Grad-CAM ( Selvaraju et al. , 2017 ) , improve gradients by using context within layers , while Integrated Gradient ( IG ) ( Sundararajan et al. , 2017 ) and DeepLIFT ( Shrikumar et al. , 2017 ) work on solving the gradient saturation problem ; ( 2 ) Another line approximates the decision boundary of model via feature perturbations like LIME ( Ribeiro et al. , 2016 ) or feature masking like L2X ( Chen et al. , 2018 ) and VIBI ( Bang et al. , 2019 ) . LIME uses a sparse linear model as a local explainer , while L2X and VIBI learn additional neural networks as explainers to generate feature masks , which maximize the mutual information ( or variation ) with the masked features and the predictions . However , these works mostly focus on the statistical interpretability , thus hardly uncovering the causation of model predictions . Interpretability in Graph Neural Networks . Interpretability in GNNs is less explored than that in CNNs . Thus , researchers get inspiration from the methods devised for CNNs and have made some attempts . For example , SA ( Baldassarre & Azizpour , 2019 ) treats the gradients of GNN ’ s loss w.r.t . the adjacency matrix as the attributions of edges . Pope et al . ( 2019 ) employ Grad-CAM family on GNN models . GraphLIME ( Huang et al. , 2020 ) extends LIME to graph-structured data . More recently , GNNExplainer ( Ying et al. , 2019 ) follows L2X and learns structure masking to maximize the mutual information between the subgraphs and GNN predictions . XGNN ( Yuan et al. , 2020 ) uses the graph generation to explore the class-level explanations , rather than the instance-wise . Despite the success , these methods inherit the limitations of the statistical interpretability . Causal Interpretability . Causality has been considered in very recent explanation works , such as structural causal models ( Chattopadhyay et al. , 2019 ; Kim & Bastani , 2019 ) , or counterfactual reasoning ( Goyal et al. , 2019 ) . For example , based on Granger causality , CXPlain ( Schwab & Karlen , 2019 ) learns to quantify the causal effect of a single feature by leaving it out . See Moraffah et al . ( 2020 ) for a survey . However , these approaches are mostly devised for CNNs and have yet to be applied in GNNs . Moreover , they possibly fall short in the ability to explain GNNs , due to particular challenges : ( 1 ) the relational and discrete data exist in graphs , unlike continuous features in images ; and ( 2 ) the graph structures are highly entangled with the representation learning of GNNs , such that different graphs likely result in various network architectures and different scales of explanations . In this work , we take a cause-effect look at the interpretability in GNNs . 3 METHODOLOGY . We first frame the explanation generation for GNNs from the causality standpoint . Afterward , we detail the proposed method towards the causal interpretability by sequentially quantifying the causal attributions of single edges . Lastly , we offer an efficient version based on clusters . 3.1 PROBLEM FORMULATION . Background of GNNs . Without loss of generality , let f : G → { 1 , · · · , S } denote a trained GNN classifier in the graph classification task , where G ∈ G is a graph instance involving the node set V and the edge set E , along with node features and optional edge features . In most GNNs , f is designed as : recursively create neural messages from neighboring nodes ( or edges ) , incorporate these messages to enrich the representation of each ego node , then aggregate all node representations as the holistic graph representation , in order to classify it into S classes . Task Description . We define a graph of interest as G = { e|e ∈ E } to highlight the structure features ( i.e. , the existence of an edge and its endpoints ) . Given its prediction ŷ = f ( G ) , our goal is to identify the top K causal edges and construct a convincing explanatory subgraph G∗K = { e∗1 , · · · , e∗K } ⊆ G. As a result , G∗K can be interpreted as the reason causing f to make the prediction ŷ for the instance G. In this work , we follow the previous works ( Baldassarre & Azizpour , 2019 ; Pope et al. , 2019 ) to generate explanations based on structure features , and leave the identification of a subset of node features in future work . Causal Attribution of A Subgraph . Technically , we obtain the causal explanatory subgraph G∗K as maximizing the causal attribution metric A ( · ) , which is formulated as : G∗K = arg maxGK⊆G A ( GK ) , s.t . |GK | = K. ( 1 ) Here we frame the metric A ( GK ) as the causal effect ( CE ) of GK , which is founded on the do ( · ) calculus ( Pearl , 2009 ) . The interventions do ( G = GK ) and do ( G = ∅ ) mean that the GNN input G receives treatment ( i.e. , feeding GK into the GNN ) and control ( i.e. , feeding nothing into the GNN ) , respectively . The CE of GK is to measure the difference between potential outcomes under treatment and control : A ( GK ) = I ( do ( G = GK ) ; ŷ ) − I ( do ( G = ∅ ) ; ŷ ) , ( 2 ) where I ( do ( G = GK ) ; ŷ ) = I ( GK ; ŷ ) is the mutual information ( Aliferis et al. , 2010 ; Belghazi et al. , 2018 ) between the intervened graph GK and the target prediction ŷ , to calculate the change in the probability of prediction when the input graph is limited to the subgraph GK ; analogously , I ( do ( G = ∅ ) ; ŷ ) = I ( ∅ ; ŷ ) . As the last term I ( do ( G = ∅ ) ; ŷ ) is a constant , the solution of Equation 1 is equivalent to obtain a subgraph G∗K with maximum predictivity to recover the target prediction ŷ . Here we summarize the attribution framework in the form of structural causal model ( well-known as causal graph ) ( Pearl , 2009 ; Pearl et al. , 2016 ; Pearl & Mackenzie , 2018 ) , as Figure 2 ( a ) shows . It indicates how variables interact with each others , e.g. , ( G , ŷ ) → I . The intervention assigns the GNN input with a certain value , such that the difference caused by the treatment and control measures the causal attribution of a subgraph as shown in Figure 2 ( b ) .

Summary: The paper proposes a procedure for identifying a subgraph $\mathcal{G}_K$ of a given size $K$ (measured by the number of edges) whose output through the GNN function $f$ is as close as possible to that of the full graph $\mathcal{G}$. The proposed method is a greedy approach which starts from an empty graph and gradually adds the next edge by minimizing the difference between the outputs using mutual information. Furthermore, to reduce the computational complexity, a node clustering is done on the graph and the attribution is applied first on the edges between $C$ identified clusters and then transferred to all edges.