Causal Screening to Interpret Graph Neural Networks

1 INTRODUCTION . Graph neural networks ( GNNs ) ( Gilmer et al. , 2017 ; Hamilton et al. , 2017 ; Velickovic et al. , 2018 ; Dwivedi et al. , 2020 ) have exhibited impressive performance in a wide range of tasks . Such a success comes from the powerful representation learning , which incorporates the graph structure with node and edge features in an end-to-end fashion . With the growing interest in GNNs , the explainability of GNN is attracting considerable attention . A prevalent technique is to offer post-hoc explanations via feature attribution . The attributions in GNNs are typically defined as the contributions of input features ( e.g. , nodes and edges ) to the model ’ s outcome ; thereafter , by selecting the most important features with top attributions , an explanatory subgraph is constructed to answer “ Why this GNN model makes such predictions ? ” . In this line , current works roughly fall into two categories : ( 1 ) decomposing the outcome prediction to graph structures via backpropagating the gradient-like signals ( Pope et al. , 2019 ; Baldassarre & Azizpour , 2019 ) ; and ( 2 ) approximating the decision boundary via structure perturbations ( Huang et al. , 2020 ) or structure masking ( Ying et al. , 2019 ) . However , these works mostly focus on the statistical interpretability ( Pearl , 2018 ; Moraffah et al. , 2020 ) , which could fail to uncover the causation of model predictions reliably . The key reason is that , they approach the input-outcome relationships from an associational standpoint , without distinguishing between causal and noncausal effects . Using correlation as causation to interpret feature importance will result in unfaithful explanations , as shown in the running example . Running Example . Consider the example in Figure 1 , where SA ( Baldassarre & Azizpour , 2019 ) and GNNExplainer ( Ying et al. , 2019 ) use gradients and masks as attributions respectively , to explain why the scene type of a scene graph is predicted as Surfing by APPNP ( Klicpera et al. , 2019 ) . Two limitations in the statistical interpretability are : ( 1 ) Confounding association . The edges with large gradient ( e.g. , ( shorts , on , man ) ) or masks ( e.g. , ( man , has , hand ) ) are highly correlated with the prediction , rather than causing it ( Moraffah et al. , 2020 ) . Such confounding associations distort the estimation of the causation ( e.g. , ( standing , on , surfboard ) ) ; ( 2 ) Redundancy . As the graph structure is highly entangled with GNNs , the gradient-like signals of edges are influenced , even scaled , by the connected edges . This makes redundant edges ( e.g. , ( man , on ocean ) and ( man , riding , waves ) ) involved in the top explanations , while forgoing other edges with unique information . In this work , we focus on the causal interpretability ( Pearl , 2018 ; Moraffah et al. , 2020 ) in GNNs . From a cause-effect standpoint , we need to answer causality-related questions like “ Was it a specific edge that caused the GNN ’ s prediction ? ” . Technically , explanatory subgraphs of interpretable features should account for two desirable characteristics : ( 1 ) Causality association . We need to identify important features , which may plausibly be causal determinants of the model outcome , away from these associated with the outcome due to confounding . As Figure 1 shows , ( standing , on , surfboard ) should be ranked at the top , instead of ( shorts , on , man ) . ( 2 ) Briefness . We need brief explanations to avoid redundancy . For example , ( man , on , surfboard ) is informative but redundant since it can be replaced by the top selection ( standing , on , surfboard ) . Hence , it is important to consider the dependency among edges . Although causality has been considered in very recent works of other domains ( Moraffah et al. , 2020 ) , they have yet to be applied in GNNs and fall short in the ability to explain GNNs . See Section 2 for an exhaustive list and full discussion . To the best of our knowledge , causal interpretability is yet unexplored in GNNs . Here we propose a novel method , Causal Screening , to take a cause-effect look at feature attribution . It takes a graph of interest , along with the prediction made by a trained GNN model , and aims to return an explanatory subgraph that plausibly has the largest causal attribution on the prediction . Towards this end , Causal Screening starts from an empty set as the explanatory subgraph , and adopts the screening strategy to incrementally select edges into the subgraph , one edge at a time step . At its core is to quantify the causal attribution of an edge candidate as its individual causal effect ( ICE ) ( Pearl , 2009 ) , answering “ What would happen to the prediction , if we add this edge into the GNN ’ s input ? ” . It coincides with maximizing the information-theoretic measure at each step — conditional mutual information ( Ay & Polani , 2008 ; Janzing et al. , 2013 ; O ’ Shaughnessy et al. , 2020 ) between the edge and the prediction , conditioned on the previously selected edges . Specifically , at each step , ICE is formulated as the difference between two outcomes , where the edge had received treatment ( i.e. , combining it with the previous selection as the GNN input ) or control ( i.e. , feeding the previous selection alone into the GNN ) . Last but not least , we propose an efficient version which considers the cause-effect of edge groups , rather than single edges , in order to speedup the exhaustive search . We apply Causal Screening to multiple graph classification datasets , generating qualitative results showcasing the effectiveness of our explanation subgraphs , which are more consistent , concise , and faithful to the predictions compared with existing methods . Contributions of this study can be summarized as : • We propose a model-agnostic method , Causal Screening , to provide a cause-effect perspective for explaining GNN models , i.e. , uncovering causal relationships between graph features and model predictions . • We conduct extensive experiments on three datasets , showcasing the effectiveness of our method w.r.t . predictive accuracy , contrastivity , and sanity checks . 2 RELATED WORK . Interpretability in Non-Graph Neural Networks . We focus preliminarily on feature attribution methods that generate post-hoc explanations for neural networks , especially convolutional neural networks ( CNNs ) . We roughly categorize current works into two groups : ( 1 ) Studies decompose the model prediction to the input features via backpropagating the gradient-like signals . Early works like Gradient ( Simonyan et al. , 2014 ) and Gradient * Input ( Shrikumar et al. , 2016 ) directly use gradients w.r.t . inputs as feature importance . Some follow-on studies , such as LRP ( Bach et al. , 2015 ) and Grad-CAM ( Selvaraju et al. , 2017 ) , improve gradients by using context within layers , while Integrated Gradient ( IG ) ( Sundararajan et al. , 2017 ) and DeepLIFT ( Shrikumar et al. , 2017 ) work on solving the gradient saturation problem ; ( 2 ) Another line approximates the decision boundary of model via feature perturbations like LIME ( Ribeiro et al. , 2016 ) or feature masking like L2X ( Chen et al. , 2018 ) and VIBI ( Bang et al. , 2019 ) . LIME uses a sparse linear model as a local explainer , while L2X and VIBI learn additional neural networks as explainers to generate feature masks , which maximize the mutual information ( or variation ) with the masked features and the predictions . However , these works mostly focus on the statistical interpretability , thus hardly uncovering the causation of model predictions . Interpretability in Graph Neural Networks . Interpretability in GNNs is less explored than that in CNNs . Thus , researchers get inspiration from the methods devised for CNNs and have made some attempts . For example , SA ( Baldassarre & Azizpour , 2019 ) treats the gradients of GNN ’ s loss w.r.t . the adjacency matrix as the attributions of edges . Pope et al . ( 2019 ) employ Grad-CAM family on GNN models . GraphLIME ( Huang et al. , 2020 ) extends LIME to graph-structured data . More recently , GNNExplainer ( Ying et al. , 2019 ) follows L2X and learns structure masking to maximize the mutual information between the subgraphs and GNN predictions . XGNN ( Yuan et al. , 2020 ) uses the graph generation to explore the class-level explanations , rather than the instance-wise . Despite the success , these methods inherit the limitations of the statistical interpretability . Causal Interpretability . Causality has been considered in very recent explanation works , such as structural causal models ( Chattopadhyay et al. , 2019 ; Kim & Bastani , 2019 ) , or counterfactual reasoning ( Goyal et al. , 2019 ) . For example , based on Granger causality , CXPlain ( Schwab & Karlen , 2019 ) learns to quantify the causal effect of a single feature by leaving it out . See Moraffah et al . ( 2020 ) for a survey . However , these approaches are mostly devised for CNNs and have yet to be applied in GNNs . Moreover , they possibly fall short in the ability to explain GNNs , due to particular challenges : ( 1 ) the relational and discrete data exist in graphs , unlike continuous features in images ; and ( 2 ) the graph structures are highly entangled with the representation learning of GNNs , such that different graphs likely result in various network architectures and different scales of explanations . In this work , we take a cause-effect look at the interpretability in GNNs . 3 METHODOLOGY . We first frame the explanation generation for GNNs from the causality standpoint . Afterward , we detail the proposed method towards the causal interpretability by sequentially quantifying the causal attributions of single edges . Lastly , we offer an efficient version based on clusters . 3.1 PROBLEM FORMULATION . Background of GNNs . Without loss of generality , let f : G → { 1 , · · · , S } denote a trained GNN classifier in the graph classification task , where G ∈ G is a graph instance involving the node set V and the edge set E , along with node features and optional edge features . In most GNNs , f is designed as : recursively create neural messages from neighboring nodes ( or edges ) , incorporate these messages to enrich the representation of each ego node , then aggregate all node representations as the holistic graph representation , in order to classify it into S classes . Task Description . We define a graph of interest as G = { e|e ∈ E } to highlight the structure features ( i.e. , the existence of an edge and its endpoints ) . Given its prediction ŷ = f ( G ) , our goal is to identify the top K causal edges and construct a convincing explanatory subgraph G∗K = { e∗1 , · · · , e∗K } ⊆ G. As a result , G∗K can be interpreted as the reason causing f to make the prediction ŷ for the instance G. In this work , we follow the previous works ( Baldassarre & Azizpour , 2019 ; Pope et al. , 2019 ) to generate explanations based on structure features , and leave the identification of a subset of node features in future work . Causal Attribution of A Subgraph . Technically , we obtain the causal explanatory subgraph G∗K as maximizing the causal attribution metric A ( · ) , which is formulated as : G∗K = arg maxGK⊆G A ( GK ) , s.t . |GK | = K. ( 1 ) Here we frame the metric A ( GK ) as the causal effect ( CE ) of GK , which is founded on the do ( · ) calculus ( Pearl , 2009 ) . The interventions do ( G = GK ) and do ( G = ∅ ) mean that the GNN input G receives treatment ( i.e. , feeding GK into the GNN ) and control ( i.e. , feeding nothing into the GNN ) , respectively . The CE of GK is to measure the difference between potential outcomes under treatment and control : A ( GK ) = I ( do ( G = GK ) ; ŷ ) − I ( do ( G = ∅ ) ; ŷ ) , ( 2 ) where I ( do ( G = GK ) ; ŷ ) = I ( GK ; ŷ ) is the mutual information ( Aliferis et al. , 2010 ; Belghazi et al. , 2018 ) between the intervened graph GK and the target prediction ŷ , to calculate the change in the probability of prediction when the input graph is limited to the subgraph GK ; analogously , I ( do ( G = ∅ ) ; ŷ ) = I ( ∅ ; ŷ ) . As the last term I ( do ( G = ∅ ) ; ŷ ) is a constant , the solution of Equation 1 is equivalent to obtain a subgraph G∗K with maximum predictivity to recover the target prediction ŷ . Here we summarize the attribution framework in the form of structural causal model ( well-known as causal graph ) ( Pearl , 2009 ; Pearl et al. , 2016 ; Pearl & Mackenzie , 2018 ) , as Figure 2 ( a ) shows . It indicates how variables interact with each others , e.g. , ( G , ŷ ) → I . The intervention assigns the GNN input with a certain value , such that the difference caused by the treatment and control measures the causal attribution of a subgraph as shown in Figure 2 ( b ) .

Summary: The paper introduces a novel method called Causal screening which takes a graph and the prediction made by a GNN, and returns an explanatory subgraph. The method aims to explain GNN models. To be precise, it starts from an empty set as the explanatory subgraph, and incrementally adds the edges, testing them for the individual causal effect.