Hard Masking for Explaining Graph Neural Networks

1 INTRODUCTION . Graph Neural Networks ( GNNs ) are a flexible and powerful family of models that build representations of nodes or edges on irregular graph-structured data and have experienced significant attention in recent years . These methods are based on the so-called “ neighborhood aggregation ” scheme in which a node representation is learned by aggregation of features from their neighbors and have achieved state-of-the-art performance on node and graph classification tasks . Despite their popularity , approaches investigating their interpretability have received limited attention . This paper focuses on explaining or interpreting the rationale underlying a given prediction of already trained graph neural networks . There have been numerous approaches proposed in the literature for the general interpretability of machine learning models . The most popular approaches are feature attribution methods that intend to attribute importance to input features given an input prediction either agnostic to the model parameter ( Ribeiro et al. , 2018 ; 2016 ) or using model-specific attribution approaches ( Xu et al. , 2015 ; Binder et al. , 2016 ; Sundararajan et al. , 2017 ) . However , models learned over graph-structured data have some unique challenges . Specifically , predictions on graphs are induced by a complex combination of nodes and paths of edges between them in addition to the node features . Thus explanations for a prediction should ideally be a small subgraph of the input graph and a small subset of node features that are most influential for the prediction ( Ying et al. , 2019 ) . The only existing approach for GNN explainability proposes to learn a real-valued graph mask that selects the important subgraph of the GNNs computation graph to maximize the mutual information with the GNNs prediction ( Ying et al. , 2019 ) . We identify two crucial limitations of such an approach . Firstly , although mathematically tractable , a continuous mask does not ensure sparsity compared to a discrete mask – a desirable property for interpretability . Secondly , suitable notions of what constitutes an explanation in a GNN model and its evaluation are missing . This paper proposes an alternate notion of interpretability for GNNs grounded in ideas from data compression in information theory . Specifically , we consider an explanation as a compressed form of the original feature matrix . The goodness of the explanation is measured by the expected deviation from the prediction of the underlying model . We formalize this idea of interpreting GNN decisions as an explicit optimization problem in a rate-distortion framework . A subgraph of the node ’ s computational graph and its set of features are relevant for a classification decision if the expected classifier score remains nearly the same when randomizing the remaining features . This formulation is arguably both a crisp , robust , and understandable notion of interpretability that is easy to evaluate . We propose a simple combinatorial procedure ZORRO that aims to find a sparse subset of features and nodes in the computational graph while adhering to a user-specified level of fidelity . Our method aims to find multiple disjoint explanations ( whenever possible ) that guarantee an acceptable lower bound on fidelity to the model ’ s decision . Another key problem in post-hoc interpretability of GNNs is that of evaluating explanation methods . Current evaluation methods , such as those used by GNNEXPLAINER , are primarily anecdotal and lack principled metrics . Secondly , especially for real-world datasets , there is no ground truth for the explanation , making comparison difficult . We , on the other hand , posit that an explanation is faithful to the underlying model if it retains enough predictive power – a crisp and measurable quantity . To this extent , our optimization metric , fidelity , encodes an information-theoretic interpretation of explanation – if the explanation is highly predictive in expectation , then it is a high qualitative explanation . We conducted extensive experimentation on three datasets and four diverse GNN approaches – Graph Convolution Networks ( Kipf & Welling , 2017 ) , Graph Attention Networks ( Veličković et al. , 2018 ) , GIN ( Xu et al. , 2019 ) , and APPNP ( Klicpera et al. , 2019 ) . Our main key findings are as follows . 1 . We show that not one but multiple diverse explanations are possible that sufficiently explain a prediction . This multiplicity of explanations indicates the possible configurations that could be utilized by the model to arrive at a decision . 2 . Unlike earlier mutual-information preserving interpretability approaches , i.e . GNNEXPLAINER ( Ying et al. , 2019 ) , we show that our explanations are both more predictive and sparse . We show that even with sparser explanations , our approach contains far more predictive capacity than GNNEXPLAINER . 3 . We then analyze the explanations across multiple GNN models to showcase differences between their learning behavior . We specifically show that GNN models rely heavily on homophily and that prediction errors are due inability to capture homophilic signals from their neighborhoods . 2 RELATED WORK . Representation learning approaches on graphs encode graph structure with or without node features into low-dimensional vector representations , using deep learning and nonlinear dimensionality reduction techniques . These representations are trained in an unsupervised ( Perozzi et al. , 2014 ; Khosla et al. , 2019 ; Funke et al. , 2020 ) or semi-supervised manner by using neighborhood aggregation strategies and task-based objectives ( Kipf & Welling , 2017 ; Veličković et al. , 2018 ) . This work focuses on the post-hoc interpretability of decisions made by semi-supervised models based on graph convolution networks for node classification tasks . Inspired by the success of convolutional neural networks , graph convolution network ( GCN ) ( Kipf & Welling , 2017 ) generalizes the convolution operation for irregular graph data . GCN and several of its variants follow a neighborhood aggregation strategy where they compute a node ’ s representation by recursive aggregation and transformation of feature representations of its neighbors . For the node classification task , the final node representations are then used to predict unlabelled nodes ’ classes . Interpretability in Machine Learning . Post-hoc approaches to model interpretability are popularized by feature attribution methods that aim to assign importance to input features given a prediction either agnostic to the model parameters ( Ribeiro et al. , 2018 ; 2016 ) or using model specific attribution approaches ( Xu et al. , 2015 ; Binder et al. , 2016 ; Sundararajan et al. , 2017 ) . Instance-wise feature selection ( IFS ) approaches ( Chen et al. , 2018 ; Carter et al. , 2018 ; Yoon et al. , 2018 ) , on the other hand , focuses on finding a sufficient feature subset or explanation that leads to little or no degradation of the prediction accuracy when other features are masked . The advantage of this formulation is that the output explanation has a precise meaning in terms of the predictive power of the chosen subset . Applying these works directly for graph models is infeasible due to the complex form of explanation , which should consider the complex association among nodes in addition to the input features . Interpretability in GNNs . Model agnostic approaches like ours to interpretability in GNNs include GNNEXPLAINER ( Ying et al. , 2019 ) and XGNN ( Yuan et al. , 2020 ) . GNNEXPLAINER learns a realvalued graph mask and feature mask such that the mutual information with GNN ’ s predictions is maximized . XGNN proposed a reinforcement learning-based graph generation approach to generate explanations for the predicted class for a graph . We instead focus on explaining node level decisions . As a model introspective approach , Pope et al . ( 2019 ) extended the gradient-based saliency map methods to GCNs , which rely on propagating gradients/relevance from the output to the original model ’ s input features . Other works ( Kang et al. , 2019 ; Idahl et al. , 2019 ) focus on explaining unsupervised network representations , which is out of scope for the current work . 3 PROBLEM DEFINITION AND APPROACH . 3.1 BACKGROUND ON GNNS . Let G = ( V , E ) be a graph where each node is associated with d dimensional input feature vector . Graph neural networks compute node representations by recursive aggregation and transformation of feature representations of its neighbors which are finally used for label prediction . Formally for a L-layer GNN , let x ( ` ) n denote the feature representation of node n ∈ V at a layer ` ∈ L and Nn denotes the set of its 1-hop neighbors . x ( 0 ) n corresponds to the input feature vector of n. The ` -th layer of a GNN can then be described as an aggregation of node features from the previous layer followed by a transformation operation . z ( ` ) n = AGGREGATION ( ` ) ( { x ( ` −1 ) n , { x ( ` −1 ) j | j ∈ Nn } } ) ( 1 ) x ( ` ) n = TRANSFORMATION ( ` ) ( z ( ` ) n ) ( 2 ) Each GNN defines its own aggregation function which is differentiable and usually a permutation invariant function . The transformation operation is usually a non-linear transformation employing ReLU non-linear activation . The final node ’ s embedding z ( L ) n is then used to make the predictions Φ ( n ) ← argmaxσ ( z ( L ) n W ) , ( 3 ) where σ is a sigmoid or softmax function depending on whether the node belongs to multiple or a single class . and W is a learnable weight matrix . The ith element of z ( L ) n W corresponds to the ( predicted ) probability that node n is assigned to some class i . 3.2 PROBLEM FORMULATION . We are interested in explaining the prediction of a GNN Φ ( n ) for any node n. We note that for a particular node , n the subgraph taking part in the computation of neighborhood aggregation operation , see Eq . ( 1 ) , fully determines the information used by GNN to predict its class . In particular , for a L-layer GNN , this subgraph would be the graph induced on nodes in the L-hop neighborhood of n. We will call this subgraph as the computational graph of the query node . We would like to pint out that the term computational graph should not be confused with the computational graph of the neural network . Let G ( n ) ⊆ G denote the computational graph of the node n. Let X ( n ) , or briefly X denotes the feature matrix restricted to the nodes of G ( n ) , where each row corresponds to a d-dimensional feature vector of the corresponding node in the computational graph . We formulate the task of explaining the model prediction for a node n , as finding a partition of the components of its computational graph into a subset , S of relevant nodes and features , and its complement Sc of non-relevant components . In particular , the subset S should be such that fixing its value to the true values already determines the model output for almost all possible assignments to the non-relevant subset Sc . The subset S is then returned as an explanation . To quantify relevance , we compute the expected value of fidelity in model ’ s prediction for the noisy assignment to the non-relevant components . Let us denote with YS the new perturbed feature matrix obtained by fixing the components of the S to their actual values and otherwise noisy entries . The values of components in Sc are then drawn from some noisy distribution , N . Let S = { Vs , Fs } be the explanation with selected nodes Vs and selected features Fs . Let S be the mask matrix such that each element Si , j = 1 if and only if ith node ( in G ( n ) ) and jth feature are included in sets Vs and Fs respectively and 0 otherwise . YS = X S + Z ( 1− S ) , Z ∼ N , ( 4 ) where denotes an element-wise multiplication , and 1 a matrix of ones with the corresponding size . Figure 1 shows how the fixed elements are selected by Fs and Vs . Definition . The fidelity of explanation S with respect to the graph neural network Φ and the noise distribution N is given by F ( S ) = EYS |Z∼N [ 1Φ ( X ) =Φ ( YS ) ] . ( 5 ) By fixing the fidelity to a certain user-defined threshold , say τ , we are then interested in all possible disjoint sets of explanations that would have the fidelity of at least τ . More precisely , our resulting set of explanations R is given as R = { S1 , S2 , . . . | ∀iF ( Si ) ≥ τ and ∩ i Si = ∅ } ( 6 ) CONNECTION TO THE RATE-DISTORTION THEORY Our problem formulation is inspired by rate-distortion theory ( Sims , 2016 ) which addresses the problem of determining the minimal information of a source signal that should be communicated over a leaky channel so that the source ( input signal ) can be approximately reconstructed at the receiver ( output signal ) without exceeding an expected distortion D. In our problem , we are interested in finding a small subset S such that having knowledge only about the signal on S and filling in the rest of the information randomly will almost surely preserve the class prediction if our chosen subset contains the information that is relevant for the model ’ s decision . Rather than measuring distortion or disagreement in the model ’ s decisions , we instead measure fidelity or agreement among the model ’ s decisions with the original and the distorted signal , respectively . Distortion can be computed using fidelity as D = 1 − F . A schematic representation of our problem in terms of rate-distortion framework is shown in Figure 2 .

Summary: This work proposes to explain graph neural networks using hard masking techniques. Specifically, it tries to find the node mask $V_s$ and feature mask $F_s$ which can identify the most important information of the input such that the masked information can yield a high fidelity score. This work proposes a greedy method, ZORRO, to explore these hard masks, which can be used as the explanations of the prediction. Experimental results are interesting and promising.