Reinforcement Learning Enhanced Explainer for Graph Neural Networks

1 Introduction . Graph Neural Networks ( GNNs ) extend neural network models on ubiquitous graph data via utilizing the message passing scheme to incorporate graph structures with node features . They have achieved state-of-the-art performance not only in classic machine learning tasks on graphs , e.g. , node classification [ 10 , 28 ] , link prediction [ 38 ] , and graph classification [ 33 ] , but also in reasoning tasks , e.g. , intuitive physics [ 4 ] , mathematical reasoning [ 24 ] , and IQ tests [ 3 ] . Similar to most deep learning methods , one major limitation of GNNs is the lack of the interpretability for the predicted results ; a post-hoc analysis is usually needed to explain the results . To enhance the interpretability of GNNs , a line of works [ 34 , 17 , 30 , 37 , 31 ] focused on developing GNN explainers . The goal of GNN explainers is to identify a most influential subgraph structure to interpret the predicted label of an instance ( e.g. , a node or a graph ) . It can be generally formulated as 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . an optimization problem that maximizes the mutual information between the predicted results and the distribution of relevant subgraphs under some size constraints . The pioneering works , e.g. , GNNExplainer [ 34 ] and PGExplainer [ 17 ] , attempt to solve the optimization problem with continuous relaxation . These methods optimize a soft mask matrix for edges , and select the important nodes/edges by the threshold . However , they can not guarantee that nodes and edges in the output subgraph are connected . Thus , their explanatory subgraphs can not explicitly visualize the message passing paths . Besides , they consider the importance of each edge independently , and ignore the interactions among selected nodes and edges . Some recent works , such as SubgraphX [ 37 ] and Causal Screening [ 31 ] , design the search criteria and use search-based methods to solve the optimization problem . Due to the combinatorial property of searching explanatory graph structures , it is difficult to design a general hand-crafted search criterion . These criteria are limited on specific situations and thus not widely applicable . To address these issues , we propose RG-Explainer , which adopts reinforcement learning to explain GNNs ’ predictions . Our framework is inspired by classic combinatorial optimization solvers , which consists of three crucial steps : starting point selection , iterative graph generation and stopping criteria learning . These three components work together to generate an explanatory graph that interprets the predicted label of a given node/graph instance , as we elaborate next . Firstly , starting point selection selects the most important node as the seed node in the instance . If the task is to interpret the prediction of a specific node label , then the most important node refers to the node itself . To explain a graph label , we design a seed locator to learn the node that influences the graph label the most . Iterative graph generation is the key module in our method , which generates the nodes in the explanatory graph sequentially . Specifically , we add an influential node ( action ) from the neighbors based on the current generated graph ( state ) at each step . It explicitly guarantees the connectivity of the generated graph . The generation process is controlled by the reward , i.e. , the mutual information between the original predicted label and the label made by the generated graph . To ensure a compact and meaningful explanatory graph , we also involve some constraints into the reward , such as size loss , radius penalty and similarity loss . Finally , stopping criteria are learned to further avoid generating very large explanatory graphs . Furthermore , our method has better generalization ability and can be applied in both transductive and inductive setting . Different from the search-based methods , we learn the heuristics from the data automatically . A well-trained RG-Explainer can infer the explanations of instances which are not involved in the training phase . We conduct extensive experiments on both synthetic and real-world datasets to show that the proposed RG-Explainer can achieve superior performance compared to state-of-the-art GNN explainers . In particular , our visualization results further demonstrate the better intepretability of our method . 2 Related Work . Graph Neural Networks . Graph neural networks ( GNNs ) have achieved great success on graphstructured data in many real-world applications , including recommender systems , chemistry and bioinformatics [ 8 , 11 , 26 ] . The majority of GNNs used today follow the message passing scheme [ 8 ] , which aggregate information from neighbors with different aggregation functions , like mean/max/LSTMpooling in GCN [ 13 ] and GraphSAGE [ 10 ] , sum-pooling in GIN [ 33 ] , attention mechanisms in GAT [ 28 ] , etc . SGC [ 32 ] observes that the superior performance of GNNs is mainly due to the neighbor aggregation rather than feature transformation and nonlinearity , and proposed a simple and fast GNN model . APPNP [ 14 ] shares the similar idea by decoupling feature transformation and neighbor aggregation . Graph Generation . There have been a variety of methods for graph generation . RVAE [ 20 ] is a variational auto-encoder ( VAE ) based method with a regularizer to ensure semantic validity . Normalizing flow based methods including GraphNVP [ 21 ] , GraphAF [ 27 ] , and GraphDF [ 18 ] utilize invertible neural networks to define mappings between latent variables and data points . Generative adversarial networks ( GANs ) [ 9 ] based methods like MolGAN [ 7 ] and GCPN [ 35 ] involve a generator and a discriminator , where the generator is adversarially trained to fool the discriminator . From the perspective of graph generation process , they can be classified into one-shot generation and iterative generation . RVAE and MolGAN directly generate adjacency matrices , while GraphAF , GraphDF and GCPN generate graphs by sequentially adding new nodes and edges . Though our proposed RG-Explainer is an iterative generation method , RG-Explainer is different from the graph generation methods in that the above methods generate graph out of the air , while RG-Explainer needs to dynamically select suitable subgraphs to explain the predictions . Graph Combinatorial Optimization with RL . With the success of deep reinforcement learning in games [ 22 ] , researchers have attempted to utilize RL techniques for the graph combinatorial optimization problems [ 6 , 19 , 39 , 25 ] . Specifically , S2V-DQN [ 6 ] uses deep Q-learning with graph embedding to learn effective algorithms for the Minimum Vertex Cover , the Maximum Cut and the Traveling Salesman problems . A graph pointer network is proposed in [ 19 ] to solve the TSP efficiently . Further , Seal [ 39 ] learns heuristics to detect communities in the graph with policy gradient . Note that explaining GNNs is also a combinatorial optimization problem . Thus , in this paper , we propose a RL-based framework with three dedicated steps to generate explanations . Post-hoc Analysis in Graph Neural Networks . By extending existing image/text explanation techniques to the graph , some gradient-based methods [ 23 , 1 ] are proposed to study the importance of nodes and edges in the graph . However , their performances have been proved to be sub-optimal [ 34 ] because they can not incorporate the special properties of graphs . GNNExplainer [ 34 ] is the first specific method proposed to explain trained GNNs . It defines the problem as an optimization task , which maximizes the mutual information between the predicted labels and the distribution of possible subgraphs under some constraints . Following the problem setting , PGExplainer [ 17 ] leverages the representations generated by the trained GNN and adopts a deep neural network to learn the crucial nodes/edges . These methods both utilize the continuous relaxation on edges , and add size and entropy constraints to make the explanation small and sparse . Specifically , they optimize a soft mask matrix for edges , and select crucial nodes/edges by the threshold . However , they compute the importance of each edge independently , which may lead to a disconnected explanatory graph with information redundancies . Our model sequentially adds important nodes from the neighbors of the current generated graph , which considers the information already involved in the current graph and ensures the connectivity . SubgraphX [ 37 ] uses Monte Carlo tree search and Shapley value as a score function to find the best connected subgraphs as explanations for GNNs . Causal Screening [ 31 ] is another search-based method , but it uses greedy search and causality measure to generate the explanations . Different from the search-based methods where heuristics are usually hand-crafted , our method uses RL to learn heuristics from data , which can be widely applicable . Besides , our learning-based method could train by a small set of instances , and infer the explanations of many other similar unseen instances much faster than the search-based methods . Different from the instance-level explanation , there also exists the model-level explanation to investigate general patterns for predictions . For example , XGNN [ 36 ] utilizes the graph generator to interpret GNNs at the model-level . In particular , the instance-level explainer interprets the prediction for a certain given instance while the model-level explainer is input-independent and less precise . 3 Preliminary . In this section , we first introduce the notations used and then give the formal problem definition . Graph . LetG = ( V , E ) denote the graph with node set V = { v1 , v2 · · · vN } and edge set E ∈ V×V . Nodes in V could be associated with d-dimensional node features X ∈ RN×d . The graph G is described by the adjacency matrix A such that each entry Aij = 1 if eij ∈ E ; 0 , otherwise . Ã = A + IN denotes the adjacency matrix with added self-loops . A symmetrically normalized adjacency matrix with self-loops Â could be computed by D̃−1/2ÃD̃−1/2 , where D̃ is the diagonal degree matrix of Ã . Message Passing . Given an input graph G and node features X , a GNN model f ( G , X ) learns node representations . To fuse the information of both node features and graph topology in node representation vectors , GNN models utilize the message passing scheme to aggregate information from node neighbors . At each layer l , a node v aggregates all the messages in the ( l−1 ) -th layer from all its neighbors to generate its embedding : hlv = update ( ∑ w∈N ( v ) message ( h l−1 v , h l−1 w ) , h l−1 v ) , where N ( v ) is the neighbor set of node v. With L layers , each node v generates its embedding vector zv = h L v from the L-hop neighborhood . In the node classification task , we train an classifier with zv as input to predict the label for node v. In the graph classification task , we first use an aggregation function readout ( { zv } ) to generate the graph representation , which is further fed into a classifier to predict the label . Problem Definition . Given an input graph G = ( V , E ) and a trained GNN model f ( · ) , GNN explainers aim to generate an explanatory subgraph S = ( VS , ES ) , where VS ∈ V and ES ∈ E . The goal is to maximize the mutual information between the original label prediction Y = f ( G ) and the label prediction distribution based on the generated explanatory subgraph . Formally , the objective can be given as maxS MI ( Y , S ) = H ( Y ) −H ( Y |S ) , where MI ( · ) is the mutual information function and H ( · ) is the entropy function . Since H ( Y ) is fixed in the explanation stage , the objective can be rewritten as minS H ( Y |S ) .

Summary: This paper proposes an RL-based GNN explainer to address the neglected interactions among nodes and edges in the generated subgraph for model interpretation in the existing works. Specifically, the subgraph generation process is modeled as a Markov decision process (MDP) where the state is the currently selected node, action space is composed by the candidate neighboring nodes, and reward signal is the combinatorial loss to encourage the RL learner to generate a compact and succinct subgraph. In this way, the subgraph generation process strictly follows the structure of the original graph and therefore considers the interactions among nodes and edges.