Explaining Neural Network Predictions on Sentence Pairs via Learning Word-Group Masks (2021)

Abstract: Explaining neural network models is important for increasing their trustworthiness in real-world applications. Most existing methods generate post-hoc explanations for neural network models by identifying individual feature attributions or detecting interactions between adjacent features. However, for models with text pairs as inputs (e.g., paraphrase identification), existing methods are not sufficient to capture feature interactions between two texts and their simple extension of computing all word-pair interactions between two texts is computationally inefficient. In this work, we propose the Group Mask (GMASK) method to implicitly detect word correlations by grouping correlated words from the input text pair together and measure their contribution to the corresponding NLP tasks as a whole. The proposed method is evaluated with two different model architectures (decomposable attention model and BERT) across four datasets, including natural language inference and paraphrase identification tasks. Experiments show the effectiveness of GMASK in providing faithful explanations to these models.

## introduction explaining deep neural networks is critical for revealing their prediction behaviors and enhancing the trustworthiness of applying them in real-world applications. many methods have been proposed to explain neural network models from the posthoc manner that generates faithful explanations based on model predictions @xcite @xcite . most existing work focuses on identifying word attributions @xcite @xcite for nlp tasks. knowing which individual features are important might not be enough for explaining model behaviors. then, other recent work exploits feature interactions as explanations @xcite @xcite . however, they could suffer computation inefficiency while computing interactions between all word pairs, and they also fall short for identifying multiple important words correlated from different input sources for predictions. such intuitions are particularly important for explaining sentence pair modeling tasks such as natural language inference (nli) @xcite and paraphrase identification (pi) @xcite . figure 1 shows an example of nli, where the model makes correct prediction as contradic-tion. the first column visualizes individual word attributions to the prediction, where the top four important words are man, banjo, guitar, a. however, the correlations between them are unclear and intuitively man and a are irrelevant to the model prediction, which makes the explanation untrustworthy. a good explanation should be able to capture correlated words between the sentence pair, and identify their importance to the model prediction. in this work, we propose group masks (gmask), a model-agnostic approach that considers the importance of correlated words from two input sentences. in particular, it distributes correlated words into a group and learns the group importance. in figure 1 , the input words are distributed in four groups with importance @xmath0. the color saturation of purple lines represents the probability of a word belonging to a group. different from individual word attributions, gmask assigns electric, guitar, and banjo into important groups (g2/g1), while man and a into unimportant groups (g3/g4). the weighted word attributions computed as the weighted sum of group importance identify the important words electric, guitar from x 1 and banjo from x 2 , which explains the model prediction. the contribution of this work is three-fold: (1) we introduce gmask method to explain sentence pair modeling tasks by learning weighted word attributions based on word correlations; (2) we propose a sampling-based method to solve the optimization objective of gmask; and (3) we evaluate the proposed method with two types neural network models (decomposable attention model @xcite and bert @xcite ), for two types of sentence pair modeling tasks on four datasets. experiments show the superiority of gmask in generating faithful explanations compared to other competitive methods. ## related work many approaches have been proposed to explain deep neural networks from the post-hoc manner, such as gradient-based explanation methods @xcite , attention-based methods @xcite , and decomposition-based methods @xcite . however these white-box explanation methods are either rendering doubt regarding faithfulness @xcite or being limited to specific neural networks. in this work, we mainly focus on model-agnostic explanation methods, which are applicable to any black-box models. feature attributions many approaches explain models by assigning feature attributions to model predictions. for example, perturbation-based methods quantify feature attributions by erasing features @xcite or using local linear approximation as lime @xcite . kernelshap (lundberg and lee, 2017) utilized shapley values @xcite to compute feature attributions. another line of work proposed learning feature attributions, such as l2x @xcite which maximizes mutual information to recognize important features, and iba @xcite which identifies feature attributions by optimizing the information bottleneck @xcite . however, these approaches produce individual word attributions without considering feature correlations. our proposed method implicitly detects correlated words and generates weighted word attributions. feature interactions some work proposed to generate explanations beyond word-level features by detecting feature interactions. @xcite proposed contextual decomposition (cd) to compute word interactions and @xcite and @xcite further proposed hierarchical versions based on that. other work adopted shapley interaction index to compute feature interactions @xcite and build hierarchical explanations @xcite . however, computing feature interactions between all word pairs is computationally inefficient @xcite . methods which only consider the interactions between adjacent features are not applicable to sentence pair modeling tasks as critical interactions usually form between words from different sentences. gmask distributes correlated words from the input text pair into a group, and learns the group importance, without explicitly detecting feature interactions between all word pairs. word masks some related work utilized word masks to select important features for building interpretable neural networks @xcite or improving the interpretability of existing models @xcite proposed to track the information flow of input features through the layers of bert models. different from the prior work, gmask applies masks on a group of correlated words. figure 2 : the left part shows that masks are applied on the word embedding layer, selecting important words for the neural network model. the outputs y and ỹ are corresponding to the original input @xmath1, x 2 ] t and masked input @xmath2, x2 ] t respectively. the right part shows the sampling process of gmask. for z, the color saturation of purple blocks represents the probability of a word belonging to a specific group (i.e. φ i,j (ι)). z is a sample of z with binary values. for g, the color of each block represents group importance. g is a onehot vector sampled from g, indicating which group being selected. w is a sample of word masks obtained by multiplying z and g. ## method this section introduces the proposed gmask method. gmask implicitly learns word correlations, and distributes correlated words from different input sentences into a group. gmask learns the importance of each group by randomly masking out groups of words. finally, the weighted word attributions are computed based on word group distributions and group importance. ## experimental setup we evaluate gmask with two kinds of neural network models, decomposable attention model (dattn) @xcite and bert @xcite , for two types of sentence pair modeling tasks on four datasets. we compare our method with four baselines. ## results and discussion we compare the faithfulness of generated post-hoc explanations via both quantitative and qualitative evaluations. ## conclusion in this paper, we focused on sentence pair modeling and proposed an effective method, gmask, learning group masks for correlated words and calculating weighted word attributions. we tested gmask with two different neural network models on four datasets, and assessed its effectiveness via both quantitative and qualitative evaluations.