RANCC : Rationalizing Neural Networks via Concept Clustering (2020)

Abstract: We propose a new self-explainable model for Natural Language Processing (NLP) text classification tasks. Our approach constructs explanations concurrently with the formulation of classification predictions. To do so, we extract a rationale from the text, then use it to predict a concept of interest as the final prediction. We provide three types of explanations: 1) rationale extraction, 2) a measure of feature importance, and 3) clustering of concepts. In addition, we show how our model can be compressed without applying complicated compression techniques. We experimentally demonstrate our explainability approach on a number of well-known text classification datasets.

## introduction deep neural network (dnn) models provide powerful and sophisticated approaches for addressing natural language processing (nlp) text classification tasks. yet their underlying behaviour is often opaque, especially in sensitive domains which can critically influence a user's decision, such as in legal and medical domains. as a result, we need to create dnns that are explainable, in other words, that can provide explanations for their predictions. our work here focuses on understanding predictions made by a dnn model, to provide explanations at inference time. there has, alternatively, been attention on models attempt to make a neural network explainable, for instance @xcite and @xcite , which create post-hoc explanations to support explainability. there are also other methods which focused on learning explanations concurrently with the prediction. for example, @xcite and @xcite proposed a neural network architecture for text classification which "justifies" its predictions by selecting relevant tokens in the input text. but this interpretable representation is then adjusted by a complex neural network, so the method is transparent as to what aspect of the input it uses for prediction, but not how it captures the salient features. our work also focuses on extracting a rationale (which can also be defined as an excerpt or justification) simultaneously while computing a prediction. in addition, we insist that explanations are provided interactively, so that a user can switch from one explanation to another to gain a better understanding about the class prediction (sometimes referred to as alternative explanations) as in the work of @xcite ). overall, we investigate how to improve the level of abstraction for dnns and go beyond measuring feature importance. to address this problem, we present a new alternative explanation mechanism, i.e., clustering similar rationales into distinct clusters concurrently with rationale extraction. our explanation mechanism for dnns can be summarized as follows: a concept is represented as a vector in the space which groups all the set of examples (rationales) that share the meaning of the concept. for example, consider sentiment classification that has the following rationales in movie reviews, such as "very nice movie," "enjoyed the movie," "but the acting was very nice," et cetera; all of these text rationales share the same general semantics of "positiveness," and will be represented as a single concept (positive sentiment). our model learns to group semantically consistent rationales across multiple texts into a single concept cluster. in addition, it is known that a non-linear deep network relies on millions or even billions of parameters, which makes it hard to be deployed in many real-world problems (e.g., on small devices such as cell phones, fpgas, etc.) due to lack of computation power and the availability of gpus. we show how our model can be compressed to work in real time applications without sacrificing significant accuracy, and without re-training, parameter pruning, or using quantization techniques. this solution is unique to our model, and to the best of our knowledge, is the first method which combines these techniques in a single model. figure 1 shows the explainability structure of our model. to explain a prediction, our model can expose the extracted rationale, measure relevant feature importance, and visualize the clustered concept. we can also identify the top salient words based on feature importance. our contributions can be summarized as follows: 1) we propose a self-explainable neural network model that concurrently extracts a rationale and predicts the classification output; 2) we extract rationales from input texts, and cluster them into concepts; 3) we improve the explainability of the black-box model by producing the extracted rationales and producing visualized clusters of the constructed concepts; and 4) our model can be compressed without losing much accuracy, and can be deployed as an on-line service in real-time applications in resource-restricted devices. ## rancc: rationalizing neural networks via concept clustering we call our model rancc -rationalizing neural networks via concept clustering. this section explains the ideas and methods in rancc: (a) how to build a self-explainable neural network model for text classification (e.g., a model that provides a rationale concurrently with the prediction), (b) how to simultaneously learn concepts of interest from the training data and cluster them, and (c) how to compress rncc model. the overall model is shown in figure 2 . ## related work most of the emerging work on explainability relies on approximating feature importance from a pretrained network, typically using perturbation and gradient-based methods @xcite @xcite . these methods assign a score to each word in the input text w.r.t the predicted class. one of the problems with post-hoc explanation is that it is often computationally expensive. for instance, the method proposed by @xcite takes up to one hour to produce a result. however, our approach is different, as it attempts to build a self-explainable network. there are other post-hoc techniques that relied on decomposing the output of lstms to learn feature importance @xcite . they introduce a technique which decomposes the output of the lstm into a sum over word coefficients, then show how these coefficients are meaningful with respect to a prediction -they later improved the approach @xcite . several other methods are based on learning from word vectors such as @xcite , while some used auto-encoders with graph theory to provide explanations @xcite . another approach for tackling explainability in text classification is to learn a rationale, i.e., a subset of short and meaningful features from the text input. these methods focus on augmenting the neural network with another network architecture to uncover the rationale by finding out which portions of the input contribute most to the prediction of a target class @xcite . their neural network creates a bernouli distribution over the set of the input variables. during training they sample from the rationale-network and during inference they use argmax over the distribution. however, the level of the abstraction of these methods is limited to the input level. in our case, we create a self-explainable neural network to extract the rationale, thus making our approach end-to-end learning, i.e., we only use a single neural network to extract a rationale for explainability. in addition, we go beyond the rationales by introducing concepts and clustering rationales through concepts without additional computation. recently some work started to consider concept extraction for nlp @xcite . however, our approach attempts to learn meaningful clusters for concepts while the existing work attempts to extract individual words to represent the concepts. finally, there is a debate on whether attention can be used for explanation or not. this is beyond the scope of this work and we refer the reader to the following papers @xcite and @xcite . in addition, our work is different from topic modelling methods @xcite as we attempt to learn clusters as well as rationales concurrently with the classification in neural networks. ## experiments our primary intent is not predictive accuracy. we used standard practices for training without much tuning. the aim of the experiments can be summarized as follows: 1) we show how rancc outperforms the state-of-the-art baselines for rationale extraction, 2) we show how rancc outperforms feature importance methods (i.e., post-hoc explanation methods), 3) we show how rancc can be used to cluster rationales without using any dimensionality reduction technique, 4) we visualize the concepts learned by the classifier, 5) we visualize a few samples of the extracted rationales, and 6) we show the performance of the compressed rancc model. the hyperparameters used for the experiments are shown in ## conclusion and future work we have presented a new approach for a self-explainable neural network applied to text classification, and presented new techniques for alternative explanations. our extracted rationals were important features affecting the prediction, and the visualization of the concept clustering improved the explainability of black-box models. in future work, we are interested in understanding more about the semantic distance between rationales and the concept vectors, as well as the semantic distance between every word in a sequence and its concept vector. we will also further investigate the explainability of the compressed rancc model, and aim to investigate our approach's explainability in other nlp tasks, such as natural language inference, and language generation in a specific domain of medicine or law.