DISK - CSV : Distilling Interpretable Semantic Knowledge with a Class Semantic Vector (2021)

Abstract: Neural networks (NN) applied to natural language processing (NLP) are becoming deeper and more complex, making them increasingly difficult to understand and interpret. Even in applications of limited scope on fixed data, the creation of these complex “black-boxes” creates substantial challenges for debugging, understanding, and generalization. But rapid development in this field has now lead to building more straightforward and interpretable models. We propose a new technique (DISK-CSV) to distill knowledge concurrently from any neural network architecture for text classification, captured as a lightweight interpretable/explainable classifier. Across multiple datasets, our approach achieves better performance than the target black-box. In addition, our approach provides better explanations than existing techniques.

## introduction deep neural networks (dnns) are popular in many applications, including computer vision and natural language processing. however, two major factors still require attention: (1) understanding the classifier's prediction for the end-users to develop trust in the model, and to enable machine learning engineers to refine it. this is especially true for high-stakes domains such as clinical decision support. there has been attention on models that attempt to make a neural network explainable, for instance, @xcite , and @xcite , which create post-hoc explanations to support explainability. secondly, (2) artificially high numbers of parameters make the inference time expensive. neural networks tend to be deep, with millions of parameters. for example, gpt-2 @xcite ) needs over 1.5 billion parameters. as a result, they are compute-intensive, thus making it difficult to deploy in real-world applications. we here propose a model-agnostic interpretable knowledge distillation method for neural network text classification. as shown in figure 1 , we learn a class semantic vector for each output class, concurrently when training the black box. we then use the semantic vectors to create the nearest neighbor classifier (compressed interpretable/explainable classifier ) from the black-box version. knowledge distillation refers to the process of transferring the implicit knowledge learned by a teacher model to a student model @xcite . dark knowledge refers to the salient information hidden in the predicted probabilities for all classes, which are more informative than the predicted classes themselves. our contributions can be summarized as follows: this work has connection with research on explainablity and model compression. explainability. most of the existing explainable ai (xai) techniques for natural language processing text classification focus on assigning a score to each word in the document w.r.t. predicted class, typically using gradient-based or perturbation-based methods @xcite @xcite . the most popular technique for model-agnostic explanation is lime @xcite , which focuses on creating an interpretable classifier by approximating it locally, with a linear model. the main drawback of these methods is that those explanations are not faithful to the target model @xcite . there are other methods, which focused on constructing a self-explainable network @xcite and @xcite . these techniques have limited explanations and thus do not explain phrases. our work is different from post-hoc and self-explainable approaches as it attempts to learn an explainable smaller classifier concurrently with the target black-box model. our explanations are also generated from the interpretable classifier itself, without extra calculation as in post-hoc techniques. ## disk-csv in a text classification task, an input sequence @xmath0, ..., x l , x @xmath1, where l is the length of the input text and d is the vector dimension, is mapped to a distribution over class labels using a parameterized θ neural network model (e.g., a long short term memory network (lstm), transformer, etc.), which we denote as f(x; θ). the output y is a vector of class probabilities, and the predicted class ŷ is a categorical outcome, such as an entailment decision. in this work, we are interested in learning a simpler compressed nearest neighbor classifier (e.g., easy-to-explain its prediction) from any neural network model, but concurrently, while training the larger model. we refer to the large model (black-box) as t and the smaller interpretable/explainable model as s. we call our method disk-csv -distilling interpretable semantic knowledge with a class semantic vector. in the next subsections we provide the following details for disk-csv: (a) how to distill knowledge from t into s, (b) how to construct interpretable representations for s, and (c) how to interact with the model to achieve better explainability (e.g., by clustering data, explaining essential phrases, and providing a semicounterfactual explanation). neural networks learn by optimizing a loss function to reflect the true objective of the end-user. for s, our objective is to generalize in the same way as t and approximate an explanation for each prediction. to demonstrate our idea, we show how we can learn s concurrently with a long short-term memory network (lstm) and then discuss how it can be generalized to different types of architectures for text classification. an lstm network processes the input word by word, and at time-step t, the memory c t and the hidden state h t are updated. the last state h @xmath2: the classifier uses cross-entropy loss to penalize miss-classification as l classif @xmath3, where @xmath4. the network's weights are updated via back-propagation when training the black-box. we intend to augment the neural nets that typically use embeddings to represent discrete variables (e.g., words) as continuous vectors. words that have a similar context will have similar meanings. the simplest form of concurrent knowledge distillation is to transfer the knowledge from the embedding space of t into a k-class semantic vectors (csvs) v @xmath5, where the dimension of each v i is equal to the dimension of the embedding vector x i , and k is the number of target classes. in other words, for each class label, we would like to learn a vector that captures the semantic information related to that class from the embedding layer. these semantic vectors have the following properties: (1) each vector v i should capture/encode the semantics about the class i from the black-box; capturing semantics: the second term of equation 2 is the second loss function we use, which attempts to encode the information of semantically consistent sentences in a single csv v ŷ. an obvious way to learn semantic information is to minimize the cosine distance between the average of the embedding x of the input sentence x and the predicted class semantic vector v ŷ. this objective will ensure that the vector v i captures the semantics of consistent inputs to encourage semantic consistency. hidden knowledge extraction: the last hidden state h l in recurrent nets is typically used by the output layer as the feature vector to predict the class label. as a result, the salient information learned by t is encoded in this feature vector. to distill this knowledge and enrich the representation of v i so that s generalizes well, we again minimize the cosine distance between the class semantic vector v i and the last hidden state h l in the third loss function, which is the third term in equation 2. this objective allows the model s to generalize similarly to the black-box t . the only constraint here is that the dimension of h l must be the same as that of x i so that we can minimize the cosine distance, i.e., h @xmath6. vector-separation: our ultimate goal is to create a simple interpretable nearest neighbor classifier s from the black-box. therefore, we want to make sure that the csvs are well separated from each other so that the cosine distance is maximum between them. to address this problem, we maximize the pairwise euclidean distance between these vectors using the fourth term in equation 2. ## conclusion and future work we have explored an approach to knowledge distillation concurrently from any black-box model to produce a simple, explainable classifier. the distilled model achieves better results than the original black-box models in terms of the model's performance. also, we showed that our distilled model provides a better explanation than lime. we have also proposed new types of explanations: first, a user can query with any-length phrases and receive feedback about the phrase's contribution to the classes. second, we also provide word(feature) importance to non-predicted classes, which can be used as a semi-counterfactual explanation. third, we showed how we could cluster the documents without employing the existing clustering method. in future work, we would like to extend this idea to pre-trained networks, and we also plan to more deeply investigate the value of counterfactual explanations.