This Reads Like That: Deep Learning for Interpretable Natural Language Processing (2023)

Abstract: Prototype learning, a popular machine learning method designed for inherently interpretable decisions, leverages similarities to learned prototypes for classifying new data. While it is mainly applied in computer vision, in this work, we build upon prior research and further explore the extension of prototypical networks to natural language processing. We introduce a learned weighted similarity measure that enhances the similarity computation by focusing on informative dimensions of pre-trained sentence embeddings. Additionally, we propose a post-hoc explainability mechanism that extracts prediction-relevant words from both the prototype and input sentences. Finally, we empirically demonstrate that our proposed method not only improves predictive performance on the AG News and RT Polarity datasets over a previous prototype-based approach, but also improves the faithfulness of explanations compared to rationale-based recurrent convolutions.

## introduction due to the increasing utilization of artificial neural networks in critical domains such as medicine and autonomous driving, the need for interpretable and explainable decisions made by these models is extremely important @xcite . while extensive research in this field has focused on post-hoc explanations for black-box machine learning models @xcite @xcite , a recent line of investigation has challenged this paradigm by proposing network architectures that inherently possess interpretability @xcite @xcite . these interpretable models aim to base their decisions on a limited set of human-understandable features, enabling users to comprehend the decision-making process while * equal contribution maintaining high predictive performance @xcite . in this work, we focus on transforming and enhancing the prototype-based model introduced by @xcite from the computer vision domain to the natural language processing (nlp) domain. prototype-based methods for nlp, as explored in previous works @xcite @xcite , involve classifying test samples based on their similarity to prototypical training samples. to improve upon these prior adaptations, we propose two key enhancements. firstly, we introduce a learned weighted similarity measure that enables the model to attend to the most informative aspects of the pre-trained sentence embedding. this improvement ensures that the method can benefit from pre-trained language models @xcite , which have shown superb empirical results, and extract task-relevant information, thereby enhancing its performance. secondly, we enhance the interpretability mechanism by conducting post-hoc analyses to determine the importance of individual words in both test samples and sentence prototypes. furthermore, we investigate the faithfulness of these explanations, examining their ability to faithfully capture and represent the underlying model's decision-making process. in summary, our contributions in this paper involve domain-specific adaptations of inherently interpretable prototype-based models from computer vision to the nlp domain, enhancing their performance through a learned weighted similarity measure, improving explainability via post-hoc analyses of word importance, and investigating the faithfulness thereof. these advancements aim to further the understanding and trustworthiness of prototype-based methods in nlp. foot_0 ## related work the three main interpretability mechanisms for deep neural networks discussed in this work are post-hoc explainability, rationale-based models, and prototype-based models. post-hoc explanation methods correlate inputs and outputs to explain a model's behavior using gradient visualization @xcite and attention mechanisms @xcite @xcite . however, the faithfulness of these methods is uncertain, as learned attention weights can lack correlation with gradient-based measures of feature importance, and different attention distributions can yield equivalent predictions @xcite . rationale-based models @xcite @xcite improve explanation faithfulness by selecting rationales that serve as explanations for predictions. while rationales indicate contributing input features, they do not reveal how these features are combined or processed within the neural network. prototype-based networks @xcite ) emulate human-like reasoning by finding similarities between input parts and prototypes to make predictions. however, existing nlp-based prototype methods @xcite @xcite require full model control and lack the ability to visualize important words in sentences, failing the criterion of sparse explanations @xcite . our proposed method addresses these limitations. further details on related work can be found in appendix a.1. ## methods in our approach, we first compute the sentence embedding using a pretrained language models bert @xcite , gpt2 @xcite , mpnet @xcite , and roberta @xcite to capture the semantic representation of the input sentence. then, we utilize a prototypical layer to measure the similarity between the sentence embedding and learned prototypes. these prototypes act as learned prototypical points in the embedding space, representing the different classes, to which they have been assigned equally. the similarity scores are multiplied by the learned prototype weights to arrive at the final, inherently interpretable prediction. to provide a meaningful interpretation in the input space, we periodically project the prototypes to their nearest training sample of the same class. for the last three epochs, the prototypes are projected and fixed, thus enabling visualization of the prototypes contributing to an input's prediction at test time. to ensure the useful spreading of prototypes, we employ the loss terms proposed by @xcite . in appendix a.4, we conduct an ablation study to assess the effect of the different terms. more information about the experimental setup and implementation can be found in appendix a.2. ## results in this section, we present experiments demonstrating the importance of our proposed method. in the first experiment, the models were run five times with different random seeds, and the mean and standard deviation thereof are reported. in the second experiment, we calculated the test set confidence intervals (cis) through 1'000-fold bootstrapping as a proxy, due to the excessive computational cost. ## conclusion our work presents several significant contributions. firstly, we advance the research on extending prototypical networks to natural language processing by introducing a learned weighted similarity measure that enables our model to focus on the crucial dimensions within sentence embeddings while disregarding less informative context. this allows us to reap the benefits of pre-trained language models while retaining an inherently interpretable model. this enhancement leads to accuracy improvements over the previous work by @xcite on both the rt movie review dataset and the ag news dataset. additionally, we propose a posthoc visualization method for identifying important words in prototypical sentences and in test sentences. our results demonstrate improved faithfulness in capturing the essential factors driving the model's predictions compared to @xcite . collectively, our findings highlight the efficacy of learned weighted similarity measure, the value of our post-hoc visualization technique, both leading to the enhanced faithfulness of the proposed model. these advancements contribute to a deeper understanding of the interpretability of neural networks in natural language processing tasks.