Exploring the Trade-off Between Model Performance and Explanation Plausibility of Text Classifiers Using Human Rationales (2024)

Abstract: Saliency post-hoc explainability methods are important tools for understanding increasingly complex NLP models. While these methods can reflect the model’s reasoning, they may not align with human intuition, making the explanations not plausible. In this work, we present a methodology for incorporating rationales, which are text annotations explaining human decisions, into text classification models. This incorporation enhances the plausibility of post-hoc explanations while preserving their faithfulness. Our approach is agnostic to model architectures and explainability methods. We introduce the rationales during model training by augmenting the standard cross-entropy loss with a novel loss function inspired by contrastive learning. By leveraging a multi-objective optimization algorithm, we explore the trade-off between the two loss functions and generate a Pareto-optimal frontier of models that balance performance and plausibility. Through extensive experiments involving diverse models, datasets, and explainability methods, we demonstrate that our approach significantly enhances the quality of model explanations without causing substantial (sometimes negligible) degradation in the original model’s performance.

## introduction the complexity of text classification models and architectures has recently grown, posing challenges in comprehending the rationale behind their decisions. consequently, the latest nlp algorithms have been called black-box algorithms. understanding the model's reasoning is essential in various text classification contexts @xcite (e.g., hate speech detection). however, this task is hindered by the black-box nature of these models. moreover, comprehending the model's reasoning can help establish trust and make informed decisions based on the underlying justifications. researchers have developed popular text classification explainability techniques, such as post-hoc local saliency (or heatmap) methods @xcite . these methods generate heatmaps over tokens (paragraphs, sentences, words, sub-words, or characters) to indicate their significance in the final decision @xcite @xcite -although their suitability is criticized @xcite , these methods are still widely applied @xcite . the estimation of importance is performed after the decision has been made using an already trained model (i.e., it is post-hoc). for instance, figure 1 illustrates word-level saliency explanations that justify the predictions of two trained models in determining whether a movie review is positive or negative. in explanation (a), highlighted in green, the most relevant words align well with human expectations, making it intuitive. however, in explanation (b), the highlighted words are irrelevant from a human perspective. both explanations may accurately reflect the models' reasoning (thus, they may be faithful, according to @xcite . nevertheless, they differ in plausibility, which refers to the extent to which the explanation matches human intuition @xcite or is "convincing of the model prediction" @xcite . ideally, we should be able to enhance the plausibility of a "non-plausible" model by "teaching" it to provide more plausible explanations. previous works, such as those by @xcite @xcite @xcite , have explored this concept. the reason is that someone training the model clearly understands what a valid explanation should entail. however, achieving plausibility while preserving faithfulness may require modifying the reasoning of the original model, which in turn risks impacting its performance on the test data. hence, an inherent trade-off exists between model performance and explanation plausibility @xcite . this paper introduces a methodology that enhances the plausibility of explanations while remaining agnostic to the model architecture and explainability method. our approach incorporates human explanations, represented as rationales (i.e., text annotations serving as ground truth for explanations), into text classification models using a novel contrastive-inspired loss. we address the trade-off between classification and the new loss within a multi-objective framework, enabling exploration of the balance between performance and plausibility. unlike other approaches, our methodology does not require modifying the model architecture (e.g., through the addition of attention mechanisms; @xcite or assuming a specific type of explanation function (e.g., a differentiable explanation function; @xcite to incorporate the explanations. in summary, our contributions are: we compare our methodology with a previous method from the literature, reinforcing our results. furthermore, we address the social and ethical implications of "teaching" explanations to text classification models. we argue that these concerns are mitigated when the explanations remain faithful to the model's decision-making process. ## theoretical background we define crucial explainability and multi-objective optimization concepts to facilitate a global understanding of our research. we also point to an overview of contrastive learning in appendix c. ## methodology we focus on text classification models to enhance the quality of local saliency post-hoc explanations regarding plausibility. we aim to align these explanations with human intuition while maintaining faithfulness. to achieve this, we leverage rationales to enhance the explanation quality and evaluate the improvement by comparing them with the model explanations. ## experiments this section describes experiments to test the methodology proposed in section 4, employing diverse models, datasets, and explainability techniques. we aim to verify the usefulness of the contrastive rationale loss (section 4.2) in incorporating human rationales and the effectiveness of the moo solver (section 4.3) in finding models that well-represent the pareto-frontier. furthermore, we also compare our methodology with previous work. implementation and execution information can be found in appendix e. ## conclusion we propose a novel approach for enhancing the explanation plausibility of text classification models by incorporating human rationales, which capture human knowledge. our method is model-agnostic and explainability method-agnostic, making it compatible with various model architectures and explainers. we introduce a new contrastive-inspired loss function that integrates the rationales into the learning process. we demonstrate the feasibility of finding models that achieve a trade-off between improved plausibility and a minimal or negligible decrease in model performance. a comparative analysis establishes the superior effectiveness of our approach in enhancing plausibility while maintaining faithfulness and model agnosticism. we validate our method using a diverse set of explainers, datasets, and models encompassing modern and traditional nlp models. furthermore, we envision the potential extension of our approach to accommodate other explainers, datasets, and models, offering a seamless pathway to enhancing the plausibility of text classification algorithms.