Contrastive Learning with Adversarial Examples for Alleviating Pathology of Language Model (2023)

Abstract: Neural language models have achieved superior performance. However, these models also suffer from the pathology of overconfidence in the out-of-distribution examples, potentially making the model difficult to interpret and making the interpretation methods fail to provide faithful attributions. In this paper, we explain the model pathology from the view of sentence representation and argue that the counter-intuitive bias degree and direction of the out-of-distribution examples’ representation cause the pathology. We propose a Contrastive learning regularization method using Adversarial examples for Alleviating the Pathology (ConAAP), which calibrates the sentence representation of out-of-distribution examples. ConAAP generates positive and negative examples following the attribution results and utilizes adversarial examples to introduce direction information in regularization. Experiments show that ConAAP effectively alleviates the model pathology while slightly impacting the generalization ability on in-distribution examples and thus helps interpretation methods obtain more faithful results.

## introduction neural language models have achieved superior performance in various natural language processing (nlp) domains and are used in many fields to accomplish critical tasks, such as toxic comment classification and rumor detection. however, the drawbacks of nlp models in test-time interpretability pose potential risks to these tasks, as existing interpretation methods always fail to obtain faithful attributions on these models, thereby failing to reveal potential flaws and biases. following @xcite , @xcite , and @xcite , the attribution obtained by a faithful interpretation method should indicate the real contribution of features in the normally tuned model is pathological, as the confidence distribution after removing important words is similar to after removing unimportant words, indicating that the interpretation method can not obtain faithful attributions. the model tuned with conaap is nonpathological, as the model can discriminate between the important and unimportant words in terms of confidence changing, and the attributions are more faithful. in this paper, we explain the model pathology, which potentially makes the model difficult to interpret, from the view of sentence representation, and intuitively show how the pathology leads to unfaithfulness attribution and how to alleviate the pathology effectively. based on our findings, we also propose a contrastive learning regularization method using adversarial examples for alleviating the pathology (conaap). we summarize our main contributions as follows: ## related work interpreting the language model. to interpret a language model, previous works utilize the gradient-based method @xcite @xcite @xcite , attention scores @xcite @xcite , occlusion @xcite @xcite @xcite , and shapley values (lundberg and lee, 2017) to attribute the model prediction. to quantitatively evaluate the faithfulness of the obtained attribution, metrics including reduced length (feng et al., 2018), comprehensiveness, sufficiency, and area over the perturbation curve (aopc) (deyoung et al., 2020) are proposed. contrastive learning. encouraged by the remarkable success of contrastive learning in computer vision (cv) in learning better representation and improving performance on downstream tasks @xcite , various methods have been proposed for nlp tasks. limited by the discrete nature of text, instead of generating contrastive pairs by cropping, resizing, and rotating the input like in cv tasks, previous works in nlp are always by back-translating, word deleting, reordering, and substituting @xcite @xcite . it is shown that contrastive learning helps improve sentence representation and model performance on downstream nlp tasks. however, few works focus on model pathology and interpretability. ## conclusion in this paper, we argue that the failure of interpretation methods to provide faithful attributions for language models is due to the model pathology that models are overconfident in out-of-distribution ex-amples when making predictions. we explain the model pathology from the perspective of sentence representation and propose conaap, a contrastive learning regularization method to calibrate the sentence representation of out-of-distribution examples. experiments demonstrate the effectiveness of conaap in alleviating model pathology, which helps interpretation methods obtain faithful results. we hope that our work will provide a new perspective on research in the field of interpretability.