Reducing Spurious Correlations for Answer Selection by Feature Decorrelation and Language Debiasing (2022)

Abstract: Deep neural models have become the mainstream in answer selection, yielding state-of-the-art performance. However, these models tend to rely on spurious correlations between prediction labels and input features, which in general suffer from robustness and generalization. In this paper, we propose a novel Spurious Correlation reduction method to improve the robustness of the neural ANswer selection models (SCAN) from the sample and feature perspectives by removing the feature dependencies and language biases in answer selection. First, from the sample perspective, we propose a feature decorrelation module by learning a weight for each instance at the training phase to remove the feature dependencies and reduce the spurious correlations without prior knowledge of such correlations. Second, from the feature perspective, we propose a feature debiasing module with contrastive learning to alleviate the negative language biases (spurious correlations) and further improve the robustness of the AS models. Experimental results on three benchmark datasets show that SCAN achieves substantial improvements over strong baselines. For reproducibility, we will release our code and data upon the publication of this paper.

## introduction answer selection, which aims to select the most applicable answers from an answer candidate pool, has broad applications in information retrieval (ir) and natural language processing (nlp). conventional answer selection methods primarily focus on designing various features, such as syntactic features @xcite , dependency trees @xcite , and translation features @xcite . however, the remarkable success of these methods relies heavily on feature engineering, which is a labor-intensive and time-consuming process. subsequently, deep neural models @xcite @xcite have been widely employed for answer selection and become the mainstream techniques for answer selection by automatically learning the contextual representations of questions and answers. to capture the relationships between the question-answer pairs, different attention mechanisms @xcite @xcite @xcite have been proposed to learn the interactive features of the questions and the answers. recently, pre-trained language models, such as bert @xcite and roberta @xcite , have been proposed and applied to answer selection @xcite @xcite , obtaining the state-of-the-art results. despite the remarkable progress of previous works, these deep neural models are prone to rely on spurious correlations between input features and prediction labels, which capture the prediction correlations that hold for most training samples but do not hold in general. the spurious correlations limit the robustness and generalization ability of the neural as models to the out-of-distribution and challenging datasets. in particular, for answer selection, the word-overlap between the question and the answers is highly correlated with the relevance prediction label. thus, the deep as models perform poorly on the out-of-distribution or challenging corpora that cannot be tackled with these superficial correlations (e.g., word overlap). this issue is also referred to as dataset bias @xcite and data distribution shift @xcite . in this paper, we propose a novel spurious correlation reduction method to improve the robustness of the neural answer selection models (scan) from the sample and feature perspectives by removing the feature dependencies and language biases in answer selection. first, from the sample perspective, we employ the feature decorre-lation module based on random fourier features @xcite to decorrelate the relevant and irrelevant features by learning a weight for each sample during the training phase, which facilitate the deep as models to reduce spurious correlations and concentrate on the true discriminative features (relevant features) for label prediction. second, from the feature perspective, we propose a feature debiasing module with contrastive learning to weaken the negative biases in language and improve the robustness of the as models. concretely, the feature debiasing module aims to make the base contextual representation of input sample close to the debiased features and away from the negative bias features. our main contributions are three-fold: ## methodology we assume there are n instances (question-answer pairs) in the training set. given a question q i and a set of k candidate answers a @xmath0, a 2 , ..., a k }, the answer selection task aims to find the best answer by ranking the candidate answers based on their relevance to the given question. benefiting from the pairwise ranking, we can reformalize the answer selection as a classification problem by predicting the relevance label y i of each question-answer pair (q i , a i ). we represent each question q i and answer a i as @xmath1, . . . , w q i n ] and a @xmath2, . . . , w a i m ], where n and m are the lengths of question q i and answer a i , respectively. in this paper, we propose a novel scan method for answer selection. as illustrated in figure 1 , the proposed scan consists of two primary modules: the feature decorrelation module based on sample weighting and the feature debiasing module based on contrastive learning @xcite . next, we will introduce the base context encoder and two key spurious correlation reduction components in detail. ## conclusion in this paper, we proposed a novel spurious correlation reduction method to improve the robustness of the answer selection models from the sample and feature perspectives. first, we devised a feature decorrelation module by learning a weight for each training instance to remove the feature dependencies and reduced the spurious correlations without prior knowledge of such correlations. second, we introduced a feature debiasing module with contrastive learning to alleviate the negative language biases and improved the robustness of the as mod- question: "what are some easy ways to get a toddler to go to sleep without being mean?" predicted by tanda: "there are plenty of ways.... -the most obvious is try to sleep. -take some pepto-bismol. -have a piece of peppermint (that's good too even if you still get sick)." (incorrect answer) predicted by scan: "let them play for a while. also, play with them. that way they'll feel like you care about them. also, try laying down with them. that used to help my son, who now is 3 years old." (correct answer)