Towards Robust Extractive Question Answering Models: Rethinking the Training Methodology (2024)

Abstract: This paper proposes a novel training method to improve the robustness of Extractive Question Answering (EQA) models. Previous research has shown that existing models, when trained on EQA datasets that include unanswerable questions, demonstrate a significant lack of robustness against distribution shifts and adversarial attacks. Despite this, the inclusion of unanswerable questions in EQA training datasets is essential for ensuring real-world reliability. Our proposed training method includes a novel loss function for the EQA problem and challenges an implicit assumption present in numerous EQA datasets. Models trained with our method maintain in-domain performance while achieving a notable improvement on out-of-domain datasets. This results in an overall F1 score improvement of 5.7 across all testing sets. Furthermore, our models exhibit significantly enhanced robustness against two types of adversarial attacks, with a performance decrease of only about one-third compared to the default models.

## introduction unanswerable questions are a valuable part of the training datasets of extractive question answering (eqa) models. by learning from these questions, models can develop the ability to avoid extracting misleading responses, ultimately improving their reliability in real-world applications. currently, there are two lines of research on unanswerable questions in eqa. firstly, @xcite introduced the squad 2.0 dataset by adding adversarial unanswerable questions into squad 1.1 @xcite . this work later inspired similar benchmarks in other languages such as french @xcite and @xcite . in the crowdsourcing process for adversarial unanswerable questions, human annotators are typically presented with a triple of context, an answerable question, and its corresponding answer(s). they are then asked to write unanswerable questions that exhibit an adversarial similarity to the presented answerable ones. in addition to the adversarially-written unanswerable questions, natural question @xcite , tydi qa @xcite , and squad agent @xcite propose more naturally constructed unanswerable questions. this category of unanswerable questions is also known as information-seeking unanswerable questions, emerging within the realm of information retrieval. these questions are initially independent of any context. the contexts are then paired with the questions as a result of the attempt to locate answers for the given questions within a large database containing multiple contexts. the distinct characteristics of these two types of unanswerable questions pose a challenge for models. models trained with one type of unanswerable questions often struggle when encountering the other type @xcite , defined in machine learning as a lack of robustness under distribution shift in the inputs. additionally, models trained on unanswerable questions also demonstrate a lack of robustness against adversarial attack @xcite . notably, models trained on adversarial unanswerable questions in squad 2.0 tend to output an "empty" response upon detecting any sign of contradiction between the attack sentence and the given question. we hypothesize that the observed lack of robustness in eqa models can be attributed to two primary factors. first, the current eqa training loss objective @xcite inaccurately treats unanswerable questions as if they have an answer span. this span is designated to start and end at the special classification token @xcite of the pre-trained model which is also the first token in the input sequence. this approach potentially misguides the model's understanding of unanswerable questions. second, the assumption that a given question can only have a single answer or no answer introduces a learning shortcut, making eqa models vulnerable to adversarial attacks. in this work, we propose a new training method for eqa models to address the two problems discussed above. first, we design a new training loss function that naturally treats unanswerable questions as lacking any answer. second, to overcome the single-answer assumption in most eqa datasets, we create a new "synthetic" answer span in a number of answerable questions. our empirical findings are summarized as follows: ## related work there are two key research areas on improving the robustness of natural language processing (nlp) models: robustness against adversarial attacks and against distribution shift @xcite . adversarial attacks involve editing a test sample to create a more challenging example for trained models without causing additional difficulty for humans. these attacks can be classified based on whether the attack process has access to models' parameters (white-box attacks, @xcite @xcite @xcite ) or not (black-box attacks, @xcite @xcite @xcite ). on the other hand, robustness against distribution shift is measured using test samples that exhibit linguistic differences from the samples encountered by models during the training phase @xcite . findings of limited robustness in nlp models have spurred significant efforts to improve their resilience. from a data-driven perspective, adversarial attacks can be employed during the training phase to enhance model robustness. augmented training data can be created by heuristically editing @xcite or through neuralbased generation @xcite @xcite . additionally, increasing the diversity of training data has proven to be an effective strategy for improving model robustness @xcite . in addition to data-driven approaches, modelbased approaches are also effective in improving model robustness. following the success of bert, various studies have shown that the pretraining process, which involves a self-supervised objective and the use of large amounts of diverse pretraining data, significantly enhances the generalization of language models in downstream tasks @xcite . another research direction involves using a biased model during the training phase to force the target model to discard some spurious patterns in the training set. these biased models can be designed with a specific targeted type of bias @xcite @xcite @xcite , or without prior knowledge about the biases present in the training dataset @xcite @xcite . our work distinguishes itself by combining both data-driven and model-driven approaches. from data-driven side, we challenge the implicit assumption of single answers in multiple current eqa datasets by augmenting "synthetic" answers to a number of training samples. we hope that our experimental results with synthetic answers will inspire the development of eqa datasets that in-corporate multi-span questions, enabling answers composed of multiple non-adjacent spans of text @xcite . on the model-driven side, we propose a novel training loss for eqa models that enhances their robustness against distribution shifts of unanswerable questions. with these novel approaches, we aim to extensively improve the robustness of models against both distribution shifts and adversarial attacks. ## models and tasks in extractive question answering (eqa), models are trained to identify the answer (a text span in the context) to the given question. the dataset may include unanswerable questions, for which a valid prediction is an "empty" answer. a common metric to evaluate mrc systems is f1-score. it measures the average overlap between the words in the predicted answer and the human-annotated gold answer. ## adversarial attacks in addition to evaluating models' robustness against distribution shift, we also measure the robustness against adversarial attacks. ## extractive question answering loss functions eqa models are typically fed a question q and a context c as input. state-of-the-art eqa models, often employing bert-style language models at their core, process q and c together as a sequence @xmath0, with [cls] and [sep] as special tokens of pre-trained tokenizer accompanying the pre-trained model. given an input sequence (pair of questioncontext) with n tokens @xmath1, t 2 , ..., t n ), we have one of the input tokens, encoding its contextual information. we then employ two single-layer feed-forward neural networks, denoted as s and e for predicting the start and end positions, respectively. both networks are designed to receive input vectors ⃗ v k and produce a scalar output. we then have that ## experiments 6.1 experiment design in the experiments in this section, we train our models using the squad 2.0 dataset. for models trained with the default loss function, the original squad 2.0 dataset is used without modifications. however, for models trained using our proposed method in this section, we introduce modifications to the squad 2.0 dataset to eliminate the single-answer assumption during the training phase. we augment approximately 20% of the answerable questions in the original dataset with an additional "synthetic" answer, resulting in these questions having two answers. in appendix b, we provide detailed information on how we generate "synthetic" answers, along with our experiments on the risks of hallucinations when training eqa models using these synthetic answers. ## further analysis 7.1 experiment design to evaluate the effectiveness of our proposed training method under different scenarios, we design two experiments. ## conclusion in this paper, we introduce a novel training methodology for eqa models aimed at enhancing their robustness against distribution shifts and adversarial attacks. our new training method is characterized by a novel training loss for the eqa problem, as well as challenging the single-answer assumption by creating a new "synthetic" answer span in a number of answerable questions. our experimental findings demonstrate that models trained using our approach exhibit significant improvement on out-of-domain testing datasets. furthermore, the robustness of these models against two tested types adversarial attacks is also significantly better than that of the default models. in section 7, we study the independent contributions of our new loss function and the augmented "synthetic" answers to the robustness of our models. our analysis reveals that the new loss function specifically benefits the performance on information-seeking unanswerable questions. this improved performance of information-seeking unanswerable questions contributes to the robustness against distribution shifts of models trained on squad 2.0 with our method. on the other hand, our training method challenges the single-answer assumption of many existing eqa datasets by creating "synthetic" answers for a number of answerable questions. our experiments indicate that these "synthetic" answers significantly contribute to the robustness of models trained with our method on both squad 2.0 and squad agent against adversarial attacks. this finding strongly corroborates our initial hypothesis, suggesting that the longstanding single-answer assumption of many eqa training datasets is a learning shortcut for models that can significantly compromise their robustness. we believe this work highlights the importance of future question answering datasets that incorporate the possibility of multiple, non-contiguous answer spans, similar to the multispanqa dataset @xcite .