Robust Question Answering against Distribution Shifts with Test-Time Adaption: An Empirical Study (2022)

Abstract: A deployed question answering (QA) model can easily fail when the test data has a distribution shift compared to the training data. Robustness tuning (RT) methods have been widely studied to enhance model robustness against distribution shifts before model deployment. However, can we improve a model after deployment? To answer this question, we evaluate test-time adaptation (TTA) to improve a model after deployment. We first introduce ColdQA, a unified evaluation benchmark for robust QA against text corruption and changes in language and domain. We then evaluate previous TTA methods on ColdQA and compare them to RT methods. We also propose a novel TTA method called online imitation learning (OIL). Through extensive experiments, we find that TTA is comparable to RT methods, and applying TTA after RT can significantly boost the performance on ColdQA. Our proposed OIL improves TTA to be more robust to variation in hyper-parameters and test distributions over time.

## introduction how to build a trustworthy nlp system that is robust to distribution shifts is important, since the real world is changing dynamically and a system can easily fail when the test data has a distribution shift compared to the training data @xcite . much previous work on robustness evaluation has found model failures on shifted test data. for example, question answering (qa) models are brittle when dealing with paraphrased questions @xcite , models for task-oriented dialogues fail to understand corrupted input @xcite , and neural machine translation degrades on noisy text input @xcite . in this 1 our source code is available at https://github.com/ oceanypt/coldqa-tta 1 predict 2 update work, we study the robustness of qa models to out-of-distribution (ood) test-time data. to build a model that is robust against distribution shifts, most previous work focuses on robustness tuning (rt) methods that improve model generalization pre-deployment, such as adversarial training @xcite . however, can we continually enhance a model post-deployment? to answer this question, we study and evaluate testtime adaptation (tta) for robust qa after model deployment. tta generalizes a model by continually updating the model with test-time data @xcite . as shown in fig. 1 , in this work, we focus on test-time adaptation in real time, where the model predicts and updates over a data stream on the fly. for each test data instance, the model first returns its prediction and then updates itself with the test data. unlike unsupervised domain adaptation @xcite studied in nlp, tta is suitable for domain generalization, since it makes no assumption about the target distribution and could adapt the model to any arbitrary distribution at test time. to clone the actions made by the source model, and the source model aims to reduce overfitting to noisy pseudo-labels in the adapted model. we further adopt causal inference to control model bias from the source model. next, to compare to tta methods, we briefly discuss previous robustness tuning (rt) methods such as adversarial training in §4. to study and analyze tta for robust qa postdeployment, we introduce coldqa in §5 which is a unified evaluation benchmark for robust qa against distribution shifts from text corruption, language change, and domain change. it differs from previous benchmarks that only study one type of distribution shifts @xcite @xcite . coldqa expects a qa model to generalize well to all three types of distribution shifts. our contributions in this work include: based on the experimental results in §6, we report the following findings: test-time adaptation tta adapts a source model with test-time data from a target distribution. tta has been verified to be very effective in image recognition @xcite @xcite . in nlp, wang et al. (2021d) learn to combine adapters on low-resource languages at test time to improve sequence labeling tasks. gao et al. (2022) and li et al. (2022) keep adapting a qa model after model deployment using user feedback, which is different from our work which requires no human involvement when adapting the model. ben-david et al. (2022) study test-time adaptation for text classification and sequence labeling, but they focus on example-based prompt learning which needs expert knowledge to design prompts. banerjee et al. (2021) explore test-time learning for qa tasks, but their work concerns how to train a qa model from scratch by using unlabeled test data, instead of adapting to out-of-distribution test data. domain adaptation different from test-time adaptation, unsupervised domain adaptation (uda) needs to know the target domain when performing adaptation pre-deployment (ben-david et al., 2010; @xcite @xcite . uda tries to minimize the gap between the source and target domain. recent work studies uda without knowing the source domain @xcite , which means the model can be adapted to any unseen target domain on the fly. however, they assume all target data is available when performing adaptation, unlike online adaptation. robustness tuning robustness tuning (rt) is another family of methods that tries to train a more generalized model pre-deployment at training time instead of test time. adversarial training is a wellstudied method to enhance model robustness @xcite @xcite . some work also uses regularization to improve model generalization @xcite @xcite . prompt tuning @xcite and adapter-based tuning @xcite can also enhance model generalization to unseen test distributions. ## robustness tuning in contrast to improving the model postdeployment with tta, robustness tuning (rt) enhances the model pre-deployment. rt has been studied in nlp to improve model generalization @xcite . rt methods are applied at training time when training the source model. we also benchmark rt methods on coldqa to compare with tta methods. first, we compare with adversarial training methods, which are fgm @xcite , pgd @xcite , freelb (zhu et al., 2020), and infobert @xcite . next, we further evaluate robustness tuning methods proposed for cross-lingual transfer, which are mvr @xcite and xtune @xcite . these two methods use regularization to enhance model robustness. all of these methods have not been comprehensively evaluated on distribution shifts arising from text corruption, language change, and domain change. combination of rt and tta. finally, we also study combining rt and tta methods. the source model is tuned by a rt method, then this model is adapted by a tta method to the test distribution. ## coldqa to study robust qa under distribution shifts, in this work we introduce coldqa, a unified evaluation benchmark against text corruption, language change, and domain change. as shown in ## conclusion we study test-time adaptation (tta) for robust question answering under distribution shifts. a unified evaluation benchmark, coldqa, over text corruption, language change, and domain change is provided. a novel tta method, oil, is proposed that achieves good performance when combined with a robustness tuning method. hai ye, qingyu tan, ruidan he, juntao li, hwee tou ng, and lidong bing. 2020. feature adaptation of pre-trained language models across languages and domains with robust self-training.