Robustifying Multi-hop QA through Pseudo-Evidentiality Training (2021)

Abstract: This paper studies the bias problem of multi-hop question answering models, of answering correctly without correct reasoning. One way to robustify these models is by supervising to not only answer right, but also with right reasoning chains. An existing direction is to annotate reasoning chains to train models, requiring expensive additional annotations. In contrast, we propose a new approach to learn evidentiality, deciding whether the answer prediction is supported by correct evidences, without such annotations. Instead, we compare counterfactual changes in answer confidence with and without evidence sentences, to generate “pseudo-evidentiality” annotations. We validate our proposed model on an original set and challenge set in HotpotQA, showing that our method is accurate and robust in multi-hop reasoning.

## introduction multi-hop question answering (qa) is a task of answering complex questions by connecting information from several texts. since the information is spread over multiple facts, this task requires to capture multiple relevant facts (which we refer as evidences) and infer an answer based on all these evidences. however, previous works @xcite @xcite observe "disconnected reasoning" in some correct answers. it happens when models can exploit specific types of artifacts (e.g., entity type), to leverage them as reasoning shortcuts to guess the correct answer. for example, assume that a given question is: "which country got independence when world war ii ended?" and a passage is: "korea got independence in 1945". although information ("world war ii ended in 1945") is insufficient, qa models * correspond to seungwonh@snu.ac.kr predict "korea", simply because its answer type is country (or, using shortcut). to address the problem of reasoning shortcuts, we propose to supervise "evidentiality" -deciding whether a model answer is supported by correct evidences (see figure 1 ). this is related to the problem that most of the early reader models for qa failed to predict whether questions are not answerable. lack of answerability training led models to provide a wrong answer with high confidence, when they had to answer "unanswerable". similarly, we aim to train for models to recognize whether their answer is "unsupported" by evidences, as well. in our work, along with the answerability, we train the qa model to identify the existence of evidences by using passages of two types: (1) evidence-positive and (2) evidence-negative set. while the former has both answer and evidence, the latter does not have evidence supporting the answer, such that we can detect models taking shortcuts. our first research question is: how do we acquire evidence-positive and negative examples for training without annotations? for evidence-positive set, the closest existing approach @xcite is to consider attention scores, which can be considered as pseudo-annotation for evidence-positive set. in other word, sentence s with high attention scores, often used as an "interpretation" of whether s is causal for model prediction, can be selected to build evidence-positive set. however, follow-up works @xcite argued that attention is limited as an explanation, because causality cannot be measured, without observing model behaviors in a counterfactual case of the same passage without s. in addition, sentence causality should be aggregated to measure group causality of multiple evidences for multi-hop reasoning. to annotate group causality as "pseudo-evidentiality", we propose interpreter module, which removes and aggregates evidences into a group, to compare predictions in observational and counterfactual cases. as a second research question, we ask how to learn from evidence-positive and evidencenegative set. to this end, we identify two objectives: (o1) qa model should not be overconfident in evidence-negative set, while (o2) confident in evidence-positive. a naive approach to pursue the former is to lower the model confidence on evidence-negative set via regularization. however, such regularization can cause violating (o2) due to correlation between confidence distributions for evidence-positive and negative set. our solution is to selectively regularize, by purposedly training a biased model violating (o1), and decorrelate the target model from the biased model. for experiments, we demonstrate the impact of our approach on hotpotqa dataset. our empirical results show that our model can improve qa performance through pseudo-evidentiality, outperforming other baselines. in addition, our proposed approach can orthogonally combine with another sota model for additional performance gains. ## related work since multi-hop reasoning tasks, such as hot-potqa, are released, many approaches for the task have been proposed. these approaches can be cat-egorized by strategies used, such as graph-based networks @xcite , external knowledge retrieval @xcite , and supporting fact selection @xcite . our focus is to identify and alleviate reasoning shortcuts in multi-hop qa, without evidence annotations. models taking shortcuts were widely observed from various tasks, such as object detection @xcite , nli @xcite , and also for our target task of multi-hop qa @xcite @xcite , where models learn simple heuristic rules, answering correctly but without proper reasoning. to mitigate the effect of shortcuts, adversarial examples @xcite can be generated, or alternatively, models can be robustifed @xcite with additional supervision for paragraph-level "sufficiency" -to identify whether a pair of two paragraphs are sufficient for right reasoning or not, which reduces shortcuts on a single paragraph. while the binary classification for paragraph-sufficiency is relatively easy (96.7 f1 in @xcite ), our target of capturing a finer-grained sentence-evidentiality is more challenging. existing qa model @xcite treats this as a supervised task, based on sentence-level human annotation. in contrast, ours requires no annotation and focuses on avoiding reasoning shortcuts using evidentiality, which was not the purpose of evidence selection in the existing model. ## proposed approach in this section, to prevent reasoning shortcuts, we introduce a new approach for data acquiring and learning. we describe this task (section 3.1) and address two research questions, of generating labels for supervision (section 3.2) and learning (section 3.3), respectively. ## experiment in this section, we formulate our research questions to guide our experiments and describe evaluation results corresponding to each question. research questions to evaluate the effectiveness of our method, we address the following research questions: implementation our implementation settings for qa model follow roberta (base version with 12 layers) @xcite . we use the adam optimizer with a learning rate of 0.00005 and a batchsize of 8 on rtx titan. we extract the evidencepositive set after 3 epoch (@xmath0. ( 11 )) and retrain for 3 epochs. as a hyper-parameter, we search λ among {1, 0.1, 0.01}, and found the best value (λ=0.01), based on 5% hold-out set sampled from the training set. metrics we report standard f1 score for hot-potqa, to evaluate the overall qa accuracy to find the correct answers. for evidence selection, we also report f1 score, precision, and recall to evaluate the sentence-level evidence retrieval accuracy. ## conclusion in this paper, we propose a new approach to train multi-hop qa models, not to take reasoning shortcuts of guessing right answers without sufficient evidences. we do not require annotations and generate pseudo-evidentiality instead, by regularizing qa model from being overconfident when evidences are insufficient. our experimental results show that our method outperforms baselines on hotpotqa and has the effectiveness to distinguish between evidence-positive and negative set.