Distantly-Supervised Dense Retrieval Enables Open-Domain Question Answering without Evidence Annotation (2021)

Abstract: Open-domain question answering answers a question based on evidence retrieved from a large corpus. State-of-the-art neural approaches require intermediate evidence annotations for training. However, such intermediate annotations are expensive, and methods that rely on them cannot transfer to the more common setting, where only question–answer pairs are available. This paper investigates whether models can learn to find evidence from a large corpus, with only distant supervision from answer labels for model training, thereby generating no additional annotation cost. We introduce a novel approach (DistDR) that iteratively improves over a weak retriever by alternately finding evidence from the up-to-date model and encouraging the model to learn the most likely evidence. Without using any evidence labels, DistDR is on par with fully-supervised state-of-the-art methods on both multi-hop and single-hop QA benchmarks. Our analysis confirms that DistDR finds more accurate evidence over iterations, which leads to model improvements. The code is available at https://github.com/henryzhao5852/DistDR .

## introduction open-domain question answering (odqa) takes a question, retrieves evidence from a large corpus, and finds an answer based on that evidence @xcite . with the help of large scale datasets, state-of-the-art approaches to qa @xcite alia) can answer both simple questions that require only a single evidence piece (i.e., one passage); and more challenging multi-hop questions: computers must jump or "hop" from passage to passage (we call these passages evidence pieces), building a reasoning chain to find the answer. figure 1 : a multi-hop question example from hot-potqa that requires finding multiple evidence pieces to form a reasoning chain (sang-wook @xmath0. red: text that overlaps between question and evidence piece; blue: span that matches the answer. state-of-the-art systems use evidence labels for training, but acquiring labeled evidence pieces is expensive. state-of-the-art (sota) methods, however, are trained with all of the intermediate evidence pieces (e.g., in figure 1 , the evidence pieces for sang-wook cheong's workplace which point you to rutgers university's location) needed for the answer. creating such intricate training data is expensive. for example, @xcite use additional experts to justify the correctness of annotated evidence. the annotation protocol is even more nuanced for multi-hop questions. for example, @xcite ask annotators to write multi-hop questions based on two linked wikipedia passages as a pre-defined reasoning chain, which creates dataset artifacts @xcite . while plenty of question-answer pairs are available without evidence labels, we cannot directly train sota models on such data. our work focuses on training odqa systems without these expensive annotations (section 2): we only start with a question-answer pair. with that starting point, we use distant supervision to infer which evidence helps us get to the answer. the technical challenge is how to find these evidence from millions of candidates. previous methods @xcite use term matching (e.g., tf-idf) for evidence retrieval, but their goal is a single piece of evidence: linking a question to a passage. as shown in figure 1 , the key to finding some evidence pieces does not appear in the question: for example, you only know to figure out that rutgers university is in new jersey after learning where professor cheong works. fortunately, navigating to an answer given a question from a search engine is not impossible: humans do it every day, building on their existing knowledge toward the answer @xcite . with each round of searching to find additional clues, human users accumulate information to find the right answer. this paper creates a computational approach (distdr) that can use similar techniques to find the evidence needed to answer a given question. distdr starts with a weak retriever then iteratively improves it by finding more useful evidence (section 3). specifically, we model evidence as a latent variable and develop a hard-em algorithm that alternates between using the up-to-date retriever to find evidence (hard e-step) and updating the model parameters to further encourage the most useful evidence in the next iteration (m-step). to implement this idea, we need a trainable retrieval system. we use dense retrieval @xcite as our retriever, which uses a neural network to encode the evidence pieces we collect at each round into query vectors. distdr provides iterative feedback within the context of a qa system to guide the encoder to better find evidence pieces. we evaluate distdr on odqa benchmarks, including both single-hop questions @xcite , where the evidence is the target passage, and multi-hop questions @xcite , where the evidence is a chain of passages. without using any annotated evidence labels, distdr's accuracy, according to several measures, is on par with fully-supervised state-of-the-art approaches on both benchmarks (section 4). our analyses confirm the intuition that over iterations, distdr selects more accurate evidence, which in turn improves the model. although some retrieved evidence from distdr does not match the annotation, it gives useful training signal, as distdr finds alternative evidence-another advantage of an automated approach (section 5). for example, you can connect sang-wook cheong to new jersey through another wikipedia page (history of rutgers university in figure 1 ). state-of-the-art approaches on odqa mainly focus on the fully-supervised setting, where the question, answer, and evidence are given in training. figure 1 shows an example multi-hop question with answer and annotated evidence. the fullysupervised setting simplifies model training but has three major challenges: (1) annotation is costly: the question-answer pair is widely available in the real world, but the evidence (sang-wook @xmath1, which is expensive to get, especially for complex questions; (2) domain generalization: the labeled evidence is only on a single corpus, generalization to other corpora (e.g., moving from wikipedia to medical records) is nontrivial; (3) alternative evidence: there are often multiple correct evidence candidates in the corpora (e.g., in figure 1 , both sang-wook @xmath2, but only one of them is annotated as "gold evidence". therefore, we explore the more common but challenging weakly-supervised setting which only requires question-answer input pairs. formally, given a question q and a textual corpus with d passages, our task is to first find a small subset of relevant passages as evidence z to the answer, where each evidence combines evidence pieces @xmath3, ..., z n with length n (@xmath4, then find a span a from evidence z as an answer. we focus on the weaklysupervised setting, where training examples consist only of question-answer pairs (q, a), we gather evidence ẑ from corpus as the distant supervision signal, based on the assumption that the presence of the answer a (source of distant supervision) in an evidence piece implies that the evidence is needed to answer the question. this is contrast to the fullysupervised setting, where the gold evidence z * is also given during training. we present distdr, a unified framework for weakly-supervised odqa. distdr is trained by retrieving evidence from a large corpus with distant supervision. distdr follows the retriever-reader framework for odqa, using dense retrieval to find evidence (section 3.1). however, in our approach, we only have questions q and answers a, so we need to induce evidence ẑ for training our retriever. we expect our initial retriever will struggle to find evidence. however, if it can find some useful evidence, we can encourage it to follow the clues to more evidence that can answer questions. this intuition is the foundation for our iterative approach (section 3.2) for evidence retrieval: an initial retriever attempts to find evidence (dense retrieval). if the evidence contains the answer (answer matching), we label it as positive evidence (otherwise it's negative evidence); then we use the retrieved evidence as labels to retrain the retriever and reader (model update). while this idea forms the basis of our algorithm, it can be led astray by false positives: evidence that contains the answer but is irrelevant to question. returning to our running example, while "named after a lawn ornament store in wayne, new jersey" has the state where professor cheong works, it is irrelevant to condensed matter physics. thus, a reader filters spurious evidence (reader filter) to keep distdr on target. ## analysis this section explores hotpotqa results to understand why distdr is on par with fully-supervised approaches. ## related work question answering datasets there is growing interest in nlp communities to build large-scale datasets @xcite @xcite for qa research. in addition to questions and answers, benchmark datasets often include annotated evidence, but it requires significant annotation protocol design and human annotations. in squad @xcite , among the first large-scale reading comprehension datasets, annotators write questions conditioned on a passage, which creates dataset artifacts @xcite . to overcome dataset artifacts, naturalquestions @xcite use real google queries as questions and ask annotators to label both evidence passages and short answers. but such annotation is expensive, as they further ask additional experts to verify whether the evidence correctly leads to the answer. instead, distdr focuses on the weakly-supervised setting with only question-answer pairs, which is significantly cheaper. annotation is more fraught for multi-hop qa datasets @xcite . to construct hot-potqa, annotators are presented with a linked wikipedia passages, as pilot studies indicate that it is difficult to ask a meaningful multi-hop question with arbitrary passages. however, some questions in hotpotqa include shortcuts that are answerable by a single passage @xcite , which is confirmed by our analysis (section 5.2). distant supervision has been successfully adopted for many nlp tasks such as relation extraction @xcite . recent work builds qa datasets with distant supervision, such as triv-iaqa @xcite , searchqa @xcite , qblink @xcite by automatically gathering evidence documents from a corpus as distant supervision for available questionanswer pairs. they use standard ir techniques to find relevant passages and match them with answer strings. these methods succeed for simple questions where terms overlap with evidence passages, this no longer holds for multi-hop questions that require a reasoning chain as evidence to the answer: evidence pieces do not overlap with the question but rather depend on the previous evidence pieces. unsupervised ir methods cannot capture such implicit relations. distdr removes the burden of little textual overlap through dense retrieval and its iterative process retrieves better evidence. @xcite first combine information retrieval and (neural) reading comprehension for open-domain qa. several works aim to improve the neural reader @xcite , or use generative models to compose an answer @xcite . recent progress @xcite , inter alia) uses dense retrieval to aid both single-hop and multi-hop questions. however, a crucial distinction is that these approaches assume the evidence is given for training, while distdr iteratively finds evidence and uses it to improve the model. @xcite al. (2019a) also use hard-em for weakly-supervised qa, butorthogonal to our approach-they assume the evidence is given and find the most likely answer mentions in the evidence, while we aim to find evidence from a large corpus. ## conclusion we present distdr, a distantly-supervised odqa system that improves over a weak retriever by iter-atively finding evidence from a corpus, and using the evidence as distant supervision for model training. without using any evidence labels, distdr matches the fully-supervised sota approaches on both multi-hop and single-hop qa benchmarks. annotating evidence for existing questionanswer pairs is generally expensive, especially for complex questions. while distdr can accurately find evidence for arbitrary complex machine reading-style questions, future work needs to validate whether this can work for other types of questions. this could improve the reader to answer numerical reasoning @xcite , temporal reasoning @xcite , multi-model reasoning @xcite , or combination of these skills @xcite .