Dense Hierarchical Retrieval for Open-domain Question Answering (2021)

Abstract: Dense neural text retrieval has achieved promising results on open-domain Question Answering (QA), where latent representations of questions and passages are exploited for maximum inner product search in the retrieval process. However, current dense retrievers require splitting documents into short passages that usually contain local, partial and sometimes biased context, and highly depend on the splitting process. As a consequence, it may yield inaccurate and misleading hidden representations, thus deteriorating the final retrieval result. In this work, we propose Dense Hierarchical Retrieval (DHR), a hierarchical framework which can generate accurate dense representations of passages by utilizing both macroscopic semantics in the document and microscopic semantics specific to each passage. Specifically, a document-level retriever first identifies relevant documents, among which relevant passages are then retrieved by a passage-level retriever. The ranking of the retrieved passages will be further calibrated by examining the document-level relevance. In addition, hierarchical title structure and two negative sampling strategies (i.e., In-Doc and In-Sec negatives) are investigated. We apply DHR to large-scale open-domain QA datasets. DHR significantly outperforms the original dense passage retriever, and helps an end-to-end QA system outperform the strong baselines on multiple open-domain QA benchmarks.

## introduction the goal of open-domain question answering (qa) is to answer a question without pre-specified source domain @xcite . one of the most prevalent architectures in open-domain qa is the retriever-reader approach @xcite . given a question, the task figure 1 : an example of distracting passages in natural question @xcite . the first dpr retrieved passage shares similar semantics with the gold passage. the document title of the second dpr retrieved passage matches most question tokens. both of the retrieved passages tend to result in a wrong answer. pretrained transformer models, such as bert @xcite , are widely used in recent studies on the retriever-reader framework @xcite @xcite . to serve as input to the retriever, documents are split into short passages, and in the dense passage retrieval, dpr @xcite , a dual encoder framework is applied to encode questions and the split passages separately. state-of-the-art dense retrievers outperform sparse term-based retrievers, like bm25 @xcite , but they suffer from several weaknesses. first, due to the lack of effective pruning strategy, extracting relevant passages from a large corpus undergoes an efficiency issue especially in the inference time. second, given a question, many passages may comprehend similar topics with subtle semantic difference. this fact requires the retriever and the reader to encode passages to their accurate semantic representations, which is an overwhelmed task. moreover, passages contain only local and specific information, thus easily leading to distracting representations. as illustrated in figure 1 , distracting passages with similar semantics may lead to a wrong answer. to alleviate these issues, we present a dense hierarchical retriever (dhr) framework, which consists of a dense document-level retriever and a dense passage-level retriever. document-level retriever aims at capturing coarse-grained semantics of documents in the sense that the embeddings of questions and their relevant documents are positively correlated. the goal of document-level retriever is to prune answer-irrelevant documents and returns relevant ones, which will serve as a refined corpus to feed into passage-level retriever. given relevant documents consisting of passages of similar topics, the passage-level retriever intends to identify the essential evidences, which may contribute to a correct answer. in order to empower dhr, we formalize the hierarchical information of documents as a tree structure with two types of nodes, title nodes and content nodes. then, a document is easily represented by its document summary, and a passage is represented by a hierarchical title list concatenated with its content. the benefit of using the hierarchical approach and exploiting the hierarchical information is three-fold: 1) coarse-grained information explicitly or implicitly covered in the document will guide the passage-level retriever to deviate from a fallacious embedding function; 2) passage-level retriever, a fine-grained component, will provide essential capability of identifying the necessary relevant evidences among similar passages; 3) document-level retriever prunes substantial amount of irrelevant and peripheral documents, and triggers a much faster inference. to further enhance the ability of the passage-level retriever in detecting gold passages among similar passages, we propose two negative sampling strategies (i.e., in-doc and in-sec negative sampling). our main contributions are summarized as: 1) we propose a hierarchical dense retrieval on opendomain qa and achieve a fast inference speed with high retrieval precision; 2) the hierarchical information is used in a more structural way, which leads to a meaningful and global passage representation consistent with its document; 3) we conduct comprehensive experiments with state-of-the-art approaches on multiple open-domain qa datasets. our empirical results demonstrate that we achieve comparable or better results in the open-retrieval setting. extensive ablation studies on various components and strategies are conducted. ## dense hierarchical retrieval (dhr) this section presents our dense hierarchical retrieval (dhr) model, which consists of a dense document-level retrieval (dhr-d) and a dense passage-level retrieval (dhr-p). figure 3 shows an overview of our proposed method. ## experimental setup in this section, we describe the dataset and basic setup for experiments. ## experiments in this section, we evaluate the performance of our dense hierarchical retriever (dhr 1 ), along with analysis on how each component affects the results and the retrieval time efficiency. for the retrieval implementation detail, please refer to appendix a.