Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval

1 INTRODUCTION . Open domain question answering is a challenging task where the answer to a given question needs to be extracted from a large pool of documents . The prevailing approach ( Chen et al. , 2017 ) tackles the problem in two stages . Given a question , a retriever first produces a list of k candidate documents , and a reader then extracts the answer from this set . Until recently , retrieval models were dependent on traditional term-based information retrieval ( IR ) methods , which fail to capture the semantics of the question beyond lexical matching and remain a major performance bottleneck for the task . Recent work on dense retrieval methods instead uses pretrained encoders to cast the question and documents into dense representations in a vector space and relies on fast maximum inner-product search ( MIPS ) to complete the retrieval . These approaches ( Lee et al. , 2019 ; Guu et al. , 2020 ; Karpukhin et al. , 2020 ) have demonstrated significant retrieval improvements over traditional IR baselines . However , such methods remain limited to simple questions , where the answer to the question is explicit in a single piece of text evidence . In contrast , complex questions typically involve aggregating information from multiple documents , requiring logical reasoning or sequential ( multihop ) processing in order to infer the answer ( see Figure 1 for an example ) . Since the process for answering such questions might be sequential in nature , single-shot approaches to retrieval are insufficient . Instead , iterative methods are needed to recursively retrieve new information at each step , conditioned on the information already at hand . Beyond further expanding the scope of existing textual open-domain QA systems , answering more complex questions usually involves multi-hop reasoning , which poses unique challenges for existing neural-based AI systems . With its practical ∗Equal Contribution 1https : //github.com/facebookresearch/multihop_dense_retrieval . and research values , multi-hop QA has been extensively studied recently ( Talmor & Berant , 2018 ; Yang et al. , 2018 ; Welbl et al. , 2018 ) and remains an active research area in NLP ( Qi et al. , 2019 ; Nie et al. , 2019 ; Min et al. , 2019 ; Zhao et al. , 2020 ; Asai et al. , 2020 ; Perez et al. , 2020 ) . The main problem in answering multi-hop open-domain questions is that the search space grows exponentially with each retrieval hop . Most recent work tackles this issue by constructing a document graph utilizing either entity linking or existing hyperlink structure in the underlying Wikipedia corpus ( Nie et al. , 2019 ; Asai et al. , 2020 ) . The problem then becomes finding the best path in this graph , where the search space is bounded by the number of hyperlinks in each passage . However , such methods may not generalize to new domains , where entity linking might perform poorly , or where hyperlinks might not be as abundant as in Wikipedia . Moreover , efficiency remains a challenge despite using these data-dependent pruning heuristics , with the best model ( Asai et al. , 2020 ) needing hundreds of calls to large pretrained models to produce a single answer . In contrast , we propose to employ dense retrieval to the multi-hop setting with a simple recursive framework . Our method iteratively encodes the question and previously retrieved documents as a query vector and retrieves the next relevant documents using efficient MIPS methods . With highquality , dense representations derived from strong pretrained encoders , our work first demonstrates that the sequence of documents that provide sufficient information to answer the multi-hop question can be accurately discovered from unstructured text , without the help of corpus-specific hyperlinks . When evaluated on two multi-hop benchmarks , HotpotQA ( Yang et al. , 2018 ) and a multi-evidence subset of FEVER ( Thorne et al. , 2018 ) , our approach improves greatly over the traditional linkingbased retrieval methods . More importantly , the better retrieval results also lead to state-of-the-art downstream results on both datasets . On HotpotQA , we demonstrate a vastly improved efficiencyaccuracy trade-off achieved by our system : by limiting the amount of retrieved contexts fed into downstream models , our system can match the best published result while being 10x faster . 2 METHOD . 2.1 PROBLEM DEFINITION . The retrieval task considered in this work can be described as follows ( see also Figure 1 ) . Given a multi-hop question q and a large text corpus C , the retrieval module needs to retrieve a sequence of passages Pseq : { p1 , p2 , ... , pn } that provide sufficient information for answering q . Practically , the retriever returns the k best-scoring sequence candidates , { P1seq , P2seq , ... , Pkseq } ( k |C| ) , with the hope that at least one of them has the desired qualities . k should be small enough for downstream modules to process in a reasonable time while maintaining adequate recall . In general , retrieval also needs to be efficient enough to handle real-world corpora containing millions of documents . 2.2 MULTI-HOP DENSE RETRIEVAL . Model Based on the sequential nature of the multi-hop retrieval problem , our system solves it in an iterative fashion . We model the probability of selecting a certain passage sequence as follows : P ( Pseq|q ) = n∏ t=1 P ( pt|q , p1 , ... , pt−1 ) , where for t = 1 , we only condition on the original question for retrieval . At each retrieval step , we construct a new query representation based on previous results and the retrieval is implemented as maximum inner product search over the dense representations of the whole corpus : P ( pt|q , p1 , ... , pt−1 ) = exp ( 〈pt , qt〉 ) ∑ p∈C exp ( 〈p , qt〉 ) , where qt = g ( q , p1 , ... , pt−1 ) and pt = h ( pt ) . Here 〈· , ·〉 is the inner product between the query and passage vectors . h ( · ) and and g ( · ) are passage and query encoders that produce the dense representations . In order to reformulate the query representation to account for previous retrieval results at time step t , we simply concatenate the question and the retrieved passages as the inputs to g ( · ) . Note that our formulation for each retrieval step is similar to existing single-hop dense retrieval methods ( Lee et al. , 2019 ; Guu et al. , 2020 ; Karpukhin et al. , 2020 ) except that we add the query reformulation process conditioned on previous retrieval results . Additionally , instead of using a bi-encoder architecture with separately parameterized encoders for queries and passages , we use a shared RoBERTa-base ( Liu et al. , 2019 ) encoder for both h ( · ) and g ( · ) . In §3.1.3 , we show this simple modification yields considerable improvements . Specifically , we apply layer normalization over the start token ’ s representations from RoBERTa to get the final dense query/passage vectors . Training and Inference The retriever model is trained as in Karpukhin et al . ( 2020 ) , where each input query ( which at each step consists of a question and previously retrieved passages ) is paired with a positive passage and m negative passages to approximate the softmax over all passages . The positive passage is the gold annotated evidence at step t. Negative passages are a combination of passages in the current batch which correspond to other questions ( in-batch ) , and hard negatives which are false adversarial passages . In our experiments , we obtain hard negatives from TF-IDF retrieved passages and their linked pages in Wikipedia . We note that using hyperlinked pages as additional negatives is neither necessary nor critical for our approach . In fact we observe only a very small degradation in performance if we remove them from training ( §3.1.3 ) . In addition to in-batch negatives , we use a memory bank ( M ) mechanism ( Wu et al. , 2018 ) to further increase the number of negative examples for each question . The memory bank stores a large number of dense passage vectors . As we block the gradient back-propagation in the memory bank , its size ( |M| batch size ) is less restricted by the GPU memory size . Specifically , after training to convergence with the shared encoder , we freeze a copy of the encoder as the new passage encoder and collect a bank of passage representations across multiple batches to serve as the set of negative passages . This simple extension results in further improvement in retrieval . ( §3.1.3 ) . For inference , we first encode the whole corpus into an index of passage vectors . Given a question , we use beam search to obtain top-k passage sequence candidates , where the candidates to beam search at each step are generated by MIPS using the query encoder at step t , and the beams are scored by the sum of inner products as suggested by the probabilistic formulation discussed above . Such inference relies only on the dense passage index and the query representations , and does not need explicit graph construction using hyperlinks or entity linking . The top-k sequences will then be fed into task-specific downstream modules to produce the desired outputs . 3 EXPERIMENTS . Datasets Our experiments focus on two datasets : HotpotQA and Multi-evidence FEVER . HotpotQA ( Yang et al. , 2018 ) includes 113k multi-hop questions . Unlike other multi-hop QA datasets ( Zhang et al. , 2018 ; Talmor & Berant , 2018 ; Welbl et al. , 2018 ) , where the information sources of the answers are knowledge bases , HotpotQA uses documents in Wikipedia . Thus , its questions are not restricted by the fixed KB schema and can cover more diverse topics . Each question in HotpotQA is also provided with ground truth support passages , which enables us to evaluate the intermediate retrieval performance . Multi-evidence FEVER includes 20k claims from the FEVER ( Thorne et al. , 2018 ) fact verification dataset , where the claims can only be verified using multiple documents . We use this dataset to validate the general applicability of our method . Implementation Details All the experiments are conducted on a machine with 8 32GB V100 GPUs . Our code is based on Huggingface Transformers ( Wolf et al. , 2019 ) . Our best retrieval results are predicted using the exact inner product search index ( IndexFlatIP ) in FAISS ( Johnson et al. , 2017 ) . Both datasets assume 2 hops , so we fix n = 2 for all experiments . Since HotpotQA does not provide the order of the passage sequences , as a heuristic , we consider the passage that includes the answer span as the final passage . 2 In §3.1.3 , we show that the order of the passages is important for effective retriever training . The hyperparameters can be found in Appendix B.1 . 3.1 EXPERIMENTS : RETRIEVAL . We evaluate our multi-hop dense retriever ( MDR ) in two different use cases : direct and reranking , where the former outputs the top-k results directly using the retriever scores and the latter applies a task-specific reranking model to the initial results from MDR . 3.1.1 DIRECT . We first compare MDR with several efficient retrieval methods that can directly find the top-k passage sequences from a large corpus , including TF-IDF , TF-IDF + Linked , DrKIT and Entity Linking . TFIDF is the standard term-matching baseline , while TF-IDF + Linked is a straightforward extension that also extracts the hyperlinked passages from TF-IDF passages , and then reranks both TF-IDF and hyperlinked passages with BM25 3 scores . DrKIT ( Dhingra et al. , 2020 ) is a recently proposed dense retrieval approach , which builds a entity-level ( mentions of entities ) dense index for retrieval . It relies on hyperlinks to extract entity mentions and prunes the search space with a binary mask that restricts the next hop to using hyperlinked entities . On FEVER , we additionally consider an entity linking baseline ( Hanselowski et al. , 2018 ) that is commonly used in existing fact verification pipelines . This baseline first uses a constituency parser to extract potential entity mentions in the fact claim and then uses the MediaWiki API to search documents with titles that match the mentions . Table 1 shows the performance of different retrieval methods . On HotpotQA the metric is recall at the top k paragraphs4 , while on FEVER the metrics are precision , recall and F1 in order to be consistent with previous results . On both datasets , MDR substantially outperforms all baselines .

Summary: This paper proposes multi-hop dense retrieval for open-domain multi-hop question answering. It extends previous dense passage retrieval into the corresponding multi-hop version by using retrieved passages to latently reformulate the query representation after each retrieval pass. In the end, it can significantly improve the performance on HotpotQA and multi-evidence FEVER dataset. The analyses are very comprehensive and extensive from almost every relevant perspective.