Adaptive Information Seeking for Open-Domain Question Answering (2021)

Abstract: Information seeking is an essential step for open-domain question answering to efficiently gather evidence from a large corpus. Recently, iterative approaches have been proven to be effective for complex questions, by recursively retrieving new evidence at each step. However, almost all existing iterative approaches use predefined strategies, either applying the same retrieval function multiple times or fixing the order of different retrieval functions, which cannot fulfill the diverse requirements of various questions. In this paper, we propose a novel adaptive information-seeking strategy for open-domain question answering, namely AISO. Specifically, the whole retrieval and answer process is modeled as a partially observed Markov decision process, where three types of retrieval operations (e.g., BM25, DPR, and hyperlink) and one answer operation are defined as actions. According to the learned policy, AISO could adaptively select a proper retrieval action to seek the missing evidence at each step, based on the collected evidence and the reformulated query, or directly output the answer when the evidence set is sufficient for the question. Experiments on SQuAD Open and HotpotQA fullwiki, which serve as single-hop and multi-hop open-domain QA benchmarks, show that AISO outperforms all baseline methods with predefined strategies in terms of both retrieval and answer evaluations.

## introduction open-domain question answering (qa) @xcite is a task of answering questions using a large collection of texts (e.g., wikipedia). it relies on a powerful information-seeking method to efficiently retrieve evidence from the given large corpus. traditional open-domain qa approaches mainly follow the two-stage retriever-reader pipeline @xcite ; karpukhin *corresponding author strategies: 1 bm25(golden retriever) bm25(𝑄) p1 bm25([𝑄, 𝑃1]) ✗ bm25(𝑄) p2 bm25([𝑄, 𝑃2]) ✗ 2 dr(mdr) dr(𝑄) p3 dr([𝑄, 𝑃3]) ✗ 3 bm25+link(grr) bm25(𝑄) p1 link(𝑃1) p2 link(𝑃2) ✗ optimal bm25(𝑄) p1 link(𝑃1) p2 dr([𝑄, 𝑃2]) p3 ans([𝑄, 𝑃2, 𝑃3]) ✓ 65 >1k >1k 3 >1k passages: p1: pitof jean-christophe "pitof" comar is a french director notable for "catwoman[link to p2]" and "vidocq[link to px]". in 2004, pitof made his english debut with the hollywood film catwoman … p2: catwoman (film) catwoman is a 2004 american action superhero film loosely based on the dc comics character of the same name directed by pitof[link to p1] … p3: catwoman (video game) catwoman is an action-adventure tie-in video game based on the 2004 film of the same name[link to p2] based on the fictional character … question: what movie directed by pitof in 2004 has a tie-in electronic game? figure 1 : an example derived from hotpotqa development set. p1, p2 and p3 are the most relevant passages, of which p2 and p3 are supporting passages, which are essential to answer the question. except for the adaptive strategy in the last row, fixed strategy methods such as using bm25 or dense retrieval multiple times and first using bm25 and then entity linking have failed, due to the rank of the remaining supporting passages larger than 1k. the number between two arrows indicates the highest rank of the remaining supporting passages in the retrieval list, unless ranked first. to tackle this issue, iterative approaches have been proposed to recurrently retrieve passages and reformulate the query based on the original question and the previously collected passages. nevertheless, all of these approaches adopt fixed information-seeking strategies in the iterative process. for example, some works employ a single retrieval function multiple times @xcite @xcite , and the other works use a pre-defined sequence of retrieval functions @xcite . however, the fixed information-seeking strategies cannot meet the diversified requirements of various problems. taking figure 1 as an example, the answer to the question is 'catwoman' in p3. due to the lack of essential supporting passages, simply applying bm25/dense retrieval (dr) multiple times (strategy 1 @xcite or 2 @xcite ), or using the mixed but fixed strategy (strategy 3 @xcite )) cannot answer the question. specifically, it is hard for @xcite to generate the ideal query 'catwoman game' by considering p1 or p2, thus bm25 @xcite suffers from the mismatch problem and fails to find the next supporting passage p3. the representation learning of salient but rare phrases (e.g. 'pitof') still remains a challenging problem @xcite , which may affect the effectiveness of dense retrieval, i.e., the supporting passage p3 is ranked 65, while p1 and p2 do not appear in the top-1000 list at the first step. furthermore, link retrieval functions fail when the current passage, e.g., p2, has no valid entity links. motivated by the above observations, we propose an adaptive information-seeking approach for open-domain qa, namely aiso. firstly, the task of open-domain qa is formulated as a partially observed markov decision process (pomdp) to reflect the interactive characteristics between the qa model (i.e., agent) and the intractable largescale corpus (i.e., environment). the agent is asked to perform an action according to its state (belief module) and the policy it learned (policy module). specifically, the belief module of the agent maintains a set of evidence to form its state. moreover, there are two groups of actions for the policy module to choose, 1) retrieval action that consists of the type of retrieval function and the reformulated query for requesting evidence, and 2) answer action that returns a piece of text to answer the question, then completes the process. thus, in each step, the agent emits an action to the environment, which returns a passage as the observation back to the agent. the agent updates the evidence set and generates the next action, step by step, until the evidence set is sufficient to trigger the answer action to answer the question. to learn such a strategy, we train the policy in imitation learning by cloning the behavior of an oracle online, which avoids the hassle of designing reward functions and solves the pomdp in the fashion of supervised learning. our experimental results show that our approach achieves better retrieval and answering performance than the state-of-the-art approaches on squad open and hotpotqa fullwiki, which are the representative single-hop and multi-hop datasets for open-domain qa. furthermore, aiso significantly reduces the number of reading steps in the inference stage. in summary, our contributions include: ## related work traditional approaches of open-domain qa mainly follow the two-stage retriever-reader pipeline @xcite : a retriever first gathers relevant passages as evidence candidates, then a reader reads the retrieved candidates to form an answer. in the retrieval stage, most approaches employ a determinate retrieval function and treat each passage independently @xcite @xcite @xcite @xcite @xcite . as an extension, some approaches further consider the relations between passages through hyperlinks or entity links and extend evidence with the linked neighbor passages @xcite @xcite . however, pipeline approaches retrieve evidence independently from reader, leading to 1) introduce less-relevant evidence to the question, and 2) hard to model the complex question which has high-order relationship between question and evidence. instead, recent iterative approaches sequentially retrieve new passages by updating the query inputted to a specific retrieval function at each step, conditioned on the information already gathered. at each step, @xcite ; feldman and el-yaniv (2019); xiong et al. (2021) reformulate the dense query vector in a latent space, while ding et al. (2019); qi et al. (2019); zhang et al. (2020); ## method in this section, we first formulate the open-domain qa task as a partially observed markov decision process (pomdp) and introduce the dynamics of the environment. then, we elaborate on how the agent interacts with the environment to seek evidence and answer a question. finally, to solve the pomdp, we describe how to train the agent via imitation learning. ## experiments we evaluate aiso and baselines on two wikipediasourced benchmarks. we first introduce the experimental setups, then describe the experimental results on evidence gathering and question answering. furthermore, detailed analyses are discussed. ## conclusion and future work this work presents an adaptive informationseeking approach for open-domain question answering, called aiso. it models the open-domain qa task as a pomdp, where the environment contains a large corpus and the agent is asked to sequentially select retrieval function and reformulate query to collect the evidence. aiso achieves stateof-the-art results on two public datasets, which demonstrates the necessity of different retrieval functions for different questions. in the future, we will explore other adaptive retrieval strategies, like directly optimizing various informationseeking metrics by using reinforcement learning techniques.