Current approaches to open-domain question answering often make crucial assumptions that prevent them from generalizing to real-world settings, including the access to parameterized retrieval systems well-tuned for the task, access to structured metadata like knowledge bases and web links, or a priori knowledge of the complexity of questions to be answered (e.g., single-hop or multi-hop). To address these limitations, we propose a unified system to answer open-domain questions of arbitrary complexity directly from text that works with off-the-shelf retrieval systems on arbitrary text collections . We employ a single multi-task  model to perform all the necessary subtasks---retrieving supporting facts, reranking them, and predicting the answer from all retrieved documents---in an iterative fashion. To emulate a more realistic setting, we also constructed a new unified benchmark by collecting about 200 multi-hop questions that require three Wikipedia pages to answer, and combining them with existing datasets . We show that our model not only outperforms state-of-the-art systems on several existing benchmarks that exclusively feature single-hop or multi-hop open-domain questions, but also achieves strong performance on the new benchmark.
We develop a unified system to answer directly from text open-domain questions that may require a varying number of retrieval steps . We employ a single multi-task transformer model to perform all the necessary subtasks -- retrieving supporting facts, reranking them, and predicting the answer from all retrieved documents -- in an iterative fashion. We avoid making crucial assumptions as previous work that do not transfer well to real-world settings, including exploiting knowledge of the fixed number of retrieval steps required to answer each question or using structured metadata like knowledge bases or web links that have limited availability. Instead, we design a system that would answer open-domain questions on any text collection without prior knowledge of reasoning complexity. To emulate this setting, we construct a new benchmark by combining existing one- and two-step datasets with a new collection of 203 questions that require three Wikipedia pages to answer, unifying Wikipedia corpora versions in the process . We show that our model demonstrates competitive performance on both existing benchmarks and this new benchmark.
R
Current approaches to open-domain question answering often make crucial assumptions that prevent them from generalizing to real-world settings, including the access to parameterized retrieval systems well-tuned for the task, access to structured metadata like knowledge bases and web links, or a priori knowledge of the complexity of questions to be answered (e.g., single-hop or multi-hop). To address these limitations, we propose a
We develop a
D
open-domain questions of arbitrary complexity
R
that works with off-the-shelf retrieval systems on arbitrary text collections
open-domain questions that may require a varying number of retrieval steps
A
transformer
R
subtasks---retrieving
subtasks -- retrieving
R
documents---in
documents -- in
R
To emulate a more realistic
We avoid making crucial assumptions as previous work that do not transfer well to real-world settings, including exploiting knowledge of the fixed number of retrieval steps required to answer each question or using structured metadata like knowledge bases or web links that have limited availability. Instead, we design a system that would answer open-domain questions on any text collection without prior knowledge of reasoning complexity. To emulate this
R
also constructed a new unified benchmark by collecting about 200 multi-hop
construct a new benchmark by combining existing one- and two-step datasets with a new collection of 203
R
and combining them with existing datasets
unifying Wikipedia corpora versions in the process
R
not only outperforms state-of-the-art systems on several existing benchmarks that exclusively feature single-hop or multi-hop open-domain questions, but also achieves strong performance on the
demonstrates competitive performance on both existing benchmarks and this