Open-Domain Conversational Question Answering with Historical Answers (2022)

Abstract: Open-domain conversational question answering can be viewed as two tasks: passage retrieval and conversational question answering, where the former relies on selecting candidate passages from a large corpus and the latter requires better understanding of a question with contexts to predict the answers. This paper proposes ConvADR-QA that leverages historical answers to boost retrieval performance and further achieves better answering performance. Our experiments on the benchmark dataset, OR-QuAC, demonstrate that our model outperforms existing baselines in both extractive and generative reader settings, well justifying the effectiveness of historical answers for open-domain conversational question answering.

## introduction conversational information seeking and conversational question answering (cqa) are fundamental tasks of dialogue systems @xcite . the conversational agents are expected to serve as nature interfaces for users' information need, providing information and answers via multi-turn natural language interactions. the multi-turn natural of cqa makes it challenging as the queries are contextualized, requiring the systems to resolve coreference and ambiguities. with recent advances in language understanding and dialogue modeling, along with the curation of large-scale datasets, e.g., quac @xcite and coqa @xcite , we have seen substantial progress in cqa. while the state-of-the-art (sota) models have achieved performance comparable or even superior than human performance on qa and cqa datasets, this setting is highly limited as it requires the source document containing evidence to be given, which is unlikely the case in real-world sce-narios. to address this issue, researchers have expanded the scheme of cqa to an open-domain setting, where the document containing evidence must be retrieved from a large candidate pool @xcite . in the open-domain setting, there are usually millions of candidate documents, making the conventional method which jointly encodes the query and the document infeasible @xcite . the dominant technique to tackle the challenge is dense retrieval @xcite @xcite , which encodes a query and documents as dense representations separately and performs nearest neighbor search that is efficient and scalable to millions of documents. it has been shown to outperform traditional sparse retrieval methods on multiple qa benchmarks. however, applying dense retrieval for conversations may need to consider the dialogue context and structure, which is not trivial. @xcite proposed orconvqa to include previous questions in the same dialogue. where the context-dependent nature of questions is shown useful. convdr @xcite further improved the retrieval performance by knowledge distillation on reformulated questions with an ad-hoc teacher model. nevertheless, simply concatenating historical questions is suboptimal. our hypothesis is that rather than relying on the model to infer helpful knowledge from historical questions, we provide direct signals by adding historical answers to the input. hence, we propose convadr-qa (conversational answeraware dense retrieval) to leverage historical answers for better retrieval and then answering performance for open-domain cqa. open-domain qa without a given target passage, most work for this task was built upon the dense retrieval framework for retrieving relevant passages for qa. dpr @xcite first showed that dense retrieval outperforms sparse retrieval methods. gar @xcite introduced pseudo relevance feedback by augmenting queries with generated texts. rider @xcite proposed a simple passage reranking method which promotes the passages containing the predicted answers. while these methods consider the predicted answers, they aim at improving single-turn question answering. we instead focus on enhancing model's ability on handling multiturn conversational questions. open-domain cqa researchers have put increasing attention on open-domain cqa with the trec conversational assistance track @xcite . however, these datasets have limited supervision, making dense retrieval hardly applicable due to its data-hungry nature. @xcite introduced the first large-scale open-domain cqa data, or-quac, by extending quac @xcite to a open-domain setting. they also proposed orconvqa, a pipeline system with a dpr retriever and an extractive reader, as a baseline system. convdr @xcite proposed to reformulate questions into their context-independent rewrites with the canard dataset @xcite , then applied knowledge distillation using a ad-hoc teacher model. our method is built upon these two methods by incorporating historical answers to aid the retriever. @xcite proposed a graph-guided retrieval method which constructs a graph using passages with historical answers and potential answers. our work does not introduce extra parameters and complex modeling, and we demonstrate that we can achieve better results with a simpler design for better practicality. the all- q 1 : when did jan berry die? a 1 : march 26, 2004. q 2 : what did he die of? a 2 : he suffered a seizure eight days before his 63rd birthday. q 3 : where was he buried? a 3 : on april 18, 2004, a "celebration of life" was held in berry's memory at the roxy theatre on the sunset strip in west hollywood, california. q 4 : why was he cremated? a 4 : "cannotanswer" a 1 : march 26, 2004. a 2 : after he suffered a seizure eight days before his 63rd birthday. a 3 : his body was cremated. previous wrong answers lead to the wrong direction ground truth answers dialogue w/ predicted answers history strategy from topiocqa @xcite is very similar to ours. however, their experimental setting is not realistic as they used ground truth answers as historical answers, which is corresponding to our oracle setting. to better leverage the historical answers for open-domain cqa, we propose convadr-qa illustrated in figure 2 , which includes a retriever for obtaining relevant passages from a large collection and a reader for cqa. ## experiments we conduct the experiments on an open-domain cqa benchmark: or-quac @xcite , we use three commonly used metrics, mrr@5, recall@5 and map@10, to evaluate the retrieval performance. in addition, we use word-level f1 and human equivalence score (heq) provided by the quac challenge to evaluate the overall performance of our system. the definitions of above metrics are detailed in appendix b. ## conclusions this work introduces convadr-qa, an opendomain cqa model that leverages historical anq1: what is roberto mangabeira unger,'s programmatic thought? a1: key in unger's thinking is the need to re-imagine social institutions before attempting to revise them. convdr: the beginning of unger's academic career began with the books knowledge and politics and law in modern society, convadr-qa: key in unger's thinking is the need to reimagine social institutions before attempting to revise them. q2: can you explain the mechanism of thinking? a2: in building this program, however, we must not entertain complete revolutionary overhaul, lest we be plagued by three false assumptions: convdr: cannotanswer convadr-qa: in building this program, however, we must not entertain complete revolutionary overhaul, lest we be plagued by three false assumptions: q3: what are the three false assumptions? a3: typological fallacy: convdr: unger finds three weaknesses that crippled the theory: foremost, the theory claimed that equilibrium would be spontaneously generated in a market economy. convadr-qa: typological fallacy: the fallacy that there is closed list of institutional alternatives in history, such as "feudalism" or "capitalism".