From Rewriting to Remembering: Common Ground for Conversational QA Models (2022)

Abstract: In conversational QA, models have to leverage information in previous turns to answer upcoming questions. Current approaches, such as Question Rewriting, struggle to extract relevant information as the conversation unwinds. We introduce the Common Ground (CG), an approach to accumulate conversational information as it emerges and select the relevant information at every turn. We show that CG offers a more efficient and human-like way to exploit conversational information compared to existing approaches, leading to improvements on Open Domain Conversational QA.

## introduction speakers involved in a conversation continuously share new information, and build on it to achieve their communicative goals. in human communication, this process takes place effortlessly. as qa systems become conversational, efforts were made to make them able to mimic human behaviour, and to interpret the question at a turn in a conversation, based on the information in the previous turns. an approach to this task is to concatenate the previous turns to the current question @xcite @xcite . the approach has a main shortcoming, namely, it introduces a great amount of noise, since not everything in the previous turns is relevant. an alternative approach is question rewriting (qr), in which the question is rewritten in a self-contained form based on the previous conversational information @xcite . qr selects only the relevant information in previous turns, thus improving over concatenation. however, as the conversation progresses and the amount of information grows, qr models often fail to compress it in a rewrite. we argue that this is not only a limitation of the models, but an intrinsic limit of this approach, since producing informative rewrites is often unnatural also for humans (see section 4). in this work, we address the shortcomings above. inspired by the studies of @xcite , we propose a methodology to represent conversational information as a set of propositions, named the common ground (cg): at each turn, the relevant information is distilled in one or more propositions, which are added to the cg. as a new question comes in, the model selects the relevant information in the cg, and uses it to answer the question. the cg can thus be considered as an optimized summary, which returns the relevant information at every turn while keeping all the information discussed so far. we use the qrecc dataset @xcite ) to test cg on the task of open-domain conversational qa (odcqa) -in which answers to questions in a conversation have to be found in a large collection of documents -and show that it improves over existing approaches for modelling conversational information. we show that this is due to the fact that cg implements a more efficient and human-like way to account for previous information, which takes the best of existing approaches while avoiding their shortcomings: on the one hand, cg can access and maintain the full previous conversational context, but it avoids the noise issue; on the other, it can distill relevant information, but it is not forced to compress it in a single rewrite. ## common ground we now detail how we created a dataset for cg, and the model we implemented to generate the cg. ## experiments the goal of accounting for contextual information is to improve the performance on a downstream task. hence, we compare cg to existing approaches on the task of odcqa. data we use again qrecc, as it meets the requirements of the task: it is conversational, and it allows to experiment in an open-domain scenario. pipeline we use a retriever-reader pipeline. the retriever returns the top n most relevant candidates from the set of documents; these are passed to the reader, which extracts the final answer. we use bertserini @xcite , using bm25 as a retriever and a bert-large as a reader. each candidate returned by the retriever has a score s ret ; the answer extracted from that candidate by the reader has a score s rea . the final score s for the answer is defined as: for the retriever, we set n to 20, and we follow @xcite in setting k 1 =0.82 and @xmath0.68. we tune the value of µ on the validation set inde-pendently for each approach (see section 3.1). we do not finetune the reader, as we want to assess how much the cg can directly benefit any qa model, without the need to finetune it. ## results and analysis we show the results of our experiments in as expected, approaches leveraging contextual information improve over the original question. among these approaches, cg is the best: it improves the performance over rewrite, and, remarkably, it matches the results obtained with gold rewrites. a further improvement in f1 is observed when using cg-full at the retriever and cg at the reader (cg-full/cg), while using only cg-full degrades the performance. this shows that using the more informative but potentially noisier cg-full improves retrieval, but one needs to feed the filtered information from cg to the reader to see improvements in f1, as also observed by del plains the results of concatenation, which obtain high performance in retrieval, but drops in f1. cg vs. qr in in the previous case, the qr model fails to produce a rewrite; in others, this is just not possible. in the 6th turn of a conversation about different kinds of data network architectures (row 3), the user asks a general question about flaw types which encompasses all the previous information: there is so much information to compress, here, that not even humans manage to do it, and the gold rewrite is the same as the original question. 7 cg sidesteps this problem simply by making available all the pieces of relevant information emerged in the conversation, which can be selected and exploited by the model, without the need to produce a long natural sentence. note that besides being more effective, this solution is also more human-like: speakers do not repeat all the contextual information as they original question ## related work approaches to modelling conversational information have used either sparse or dense representation @xcite . this work focuses on the former. in this group, concatenation was proposed as an initial approach @xcite @xcite , followed by question rewriting @xcite . the main models for qr are either generative @xcite or extractive one @xcite ) -i.e., the relevant to-kens in the context are appended to the question. when a single model is used for both retriever and reader, generative model overperform extractive ones @xcite ; however, mixing the two approaches further improves the performance @xcite . our work is related to @xcite , as we also aim at extracting the relevant contextual information. however, instead of appending this information to the question, we stack it in the cg, and enable the model to pick the relevant information at each turn. ## conclusions we introduced the common ground, a novel approach for leveraging contextual information. we show that cg outperforms the main existing approaches in the odcqa task, due to its ability to select and maintain the relevant information in a more effective and human-like way. we see two main directions for future research on cg. first, we will exploit the ability of cg to include several kinds of information to make it more informative. for example, to answer the question 'how many covid cases today?', a qa system needs to be aware of the time and location of the person asking it @xcite . we want to include these and other information in the cg. second, we want to use cg to make qa models more transparent. currently, virtual assistants (such as alexa, siri and google assistant) are black boxes, i.e, the user does not know which information they extract from the input question, and which one they leverage to provide answers. this can make the interaction with them frustrating. cg offers a solution to the problem, as it allows to see what the assistant has in mind at each conversational turn. we will conduct experiments in which the cg is shared with the user, and see how this can make the interaction with the assistant more engaging and successful.