Learn to Resolve Conversational Dependency: A Consistency Training Framework for Conversational Question Answering (2021)

Abstract: One of the main challenges in conversational question answering (CQA) is to resolve the conversational dependency, such as anaphora and ellipsis. However, existing approaches do not explicitly train QA models on how to resolve the dependency, and thus these models are limited in understanding human dialogues. In this paper, we propose a novel framework, ExCorD (Explicit guidance on how to resolve Conversational Dependency) to enhance the abilities of QA models in comprehending conversational context. ExCorD first generates self-contained questions that can be understood without the conversation history, then trains a QA model with the pairs of original and self-contained questions using a consistency-based regularizer. In our experiments, we demonstrate that ExCorD significantly improves the QA models’ performance by up to 1.2 F1 on QuAC, and 5.2 F1 on CANARD, while addressing the limitations of the existing approaches.

## introduction conversational question answering (cqa) involves modeling the information-seeking process of humans in a dialogue. unlike single-turn question answering (qa) tasks @xcite , cqa is a multi-turn qa task, where questions in a dialogue are contextdependent; 2 hence they need to be understood with the conversation history @xcite . as illustrated in figure 1 , to answer title : leonardo da vinci conversation history with his pupils salai and melzi. a1 who were his pupils? q leonardo's most intimate relationships a2 was he close to his pupils? q was he close with anyone else? q was leonardo da vinci close with anyone else other than his pupils salai and melzi? @xcite . owing to linguistic phenomena in human conversations, such as anaphora and ellipsis, the current question q 3 should be understood based on the conversation history: q 1 , a 1 , q 2 , and a 2 . question q 3 can be reformulated as a self-contained question q3 via a question rewriting (qr) process. the current question "was he close with anyone else?," a model should resolve the conversational dependency, such as anaphora and ellipsis, based on the conversation history. a line of research in cqa proposes the end-toend approach, where a single qa model jointly encodes the evidence document, the current question, and the whole conversation history @xcite @xcite . in this approach, models are required to automatically learn to resolve conversational dependencies. however, existing models have limitations to do so without explicit guidance on how to resolve these dependencies. in the example presented in figure 1 , models are trained without explicit signals that "he" refers to "leonardo da vinci," and "anyone else" can be more elaborated with "other than his pupils, salai and melzi." another line of research proposes a pipeline approach that decomposes the cqa task into question rewriting (qr) and qa, to reduce the complexity of the task @xcite . based on the conversation history, qr models first generate selfcontained questions by rewriting the original questions, such that the self-contained questions can be understood without the conversation history. for instance, the current question q 3 is reformulated as the self-contained question q3 by a qr model in figure 1 . after rewriting the question, qa models are asked to answer the self-contained questions rather than the original questions. in this approach, qa models are trained to answer relatively simple questions whose dependencies have been resolved by qr models. thus, this limits reasoning abilities of qa models for the cqa task, and causes qa models to rely on qr models. in this paper, we emphasize that qa models can be enhanced by using both types of questions with explicit guidance on how to resolve the conversational dependency. accordingly, we propose excord (explicit guidance on how to resolve conversational dependency), a novel training framework for the cqa task. in this framework, we first generate self-contained questions using qr models. we then pair the self-contained questions with the original questions, and jointly encode them to train qa models with consistency regularization @xcite . specifically, when original questions are given, we encourage qa models to yield similar answers to those when self-contained questions are given. this training strategy helps qa models to better understand the conversational context, while circumventing the limitations of previous approaches. to demonstrate the effectiveness of excord, we conduct extensive experiments on the three cqa benchmarks. in the experiments, our framework significantly outperforms the existing approaches by up to 1.2 f1 on quac @xcite and by 5.2 f1 on canard @xcite . in addition, we find that our framework is also effective on a dataset coqa @xcite that does not have the self-contained questions generated by human annotators. this indicates that the proposed framework can be adopted on various cqa datasets in future work. we summarize the contributions of this work as follows: ## excord: explicit guidance on resolving conversational dependency we introduce a unified framework that jointly encodes the original and self-contained questions as illustrated in figure 2c . our framework consists of two stages: (1) generating self-contained questions using a qr model ( §3.1) and ( 2 ) training a qa model with the original and self-contained questions via consistency regularization ( §3.2). ## experiments in this section, we describe our experimental setup and compare our framework to baseline approaches (i.e., the end-to-end and pipeline approaches). ## analysis and discussion we elaborate on analyses regarding component ablation and transferability. we also describe a case study carried out to highlight such differences between our and baseline approaches. ## related work conversational question answering recently, several works introduced cqa datasets such as quac @xcite and coqa @xcite . we classified proposed methods to solve the datasets into two approaches: (1) end-to-end and (2) pipeline. most works based on the endto-end approach focused on developing a model structure @xcite @xcite or training strategy such as multitask with rationale tagging @xcite that are specialized in the cqa task or datasets. several works demonstrated the effectiveness of the flow mechanism in cqa @xcite @xcite . with the advent of a dataset consisting of selfcontained questions rewritten by human annotators @xcite , the pipeline approach has drawn attention as a promising method for cqa in recent days @xcite . the approach is particularly useful for the open-domain cqa or passage-retrieval (pr) tasks @xcite @xcite since self-contained questions can be fed into existing non-conversational search engines such as bm25. note that our framework can be used jointly with the pipeline approach in the opendomain setting because our framework can improve qa models' ability to find the answers from the retrieved documents. we will test our framework in the open-domain setting in future work. question rewriting qr has been studied for augmenting training data @xcite @xcite or clarifying ambiguous questions @xcite . in cqa, qr can be viewed as a task of simplifying difficult questions that include anaphora and ellipsis in a conversation. elgohary et al. (2019) first proposed the question rewriting task as a sub-task of cqa and the canard dataset for the task, which consists of pairs of original and self-contained questions that are generated by human annotators. @xcite used a coreference-based model @xcite and gpt-2 @xcite as qr models and tested the models in the qr and pr tasks. @xcite conducted the qr task using t5 @xcite and achieved on performance comparable to humans on canard. following @xcite , we use t5 in our experiments to generate high-quality questions for enhancing qa models. consistency training consistency regularization @xcite has been mainly explored in the context of semisupervised learning (ssl) @xcite , which has been adopted in the textual domain as well @xcite @xcite . however, the consistency training framework is also applicable when only the labeled samples are available @xcite @xcite . the consistency regularization requires adding noise to the sample, which can be either discrete @xcite or continuous @xcite . existing works regularize the predictions of the perturbed samples to be equivalent to be that of the originals'. on the other hand, our method encourages the models' predictions for the original asnwers to be similar to those from the rewritten questions, i.e., synthetic ones. we propose a consistency training framework for conversational question answering, which enhances qa models' abilities to understand conversational context. our framework leverages both the original and self-contained questions for explicit guidance on how to resolve conversational dependency. in our experiments, we demonstrate that our framework significantly improves the qa model's performance on quac and canard, compared to the existing approaches. in addition, we verified that our framework can be extended to coqa. in future work, the transferability of our framework can be further improved by fine-tuning the qr model on target datasets. furthermore, future work would include applying our framework to the open-domain setting.