Can Question Rewriting Help Conversational Question Answering? (2022)

Abstract: Question rewriting (QR) is a subtask of conversational question answering (CQA) aiming to ease the challenges of understanding dependencies among dialogue history by reformulating questions in a self-contained form. Despite seeming plausible, little evidence is available to justify QR as a mitigation method for CQA. To verify the effectiveness of QR in CQA, we investigate a reinforcement learning approach that integrates QR and CQA tasks and does not require corresponding QR datasets for targeted CQA.We find, however, that the RL method is on par with the end-to-end baseline. We provide an analysis of the failure and describe the difficulty of exploiting QR for CQA.

## introduction the question rewriting (qr) task has been introduced as a mitigation method for conversational question answering (cqa). cqa asks a machine to answer a question based on the provided passage and a multi-turn dialogue @xcite , which poses an additional challenge to comprehend the dialogue history. to ease the challenge, qr aims to teach a model to paraphrase a question into a self-contained format using its dialogue history @xcite . except for @xcite , however, no one has provided evidence that qr is effective for cqa in practice. existing works on qr often (i) depend on the existence of a qr dataset for every target cqa dataset, and (ii) focus more on generating high-quality rewrites than improving cqa performance, making them unsatisfactory for the justification of qr. to verify the effectiveness of qr, we explore a reinforcement learning (rl) approach that integrates qr and cqa tasks without corresponding labeled qr datasets. in the rl framework, a qr model plays the role of "the agent" that receives figure 1 : overview of the rl approach. the current question q t and its dialogue history are reformulated into a self-contained question qt by the qr model. then, qt is passed to the qa model to extract an answer span ãt from the evidence document. we train the qr model by maximizing the reward obtained by comparing the predicted answer span ãt with the gold span a t . despite the potential and plausibility of the rl approach, our experimental results suggest an upper bound of the performance, and it is on par with the baselines without qr. in this paper, we provide analysis to (i) understand the reason for the failure of the rl approach and (ii) reveal that qr cannot improve cqa performance even with the non-rl approaches. the code is available at https: //github.com/hltchkust/cqr4cqa . ## related work the cqa task aims to assist users in seeking information @xcite @xcite . the key challenge is to resolve the conversation history and understand a highly-contextualized question. most prior works focus on model structures @xcite @xcite or training techniques @xcite to improve the performance. qr tasks have been proposed to further improve cqa systems by paraphrase a question into a self-contained styles @xcite @xcite . while many of the existing works on qr put more effort toward generating high-quality rewrites @xcite , @xcite introduced a framework to leverage qr to finetune cqa models with a consistency-based regularization. qr has also been studied in single-turn qa and other information-seeking tasks @xcite . ## methodology we denote a cqa dataset as {d n } n @xmath0, a i )} t @xmath1, where q t is the question and a t is the answer. along with the qa pairs, the corresponding evidence documents y t are also given. as depicted in figure 1 , our proposed rl framework involves a qa model as an environment and a qr model as an agent. let @xmath2. the objective of the qr model is to rewrite the question q t at turn t into a self-contained version, based on the current question and the dialogue history d t-1 . the agent takes an input state x @xmath3, q t ) and generates a paraphrase qt . then, @xmath4, qt ) and an evidence document y t are provided to an environment, namely, the qa model f ϕ , which extracts an answer span ãt = f ϕ ( xt , y t ). we aim for the agent, a qr model π θ , to learn to generate a high-quality para-phrase of the given question based on the reward received from the environment. our goal is to maximize the expected reward of the answer returned under the policy, namely, prior to the training process, the qa model f ϕ is fine-tuned on {d n } and the qr model is initialized with π θ = π θ 0 , where π θ 0 is a pretrained language model. we apply proximal policy optimization (ppo) @xcite to train π θ . ppo is a policy gradient method which alternates between sampling data through interaction with the environment and optimizing a surrogate objective function via stochastic gradient ascent. following ziegler et al. (2019) , we apply a kl-penalty to the reward r so as to prevent the policy π θ from drifting too far away from π θ 0 : ## discussion in this section, we provide an analysis to (i) raise a sensitivity problem of the qa model to explain the failure of rl and (ii) disclose that there is no justification for qr, even in the non-rl approaches. ## conclusion in this paper, we explore the rl approach to verify the effectiveness of qr in cqa, and report that the rl approach is on par with simple end-to-end baselines. we find the sensitivity of the qa models would disadvantage the rl training. future work is needed to verify that qr is a promising mitigation method for cqa since even the non-rl approaches perform unsatisfactorily.