Capturing Conversational Interaction for Question Answering via Global History Reasoning (2022)

Abstract: Conversational Question Answering (ConvQA) is required to answer the current question, conditioned on the observable paragraph-level context and conversation history. Previous works have intensively studied history-dependent reasoning. They perceive and absorb topic-related information of prior utterances in the interactive encoding stage. It yielded significant improvement compared to history-independent reasoning. This paper further strengthens the ConvQA encoder by establishing long-distance dependency among global utterances in multi-turn conversation. We use multi-layer transformers to resolve long-distance relationships, which potentially contribute to the reweighting of attentive information in historical utterances. Experiments on QuAC show that our method obtains a substantial improvement (1%), yielding the F1 score of 73.7%. All source codes are available at https://github.com/jaytsien/GHR .

## introduction convqa is a task of answering questions conditioned on the conversation history as well as referential contexts. it heavily relies on the traceback of conversation history. for example, the pronoun "it" in q 2 in recently, utilizing global conversation history for enhancement is increasingly gaining interest, because it potentially contributes to capturing longdistance relevant evidence for answering. both historical-answer-aware dynamic encoding of context @xcite and flow-based interaction in this paper, we develop a global history reasoning (ghr) model. ghr is not only capable of separately encoding different rounds of questionanswering (qa) conversations, but sequentially fusing the encoded information of all qa pairs in visible conversations by a multi-layer attention network. it is designed to avoid the omission of available historical information and disconnection. we experiment on quac @xcite . the test results show that ghr yields substantial improvements when using bert and electra as the baselines, and it achieves competitive performance compared to state-of-the-art methods. ## approach the input of ghr comprises the referential context c, the current question in the t-th round, and all his- allow to attend prevent from attending history attention mask ... ... ... ... ... ... [cls] figure 1 shows the overall architecture of ghr, where the sequentially stacked cards (drawn with light green rectangles) denote the encoding stages for the questions issued at different times. for example, the visible structure in the top card illustrates the encoding stage for the first question q 1 , and at the time, both the local conversation history d 1 and the visible global history are null. the global attention mechanism is shown at the right side of the diagram, whose input is the representation encoded by each transformer layer (blue arrows), while the output is the refined representation by the masked global interaction (black arrows). the masking operation is used to temporally disable the subsequent qa conversations when the current question is being dealt with (as required in the task of convqa). ## conclusion in this paper, we propose a global history reasoning (ghr) approach to capture interactions between all utterances for conversational question answering. experimental results conducted on quac show the effectiveness of the proposed model. in the future, we will explore how to implement interactions between conversation history and context by the position features of history answers. in addition, we will extend our conversation history modeling approach to the knowledge-grounded conversation generation task (see more related work in appendix a.1).