FlowQA: Grasping Flow in History for Conversational Machine Comprehension

1 INTRODUCTION Humans seek information in a conversational manner , by asking follow-up questions for additional information based on what they have already learned . Recently proposed conversational machine comprehension ( MC ) datasets ( Reddy et al. , 2019 ; Choi et al. , 2018 ) aim to enable models to assist in such information seeking dialogs . They consist of a sequence of question/answer pairs where questions can only be understood along with the conversation history . Figure 1 illustrates this new challenge . Existing approaches take a single-turn MC model and augment the current question and context with the previous questions and answers ( Choi et al. , 2018 ; Reddy et al. , 2019 ) . However , this offers only a partial solution , ignoring previous reasoning1 processes performed by the model . ∗Work done during internship at Allen Institute for Artificial Intelligence . 1We use “ reasoning ” to refer to the model process of finding the answer . We present FLOWQA , a model designed for conversational machine comprehension . FLOWQA consists of two main components : a base neural model for single-turn MC and a FLOW mechanism that encodes the conversation history . Instead of using the shallow history , i.e. , previous questions and answers , we feed the model with the entire hidden representations generated during the process of answering previous questions . These hidden representations potentially capture related information , such as phrases and facts in the context , for answering the previous questions , and hence provide additional clues on what the current conversation is revolving around . This FLOW mechanism is also remarkably effective at tracking the world states for sequential instruction understanding ( Long et al. , 2016 ) : after mapping world states as context and instructions as questions , FLOWQA can interpret a sequence of inter-connected instructions and generate corresponding world state changes as answers . The FLOW mechanism can be viewed as stacking single-turn QA models along the dialog progression ( i.e. , the question turns ) and building information flow along the dialog . This information transfer happens for each context word , allowing rich information in the reasoning process to flow . The design is analogous to recurrent neural networks , where each single update unit is now an entire question answering process . Because there are two recurrent structures in our modeling , one in the context for each question and the other in the conversation progression , a naive implementation leads to a highly unparallelizable structure . To handle this issue , we propose an alternating parallel processing structure , which alternates between sequentially processing one dimension in parallel of the other dimension , and thus speeds up training significantly . FLOWQA achieves strong empirical results on conversational machine comprehension tasks , and improves the state of the art on various datasets ( from 67.8 % to 75.0 % on CoQA and 60.1 % to 64.1 % on QuAC ) . While designed for conversational machine comprehension , FLOWQA also shows superior performance on a seemingly different task – understanding a sequence of natural language instructions ( framed previously as a sequential semantic parsing problem ) . When tested on SCONE ( Long et al. , 2016 ) , FLOWQA outperforms all existing systems in three domains , resulting in a range of accuracy improvement from +1.8 % to +4.4 % . Our code can be found in https : //github.com/momohuang/FlowQA . 2 BACKGROUND : MACHINE COMPREHENSION . In this section , we introduce the task formulations of machine comprehension in both single-turn and conversational settings , and discuss the main ideas of state-of-the-art MC models . 2.1 TASK FORMULATION . Given an evidence document ( context ) and a question , the task is to find the answer to the question based on the context . The context C = { c1 , c2 , . . . cm } is described as a sequence of m words and the question Q = { q1 , q2 . . . qn } a sequence of nwords . In the extractive setting , the answer A must be a span in the context . Conversational machine comprehension is a generalization of the singleturn setting : the agent needs to answer multiple , potentially inter-dependent questions in sequence . The meaning of the current question may depend on the conversation history ( e.g. , in Fig . 1 , the third question such as ‘ Where ? ’ can not be answered in isolation ) . Thus , previous conversational history ( i.e. , question/answer pairs ) is provided as an input in addition to the context and the current question . 2.2 MODEL ARCHITECTURE . For single-turn MC , many top-performing models share a similar architecture , consisting of four major components : ( 1 ) question encoding , ( 2 ) context encoding , ( 3 ) reasoning , and finally ( 4 ) answer prediction . Initially the word embeddings ( e.g. , Pennington et al. , 2014 ; Peters et al. , 2018 ) of question tokens Q and context tokens C are taken as input and fed into contextual integration layers , such as LSTMs ( Hochreiter & Schmidhuber , 1997 ) or self attentions ( Yu et al. , 2018 ) , to encode the question and context . Multiple integration layers provide contextualized representations of context , and are often inter-weaved with attention , which inject question information . The context integration layers thus produce a series of query-aware hidden vectors for each word in the context . Together , the context integration layers can be viewed as conducting implicit reasoning to find the answer span . The final sequence of context vectors is fed into the answer prediction layer to select the start and end position of answer span . To adapt to the conversational setting , existing methods incorporate previous question/answer pairs into the current question and context encoding without modifying higher-level ( reasoning and answer prediction ) layers of the model . 3 FLOWQA . Our model aims to incorporate the conversation history more comprehensively via a conceptually simple FLOW mechanism . We first introduce the concept of FLOW ( Section 3.1 ) , propose the INTEGRATION-FLOW layers ( Section 3.2 ) , and present an end-to-end architecture for conversational machine comprehension , FLOWQA ( Section 3.3 ) . 3.1 CONCEPT OF FLOW . Successful conversational MC models should grasp how the conversation flows . This includes knowing the main topic currently being discussed , as well as the relevant events and facts . Figure 2 shows a simplified CoQA example where such conversation flow is crucial . As the conversation progresses , the topic being discussed changes over time . Since the conversation is about the context C , we consider FLOW to be a sequence of latent representations based on the context tokens ( the middle part of Fig 2 ) . Depending on the current topic , the answer to the same question may differ significantly . For example , when the dialog is about the author ’ s father ’ s funeral , the answer to the question What did he feel ? would be lonely , but when the conversation topic changes to five years after the death of the author ’ s father , the answer becomes we saved each other.2 Our model integrates both previous question/answer pairs and FLOW , the intermediate context representation from conversation history ( See Fig . 1 ) . In MC models , the intermediate reasoning process is captured in several context integration layers ( often BiLSTMs ) , which locate the answer candidates in the context . Our model considers these intermediate representations , Chi , generated during the h-th context integration layer of the reasoning component for the i-th question . FLOW builds information flow from the intermediate representation Ch1 , . . . , C h i−1 generated for the previous question Q1 , . . . , Qi−1 to the current process for answering Qi , for every h and i . 3.2 INTEGRATION-FLOW LAYER . A naive implementation of FLOW would pass the output hidden vectors from each integration layer during the ( i − 1 ) -th question turn to the corresponding integration layer for Qi . This is highly unparalleled , as the contexts have to be read in order , and the question turns have to be processed sequentially . To achieve better parallelism , we alternate between them : context integration , processing sequentially in context , in parallel of question turns ; and flow , processing sequentially in question turns , in parallel of context words ( see Fig . 3 ) . This architecture significantly improves efficiency during training . Below we describe the implementation of an INTEGRATION-FLOW ( IF ) layer , which is composed of a context integration layer and a FLOW component . Context Integration We pass the current context representation Chi for each question i into a BiLSTM layer . All question i ( 1 ≤ i ≤ t ) are processed in parallel during training . Ĉhi = ĉ h i,1 , . . . , ĉ h i , m = BiLSTM ( [ C h i ] ) ( 1 ) 2More detailed analysis on this example can be found in Appendix B . FLOW After the integration , we have t context sequences of length m , one for each question . We reshape it to become m sequences of length t , one for each context word . We then pass each sequence into a GRU3 so the entire intermediate representation for answering the previous questions can be used when processing the current question . We only consider the forward direction since we do not know the ( i + 1 ) -th question when answering the i-th question . All context word j ( 1 ≤ j ≤ m ) are processed in parallel . fh+11 , j , . . . , f h+1 t , j = GRU ( ĉ h 1 , j , . . . , ĉ h t , j ) ( 2 ) We reshape the outputs from the FLOW layer to be sequential to context tokens , and concatenate them to the output of the integration layer . Fh+1i = { f h+1 i,1 , . . . , f h+1 i , m } ( 3 ) Ch+1i = c h+1 i,1 , . . . , c h+1 i , m = [ ĉ h i,1 ; f h+1 i,1 ] , . . . , [ ĉ h i , m ; f h+1 i , m ] ( 4 ) In summary , this process takes Chi and generates C h+1 i , which will be used for further contextualization to predict the start and end answer span tokens . When FLOW is removed , the IF layer becomes a regular context integration layer and in this case , a single layer of BiLSTM . 3.3 FULL ARCHITECTURE OF FLOWQA . We construct our conversation MC model , FLOWQA , based on the single-turn MC structure ( Sec . 2.2 ) with fully-aware attention ( Huang et al. , 2018 ) . The full architecture is shown in Fig . 4 . In this section , we describe its main components : initial encoding , reasoning and answer prediction . 3.3.1 QUESTION/CONTEXT ENCODING . Word Embedding We embed the context into a sequence of vectors , C = { c1 , . . . , cm } with pretrained GloVe ( Pennington et al. , 2014 ) , CoVE ( McCann et al. , 2017 ) and ELMo ( Peters et al. , 2018 ) embeddings . Similarly , each question at the i-th turn is embedded into a sequence of vectors Qi = { qi,1 , . . . , qi , n } , where n is the maximum question length for all questions in the conversation . Attention ( on Question ) Following DrQA ( Chen et al. , 2017 ) , for each question , we compute attention in the word level to enhance context word embeddings with question . The generated question-specific context input representation is denoted as C0i . For completeness , a restatement of this representation can be found in Appendix C.1 . 3We use GRU because it is faster and performs comparably to LSTM based on our preliminary experiments . Question Integration with QHierRNN Similar to many MC models , contextualized embeddings for the questions are obtained using multiple layers of BiLSTM ( we used two layers ) . Q1i = q 1 i,1 , . . . , q 1 i , n = BiLSTM ( Qi ) , Q 2 i = q 2 i,1 , . . . , q 2 i , n = BiLSTM ( Q 1 i ) ( 5 ) We build a pointer vector for each question to be used in the answer prediction layer by first taking a weighted sum of each word vectors in the question . q̃i = n∑ k=1 αi , k · q2i , k , αi , k ∝ exp ( wT q2i , k ) , ( 6 ) where w is a trainable vector . We then encode question history hierarchically with LSTMs to generate history-aware question vectors ( QHierRNN ) . p1 , . . . , pt = LSTM ( q̃i , . . . , q̃t ) ( 7 ) The final vectors , p1 , . . . , pt , will be used in the answer prediction layer .

Summary: In this paper, authors proposed a so-called FLOWQA for conversational question answering (CoQA). Comparing with machine reading comprehension (MRC),  CoQA includes a conversation history. Thus, FLOWQA makes use of this property of CoQA and adds an additional encoder to handle this. It also includes one classifier to handle with no-answerable questions.