FlowQA: Grasping Flow in History for Conversational Machine Comprehension

1 INTRODUCTION Humans seek information in a conversational manner , by asking follow-up questions for additional information based on what they have already learned . Recently proposed conversational machine comprehension ( MC ) datasets ( Reddy et al. , 2019 ; Choi et al. , 2018 ) aim to enable models to assist in such information seeking dialogs . They consist of a sequence of question/answer pairs where questions can only be understood along with the conversation history . Figure 1 illustrates this new challenge . Existing approaches take a single-turn MC model and augment the current question and context with the previous questions and answers ( Choi et al. , 2018 ; Reddy et al. , 2019 ) . However , this offers only a partial solution , ignoring previous reasoning1 processes performed by the model . ∗Work done during internship at Allen Institute for Artificial Intelligence . 1We use “ reasoning ” to refer to the model process of finding the answer . We present FLOWQA , a model designed for conversational machine comprehension . FLOWQA consists of two main components : a base neural model for single-turn MC and a FLOW mechanism that encodes the conversation history . Instead of using the shallow history , i.e. , previous questions and answers , we feed the model with the entire hidden representations generated during the process of answering previous questions . These hidden representations potentially capture related information , such as phrases and facts in the context , for answering the previous questions , and hence provide additional clues on what the current conversation is revolving around . This FLOW mechanism is also remarkably effective at tracking the world states for sequential instruction understanding ( Long et al. , 2016 ) : after mapping world states as context and instructions as questions , FLOWQA can interpret a sequence of inter-connected instructions and generate corresponding world state changes as answers . The FLOW mechanism can be viewed as stacking single-turn QA models along the dialog progression ( i.e. , the question turns ) and building information flow along the dialog . This information transfer happens for each context word , allowing rich information in the reasoning process to flow . The design is analogous to recurrent neural networks , where each single update unit is now an entire question answering process . Because there are two recurrent structures in our modeling , one in the context for each question and the other in the conversation progression , a naive implementation leads to a highly unparallelizable structure . To handle this issue , we propose an alternating parallel processing structure , which alternates between sequentially processing one dimension in parallel of the other dimension , and thus speeds up training significantly . FLOWQA achieves strong empirical results on conversational machine comprehension tasks , and improves the state of the art on various datasets ( from 67.8 % to 75.0 % on CoQA and 60.1 % to 64.1 % on QuAC ) . While designed for conversational machine comprehension , FLOWQA also shows superior performance on a seemingly different task – understanding a sequence of natural language instructions ( framed previously as a sequential semantic parsing problem ) . When tested on SCONE ( Long et al. , 2016 ) , FLOWQA outperforms all existing systems in three domains , resulting in a range of accuracy improvement from +1.8 % to +4.4 % . Our code can be found in https : //github.com/momohuang/FlowQA . 2 BACKGROUND : MACHINE COMPREHENSION . In this section , we introduce the task formulations of machine comprehension in both single-turn and conversational settings , and discuss the main ideas of state-of-the-art MC models . 2.1 TASK FORMULATION . Given an evidence document ( context ) and a question , the task is to find the answer to the question based on the context . The context C = { c1 , c2 , . . . cm } is described as a sequence of m words and the question Q = { q1 , q2 . . . qn } a sequence of nwords . In the extractive setting , the answer A must be a span in the context . Conversational machine comprehension is a generalization of the singleturn setting : the agent needs to answer multiple , potentially inter-dependent questions in sequence . The meaning of the current question may depend on the conversation history ( e.g. , in Fig . 1 , the third question such as ‘ Where ? ’ can not be answered in isolation ) . Thus , previous conversational history ( i.e. , question/answer pairs ) is provided as an input in addition to the context and the current question . 2.2 MODEL ARCHITECTURE . For single-turn MC , many top-performing models share a similar architecture , consisting of four major components : ( 1 ) question encoding , ( 2 ) context encoding , ( 3 ) reasoning , and finally ( 4 ) answer prediction . Initially the word embeddings ( e.g. , Pennington et al. , 2014 ; Peters et al. , 2018 ) of question tokens Q and context tokens C are taken as input and fed into contextual integration layers , such as LSTMs ( Hochreiter & Schmidhuber , 1997 ) or self attentions ( Yu et al. , 2018 ) , to encode the question and context . Multiple integration layers provide contextualized representations of context , and are often inter-weaved with attention , which inject question information . The context integration layers thus produce a series of query-aware hidden vectors for each word in the context . Together , the context integration layers can be viewed as conducting implicit reasoning to find the answer span . The final sequence of context vectors is fed into the answer prediction layer to select the start and end position of answer span . To adapt to the conversational setting , existing methods incorporate previous question/answer pairs into the current question and context encoding without modifying higher-level ( reasoning and answer prediction ) layers of the model . 3 FLOWQA . Our model aims to incorporate the conversation history more comprehensively via a conceptually simple FLOW mechanism . We first introduce the concept of FLOW ( Section 3.1 ) , propose the INTEGRATION-FLOW layers ( Section 3.2 ) , and present an end-to-end architecture for conversational machine comprehension , FLOWQA ( Section 3.3 ) . 3.1 CONCEPT OF FLOW . Successful conversational MC models should grasp how the conversation flows . This includes knowing the main topic currently being discussed , as well as the relevant events and facts . Figure 2 shows a simplified CoQA example where such conversation flow is crucial . As the conversation progresses , the topic being discussed changes over time . Since the conversation is about the context C , we consider FLOW to be a sequence of latent representations based on the context tokens ( the middle part of Fig 2 ) . Depending on the current topic , the answer to the same question may differ significantly . For example , when the dialog is about the author ’ s father ’ s funeral , the answer to the question What did he feel ? would be lonely , but when the conversation topic changes to five years after the death of the author ’ s father , the answer becomes we saved each other.2 Our model integrates both previous question/answer pairs and FLOW , the intermediate context representation from conversation history ( See Fig . 1 ) . In MC models , the intermediate reasoning process is captured in several context integration layers ( often BiLSTMs ) , which locate the answer candidates in the context . Our model considers these intermediate representations , Chi , generated during the h-th context integration layer of the reasoning component for the i-th question . FLOW builds information flow from the intermediate representation Ch1 , . . . , C h i−1 generated for the previous question Q1 , . . . , Qi−1 to the current process for answering Qi , for every h and i . 3.2 INTEGRATION-FLOW LAYER . A naive implementation of FLOW would pass the output hidden vectors from each integration layer during the ( i − 1 ) -th question turn to the corresponding integration layer for Qi . This is highly unparalleled , as the contexts have to be read in order , and the question turns have to be processed sequentially . To achieve better parallelism , we alternate between them : context integration , processing sequentially in context , in parallel of question turns ; and flow , processing sequentially in question turns , in parallel of context words ( see Fig . 3 ) . This architecture significantly improves efficiency during training . Below we describe the implementation of an INTEGRATION-FLOW ( IF ) layer , which is composed of a context integration layer and a FLOW component . Context Integration We pass the current context representation Chi for each question i into a BiLSTM layer . All question i ( 1 ≤ i ≤ t ) are processed in parallel during training . Ĉhi = ĉ h i,1 , . . . , ĉ h i , m = BiLSTM ( [ C h i ] ) ( 1 ) 2More detailed analysis on this example can be found in Appendix B . FLOW After the integration , we have t context sequences of length m , one for each question . We reshape it to become m sequences of length t , one for each context word . We then pass each sequence into a GRU3 so the entire intermediate representation for answering the previous questions can be used when processing the current question . We only consider the forward direction since we do not know the ( i + 1 ) -th question when answering the i-th question . All context word j ( 1 ≤ j ≤ m ) are processed in parallel . fh+11 , j , . . . , f h+1 t , j = GRU ( ĉ h 1 , j , . . . , ĉ h t , j ) ( 2 ) We reshape the outputs from the FLOW layer to be sequential to context tokens , and concatenate them to the output of the integration layer . Fh+1i = { f h+1 i,1 , . . . , f h+1 i , m } ( 3 ) Ch+1i = c h+1 i,1 , . . . , c h+1 i , m = [ ĉ h i,1 ; f h+1 i,1 ] , . . . , [ ĉ h i , m ; f h+1 i , m ] ( 4 ) In summary , this process takes Chi and generates C h+1 i , which will be used for further contextualization to predict the start and end answer span tokens . When FLOW is removed , the IF layer becomes a regular context integration layer and in this case , a single layer of BiLSTM . 3.3 FULL ARCHITECTURE OF FLOWQA . We construct our conversation MC model , FLOWQA , based on the single-turn MC structure ( Sec . 2.2 ) with fully-aware attention ( Huang et al. , 2018 ) . The full architecture is shown in Fig . 4 . In this section , we describe its main components : initial encoding , reasoning and answer prediction . 3.3.1 QUESTION/CONTEXT ENCODING . Word Embedding We embed the context into a sequence of vectors , C = { c1 , . . . , cm } with pretrained GloVe ( Pennington et al. , 2014 ) , CoVE ( McCann et al. , 2017 ) and ELMo ( Peters et al. , 2018 ) embeddings . Similarly , each question at the i-th turn is embedded into a sequence of vectors Qi = { qi,1 , . . . , qi , n } , where n is the maximum question length for all questions in the conversation . Attention ( on Question ) Following DrQA ( Chen et al. , 2017 ) , for each question , we compute attention in the word level to enhance context word embeddings with question . The generated question-specific context input representation is denoted as C0i . For completeness , a restatement of this representation can be found in Appendix C.1 . 3We use GRU because it is faster and performs comparably to LSTM based on our preliminary experiments . Question Integration with QHierRNN Similar to many MC models , contextualized embeddings for the questions are obtained using multiple layers of BiLSTM ( we used two layers ) . Q1i = q 1 i,1 , . . . , q 1 i , n = BiLSTM ( Qi ) , Q 2 i = q 2 i,1 , . . . , q 2 i , n = BiLSTM ( Q 1 i ) ( 5 ) We build a pointer vector for each question to be used in the answer prediction layer by first taking a weighted sum of each word vectors in the question . q̃i = n∑ k=1 αi , k · q2i , k , αi , k ∝ exp ( wT q2i , k ) , ( 6 ) where w is a trainable vector . We then encode question history hierarchically with LSTMs to generate history-aware question vectors ( QHierRNN ) . p1 , . . . , pt = LSTM ( q̃i , . . . , q̃t ) ( 7 ) The final vectors , p1 , . . . , pt , will be used in the answer prediction layer .

Summary: The paper presents a new model FlowQA for conversation reading comprehension. Compared with the previous work on single-turn reading comprehension, the idea in this paper differs primarily in that it alternates between the context integration and the question flow in parallel. The parallelism enables the model to be trained 5 to 10 times faster. Then this process is formulated as layers of a neural network that are further stacked multiple times. Besides, the unanswerable question is predicted with additional trainable parameters. Empirical studies confirm FlowQA works well on a bunch of datasets. For example, it achieves new state-of-the-art results on two QA datasets, i.e., CoQA and QuAC, and outperforms the best models on all domains in SCONE. Ablation studies also indicates the importance of the concept Flow.