F LOW QA: G RASPING F LOW IN H ISTORY FOR C ONVERSATIONAL M ACHINE C OMPREHENSION
Anonymous authors
Paper under double-blind review
A BSTRACT
Conversational machine comprehension requires a deep understanding of the conversation history.
To enable traditional, single-turn models to encode the history comprehensively, we introduce F LOW , a mechanism that can incorporate intermediate representations generated during the process of answering previous questions, through an alternating parallel processing structure.
Compared to shallow approaches that concatenate previous questions/answers as input, F LOW integrates the latent semantics of the conversation history more deeply.
Our model, F LOW QA, shows superior performance on two recently proposed conversational challenges (+7.2% F 1 on CoQA and +4.0% on QuAC).
The effectiveness of F LOW also shows in other tasks.
By reducing sequential instruction understanding to conversational machine comprehension, F LOW QA outperforms the best models on all three domains in SCONE, with +1.8% to +4.4% improvement in accuracy.
I NTRODUCTION
[Figure]
Humans seek information in a conversational manner, by asking follow-up questions for additional information based on what they have already learned.
Recently proposed conversational machine comprehension (MC) datasets (Reddy et al., 2018; Choi et al., 2018) aim to enable models to assist in such information seeking dialogs.
They consist of a sequence of question/answer pairs where questions can only be understood along with the conversation history.
Figure 1 illustrates this new challenge.
Existing approaches take a single-turn MC model and augment the current question and context with the previous questions and answers (Choi et al., 2018; Reddy et al., 2018).
However, this offers only a partial solution, ignoring previous reasoning 2 processes performed by the model.
We present F LOW QA, a model designed for conversational machine comprehension.
F LOW QA consists of two main components: a base neural model for single-turn MC and a F LOW mechanism that encodes the conversation history .
Instead of just using the shallow history like previous questions and answers, we feed the model with the entire hidden representations generated during the process of answering previous questions.
These hidden representations potentially capture related information, such as phrases and facts in the context, for answering the previous questions, and hence provide additional clues on what the current conversation is revolving around.
This F LOW mechanism is also remarkably effective at tracking the world states for sequential instruction understanding (Long et al., 2016): after mapping world states as context and instructions as questions,
F LOW QA can interpret a sequence of inter-connected instructions and generate corresponding world state changes as answers.
The F LOW mechanism can be viewed as stacking single-turn QA models along the dialog progression (i.e., the question turns) and building information ﬂow along the dialog.
This information transfer happens for each context word , allowing rich information in the reasoning process to ﬂow.
This design is analogous to recurrent neural networks, where each single update unit is now an entire question answering process.
Because there are two recurrent structures in our modeling, one in the context for each question and the other in the conversation progression, a naive implementation leads to a highly unparallelizable structure.
To handle this issue, we propose an alternating parallel processing structure, which alternates between sequentially processing one dimension in parallel of the other dimension, and thus speeds up training signiﬁcantly.
F LOW QA achieves strong empirical results on conversational machine comprehension tasks, and improves the state of the art on various datasets (from 67.8% to 75.0% on CoQA and 60.1% to 64.1% on QuAC).
Perhaps more impressively, although designed for conversational machine comprehension, F LOW QA also shows superior performance on a seemingly different task – understanding a sequence of natural language instructions (framed previously as a sequential semantic parsing problem).
When tested on SCONE (Long et al., 2016), F LOW QA outperforms all existing systems in three different domains, resulting in a range of accuracy improvement from +1.8% to +4.4%.
B ACKGROUND : M ACHINE C OMPREHENSION T ASKS AND M ODELS
In this section, we introduce the task formulations of machine comprehension in both single-turn and conversational settings, and discuss the main ideas of state-of-the-art MC models.
2. T ASK F ORMULATION
Given an evidence document (context) and a question, the task is to ﬁnd the answer.
The context
C = { c 1 , c 2 , . . .
c m } is described as a sequence of m words and the question Q = { q 1 , q 2 . . .
q n } a sequence of n words.
In the extractive setting, the answer A must be a span in the context.
Conversational machine comprehension is a generalization of the single-turn setting: the agent needs to answer multiple, potentially inter-dependent questions in an interactive fashion.
The meaning of the current question may depend on the conversation history (e.g., in Fig.
1, a question such as ‘Where?’ cannot be answered in isolation).
Thus, previous conversational history (i.e., question/answer pairs) is provided in addition to the context and the current question.
2. B ASIC D ESIGN OF M ACHINE C OMPREHENSION M ODELS
For single-turn MC, many top-performing models share a similar architecture, consisting of four major components: (1) question encoding, (2) context encoding, (3) reasoning, and ﬁnally (4) answer prediction.
Initially the word embeddings (e.g., Pennington et al., 2014; Peters et al., 2018) of question tokens Q and context tokens C are taken as input and fed into contextual integration layers, such as LSTMs (Hochreiter & Schmidhuber, 1997) or self attentions (Yu et al., 2018), to encode the question and context.
Multiple integration layers provide contextualized representations of context, and are often inter-weaved with attention, which inject question information.
The context integration layers thus produce a series of query-aware hidden vectors for each word in the context.
Together, the context integration layers can be viewed as conducting implicit reasoning to ﬁnd the answer candidates.
The ﬁnal output is fed into the answer prediction layer to select the answer.
To adapt to the conversational setting, existing methods incorporate previous question/answer pairs into the current question and context encoding without modifying higher-level (reasoning and answer prediction) layers of the model.
F LOW QA
Our model aims to incorporate the conversation history more comprehensively via a conceptually simple F LOW mechanism.
We ﬁrst introduce the concept of F LOW (Section 3.1), propose the
[Figure] [Figure]
3. C ONCEPT OF F LOW
Successful conversational MC models should grasp the conversation ﬂow, such as the main topics revolving the dialog, relevant events being discussed, or related facts about the topic.
Since the conversation is based on the context C , we consider the conversation ﬂow as a representation based of the context tokens.
Conversation ﬂow can be inferred by just using previous question answer pairs, but for machine to better grasp the conversation ﬂow, we utilize the intermediate machine process for answering previous questions (See Fig. 1).
In MC models, the intermediate machine process is captured in the several context integration layers (often BiLSTMs), which locate the answer candidates in the context.
Our model considers this intermediate representations, C hi , generated during the h -th context integration layer of the reasoning component for the i -th question.
F LOW builds information ﬂow from the intermediate representation C h 1 , . .
, C hi − 1 generated for the previous question
Q 1 , . .
, Q i − 1 to the current process for answering Q i , for every h
and i
. 3. I NTEGRATION -F LOW L AYER
A naive sequential implementation of F LOW would pass the output hidden vectors from each integration layer during the ( i − 1) -th question turn to the corresponding integration layer for Q i .
This is highly unparalleled, as the contexts have to be read in order, and the question turns have to be processed sequentially.
To achieve better parallelism, we alternate between them: context integration , processing sequentially in context, in parallel of question turns; and ﬂow , processing sequentially in question turns, in parallel of context words (see Fig. 2).
This gives a 5 to 10 times speedup 3 at training time.
Throughout this section, we will use i ( 1 ≤ i ≤ t ) to iterate over question, j ( 1 ≤ j ≤ m ) to iterate over context, where t is the length of the dialog and m is the length of the context.
Below we describe the implementation of an I NTEGRATION -F LOW (IF) layer, which is composed of a context integration layer and a F LOW component.
Context Integration We pass the current context representation C hi for each question i into a
BiLSTM layer.
All question i ( 1 ≤ i ≤ t ) are processed in parallel during training.
[Equation]
F LOW
After the integration, we have t context sequences of length m , one for each question.
We reshape it to become m sequences of length t , one for each context word.
We then pass each sequence into a GRU 4 so the entire intermediate representation for answering the previous questions can be used when processing the current question.
We only consider the forward direction since we do not know the ( i + 1) -th question when answering the i -th question.
All context word j ( 1 ≤ j ≤ m ) are processed in parallel.
[Equation]
We reshape the outputs from the F LOW layer back, and concatenate them to the output of the integration layer.
[Equation]
In summary, this process takes C hi and generates C h
+1 i , which will be used for further contextualization to predict the start and end answer span tokens.
When F LOW is removed, the IF layer becomes a regular context integration layer and in this case, a single layer of BiLSTM.
3. D ETAILED D ESIGN OF F LOW QA
We construct our conversation MC model, F LOW QA, based on the single-turn MC structure (Sec. 2.2) with fully-aware attention (Huang et al., 2018).
The full architecture is shown in Fig.
In this section, we describe its main components: initial encoding, reasoning and answer prediction.
3.3. Q UESTION /C ONTEXT E NCODING
Word Embedding We embed the context into a sequence of vectors, C = { c 1 , . .
, c m } with pretrained GloVe (Pennington et al., 2014), CoVE (McCann et al., 2017) and ELMo (Peters et al., 2018) embeddings.
Similarly, each question at the i -th turn is embedded into a sequence of vectors
Q i = { q i, 1 , . . .
, q i,n } , where n is the maximum question length for all questions in the conversation.
Attention (on Question)
Following DrQA (Chen et al., 2017), for each question, we compute attention in the word level to enhance context word embeddings with question.
[Equation]
where g Qi,k is the GloVe embedding for the k -th question word in the i -th question, and g Cj is the
GloVe embedding for the j -th context word.
The ﬁnal question-speciﬁc context input representation
C
i contains: (1) word embeddings, (2) a binary indicator em
i,j , whether the j -th context word occurs in the i -th question, and (3) output from the attention.
[Equation]
Question Integration with QHierRNN Similar to many MC models, contextualized embeddings for the questions are obtained using multiple layers of BiLSTM (we used two layers).
[Equation]
We build a pointer vector for each question to be used in answer prediction layer by ﬁrst taking a weighted sum of each word vectors in the question.
[Equation]
where w is a trainable vector.
We then encode question history hierarchically with LSTMs to generate history-aware question vectors (QHierRNN).
[Equation]
The answer pointer vectors, p 1 , . .
. , p t , will be used in the answer prediction layer.
3.3. R EASONING
The reasoning component has several IF layers on top of the context encoding, inter-weaved with attention (ﬁrst on question, then on context itself).
We use fully-aware attention (Huang et al., 2018), which concatenates all layers of hidden vectors and uses S ( x, y )
=
ReLU( U x ) T D ReLU( U y ) to compute the attention score between x, y , where U , D are trainable parameters and D is a diagonal matrix.
Below we give the details of each layer (from bottom to top).
Integration-Flow ×
First, we take the question-augmented context representation C
i and pass it to two IF layers.
[Equation]
Attention (on Question)
After contextualizing the context representation, we perform fully-aware attention on the question for each context words.
[Equation]
Integration-Flow We concatenate the output from the previous IF layer with the attended question vector, and pass it as an input.
[Equation]
Attention (on Context)
We apply fully-aware attention on the context itself (self-attention).
[Equation]
Integration We concatenate the output from the the previous IF layer with the attention vector, and feed it to the last BiLSTM layer.
[Equation]
[Equation] [Equation]
. A NSWER P REDICTION
We use the same answer span selection method (Wang et al., 2017; Wang & Jiang, 2017; Huang et al., 2018) to estimate the start and end probabilities P Si,j , P Ei,j of the j -th context token for the i -th question.
[Equation]
To address unanswerable questions, we compute the probability of having no answer:
[Equation]
For each question Q i , we ﬁrst use P ∅ i to predict whether it has no answer.
6 If it is answerable, we predict the span to be j s , j e with the maximum P Si,j s P Ei,j e subject to the constraint 0 ≤ j e − j s ≤ 15 .
E XPERIMENTS : C ONVERSATIONAL M ACHINE C OMPREHENSION
In this section, we evaluate F LOW QA on recently released conversational MC datasets.
Data and Evaluation Metric
We experiment with the QuAC (Choi et al., 2018) and
CoQA (Reddy et al., 2018) datasets.
While both datasets follow the conversational setting (Section 2.1), QuAC asked crowdworkers to highlight answer spans from the context and CoQA asked for free text as an answer to encourage natural dialog.
While this may call for a generation approach,
Yatskar (2018) shows that the an extractive approach which can handle Yes/No answers has a high upper-bound – 97.8 F 1 .
Following this, we apply the extractive approach to CoQA.
We handle the
Yes/No questions by computing P Yi , P Ni using the same equation for P ∅ i (Eq. 17), and ﬁnd a span in the context for other questions.
The decision threshold
[Table]
[Table] [Table]
BiDAF++ (
[Table] [Table]
BiDAF++ (1-ctx)
[Table]
BiDAF++ (
[Table] [Table]
BiDAF++ (
[Table] [Table]
The main evaluation metric is F 1 , the harmonic mean of precision and recall at the word level.
In
CoQA, we report the performance for each context domain (children’s story, literature from Project
Gutenberg, middle and high school English exams, news articles from CNN, Wikipedia, AI2 Science
Questions, Reddit articles) and the overall performance.
For QuAC, we use its original evaluation metrics: F 1 and Human Equivalence Score (HEQ).
HEQ-Q is the accuracy of each question, where the answer is considered correct when the model’s F 1 score is higher than the average human Fscore.
Similarly, HEQ-D is the accuracy of each dialog – it is considered correct if all the questions in the dialog satisfy HEQ.
Comparison Systems
We compare F LOW QA with baseline models from CoQA and QuAC.
Reddy et al.
(2018) presented PGNet (Seq2Seq with copy mechanism),
DrQA
(Chen et al., 2017) and DrQA+PGNet (PGNet on predictions from DrQA) to address abstractive answer.
To incorporate dialog history, CoQA baselines concatenate the most recent previous question and answer to the current question.
8 Choi et al.
(2018) used BiDAF++, a strong extractive QA model to QuAC dataset.
They concatenate a feature vector encoding turn number to the question embedding and a feature vector encoding previous N answer locations to the context embeddings (denoted as N -ctx).
They found this to perform better than just concatenating previous question answer pairs.
Yatskar (2018) applied the same model to CoQA by modifying the system to ﬁrst make a Yes/No decision, and output an answer span only if Yes/No was not selected.
F LOW QA ( N -Ans) is our model: similar to BiDAF++ ( N -ctx), we concatenate the binary feature vector encoding previous N answer spans to the context embeddings.
Here we brieﬂy describe ablated systems: - F LOW removes the ﬂow component from IF layer (Eq. 2-4 in Section 3.2), -
QH IER RNN removes hierarchical LSTM on question pointer vectors (Eq. 9 in Section 3.3).
Results Tables 1 and 2 report model performance on CoQA and QuAC, respectively.
F LOW QA yields substantial improvement over existing models on both datasets (+7.2% F 1 on CoQA, +4.0%
F 1 on QuAC).
The larger gain on CoQA, which contains longer dialog chains, 9 suggests that our
F LOW architecture can capture long-range conversation history more effectively.
[Figure]
The corresponding units in the semantic parsing approach are shown to the right.
Sce.
Tan.
Alc.
Suhr & Artzi (2018) 56.
[Figure]
[Table] [Table]
[Table] [Table]
Table 3 shows the contributions of three components: (1) QHierRNN, hierarchical LSTM for encoding past questions, (2) F LOW , augmenting the intermediate representation from machine reasoning process in the conversation history, and (3) N -Ans, marking the gold answers for N previous turn in the context.
We ﬁnd that F LOW is a critical component.
Removing QHierRNN has a minor impact (0.1% on both datasets), while removing F LOW results in a substantial performance drop, with or without using QHierRNN (2-3% on QuAC, 4.1% on CoQA).
Without both components, our model performs comparably to the BiDAF++ model (1.0% gain).
10 Our model exploits the entire conversation history while prior models could leverage up to three previous turns.
By comparing 0-Ans and 1-Ans on two datasets, we can see that providing gold answers is more crucial for QuAC.
We hypothesize that QuAC contains more open-ended questions with multiple valid answers because the questioner cannot see the text.
The semantics of follow-up questions may change based on the answer span selected by the teacher among many valid answer spans, so knowing the exact answer is crucial.
E XPERIMENTS : S EQUENTIAL I NSTRUCTION U NDERSTANDING
In this section, we consider the task of understanding a sequence of natural language instructions.
We reduce this problem to a conversational MC task and apply F LOW QA.
Task Given a sequence of instructions, where the meaning of each instruction may depend on the entire history and world state, the task is to understand the instructions and modify the world accordingly.
More formally, given the initial world state W 0 and a sequence of natural language instructions { I 1 , . .
, I K } , the model has to perform the correct sequence of actions on W 0 , to obtain { W 1 , . . .
, W K } , the correct world states after each instruction.
Fig.
4 gives a simpliﬁed example from (Long et al., 2016).
Reducing Sequential Instruction Understanding to Conversational MC We reduce instruction understanding to machine comprehension using the follow mapping.
An illustration of this reduction is also in Fig.
4. • Context C
i : We encode the current world state W i − 1 as a sequence of tokens.
• Question Q
i : We simply treat each natural language instruction
I i as a question.
• Answer A
i : We encode the world state change from W i − 1 to W i as a sequence of tokens
At each time step i , the current context C i and question Q i are given to the system, which outputs the answer A i .
We simpliﬁed F LOW QA to prevent overﬁtting.
12 Appendix A.2 contains the details on model simpliﬁcation and reduction rules, i.e., mapping from the world state and state change to a sequence of token.
We also encode history explicitly by concatenating preceding questions with the current one and by marking previous answers in the current context similar to N -Ans in conversational MC.
During training, the previous gold answer (i.e., the world state change after each previous instruction) is given to the model.
In testing, the model predictions to the previous instructions are used.
5. R ESULTS
We evaluate our model on the sequential instruction understanding dataset SCONE (Long et al., 2016), which contains three domains (S CENE , T ANGRAMS , A LCHEMY ).
Each domain has a different environment setting (see Appendix A.2).
We compare our approaches with prior works (Long et al., 2016; Guu et al., 2017; Suhr & Artzi, 2018; Fried et al., 2018), which are semantic parsers that map each instruction into a logical form, and then execute the logical form to update the world state.
The model performance is evaluated by the correctness of the ﬁnal world state after ﬁve instructions.
The development and test set results are reported in Tables 4 and 5.
Even without F LOW , our model (F LOW QA-F LOW ) achieves comparable results in two domains (Tangrams and Alchemy) since we still encode the history explicitly.
When augmented with F LOW , our F LOW QA model gains decent improvements and outperforms the state-of-the-art models for all three domains.
R ELATED W ORK
Sequential question answering has been studied in the knowledge base setting (Iyyer et al., 2017;
Saha et al., 2018; Talmor & Berant, 2018), often framed as a semantic parsing problem.
Recent datasets (Choi et al., 2018; Reddy et al., 2018; Elgohary et al., 2018; Saeidi et al., 2018) enabled studying it in the textual setting, where the information source used to answer questions is a given article.
Existing approaches attempted on these datasets are often extensions of strong single-turn models, such as BiDAF (Seo et al., 2016) and DrQA (Chen et al., 2017), with some manipulation of the input.
In contrast, we propose a new architecture suitable for multi-turn MC tasks by passing the hidden model representations of preceding questions using the F LOW design.
Dialogue response generation requires reasoning about the conversation history as in conversational
MC.
This has been studied in social chit-chats (e.g., Ritter et al., 2011; Li et al., 2017; Ghazvininejad et al., 2018) and goal-oriented dialogs (e.g., Bordes & Weston, 2016; Lewis et al., 2017).
Prior work also modeled hierarchical representation of the conversation history (Park et al., 2018; Suhr & Artzi, 2018).
While these tasks target reasoning with the knowledge base or exclusively on the conversation history, the main challenge in conversational MC lies in reasoning about context based on the conversation history, which is the main focus in our work.
C ONCLUSION
We presented a novel F LOW component for conversational machine comprehension.
By applying
F LOW to a state-of-the-art machine comprehension model, our F LOW QA is able to encode the conversation history more comprehensively, and thus yields better performance.
When evaluated on two recently proposed conversational challenge datasets and three domains of a sequential instruction understanding task (through reduction), F LOW QA outperforms all current best models.
While our approach provides a substantial performance gain, there is still room for improvement.
In the future, we would like to investigate even more efﬁcient and ﬁne-grained ways to model the conversation ﬂow, as well as methods that enable machines to engage an more active and natural conversational behaviors, such as asking clariﬁcation questions.
A I MPLEMENTATION D ETAILS A. C ONVERSATIONAL Q UESTION A NSWERING
We make use of spaCy for tokenization.
We additionally ﬁne-tuned the GloVe embeddings of the top 1000 frequent question words.
During training, we use a dropout rate of 0 .
4 (Srivastava et al., 2014) after the embedding layer (GloVe, CoVe and ELMo) and before applying any linear transformation.
In particular, we share the dropout mask when the model parameter is shared, which is also known as variational dropout (Gal & Ghahramani, 2016).
We batch the dialogs rather than individual questions.
The batch size is set to one dialog for CoQA (since there can be as much as 20+ questions in each dialog), and three dialog for QuAC (since the question number is smaller).
The optimizer is
Adamax (Kingma & Ba, 2015) with a learning rate α = 0 .
002 , β = (0 . 9 , 0 . 999) and (cid:15) =
10 − 8 .
A ﬁxed random seed is used across all experiments.
All models are implemented in PyTorch ( http: //pytorch.org/ ).
We use a maximum of 20 epochs, with each epochs passing through the data once.
It roughly takes 10 to 20 epochs to converge.
A. S EQUENTIAL S EMANTIC P ARSING
We begin by elaborating the simpliﬁcation for FlowQA for the sequential semantic parsing task.
First, we use the 100-dim GloVe embedding instead of the 300-dim GloVe and we did not use any contextualized word embedding.
The GloVe embedding is ﬁxed throughout the training.
Secondly, the embedding for tokens in the context C are trained from scratch, since C consists of synthetic tokens.
Also, we removed word-level attention because the tokens in contexts and questions are very different (one is synthetic, while the other is natural language).
Additionally, we removed selfattention since we ﬁnd it unhelpful in this reduced QA setting (we speculate it’s because the context here is usually very short).
We use the same hidden size for both integration LSTMs and Flow
GRUs.
But we tune the hidden size for the three domains independently.
In S CENE , we use hidden size of h = 100 .
In T ANGRAMS , we use h = 75 .
And in A LCHEMY , we use h = 50 .
We also batch by dialog and use a batch size of 8.
A dropout rate of 0.3 is used and is applied before every linear transformations.
Environment for the Three Domains In S CENE , each environment has ten positions with at most one person at each position.
The domain covers four actions (enter, leave, move, and trade-hats) and two properties (hat color, shirt color).
In T ANGRAMS , the environment is a list containing at most ﬁve shapes.
This domain contains three actions (add, move, swap) and one property (shape).
Lastly, in A LCHEMY , each environment is seven numbered beakers and covers three actions (pour, drain, mix) dealing with two properties (color, amount).
Reducing World State to Context Now, we give details on the encoding of context from the world state.
In S CENE , there are ten positions.
For each position, there could be a person with shirt and hat, a person with a shirt, or no person.
We encode each position as two integers, one for shirt and one for hat.
(so the context length is ten)
Both integer take the value that corresponds to being a color or being empty.
In T ANGRAMS , originally there are ﬁve images.
But some commands could reduce the number of images or bring back removed images.
Since the number of images present is no greater than ﬁve, we always have ﬁve positions available (so the context length is ﬁve).
Each position consists of an integer, representing the ID of the image, and a binary feature.
Every time an image is removed, we append it at the back.
The binary feature is used to indicate if the image is still present or not.
In A LCHEMY , there are always seven beakers.
So the context length is seven.
Each position consists of two numbers, the color of the liquid at the top unit and the number of units in the beaker.
An embedding layer is used to turn each integers into a 10-dim vector.
Reducing the Logical Form to Answer Next, we encode the change of world state (i.e., the answer) into four integers.
The ﬁrst integer is the type of action that’s performed.
The second and third integer represent the position of the context, which the action is acted upon.
And the fourth integer represents additional property for action performed.
For example, in the A LCHEMY domain, we use (0 , i, j, 2) to mean “pour 2 units of liquid from beaker i to beaker j ”, and (1 , i, i, 3) to mean “throw out 3 units of liquid in beaker i ”.