Title: GRAPHFLOW: EXPLOITING CONVERSATION FLOW WITH GRAPH NEURAL NETWORKS FOR CONVERSA-TIONAL MACHINE COMPREHENSION

Abstract: Conversational machine comprehension (MC) has proven significantly more challenging compared to traditional MC since it requires better utilization of conversation history. However, most existing approaches do not effectively capture conversation history and thus have trouble handling questions involving coreference or ellipsis. We propose a novel graph neural network (GNN) based model, namely GRAPHFLOW, which captures conversational flow in the dialog. Specifically, we first propose a new approach to dynamically construct a question-aware context graph from passage text at each turn. We then present a novel flow mechanism to model the temporal dependencies in the sequence of context graphs. The proposed GRAPHFLOW model shows superior performance compared to existing state-ofthe-art methods. For instance, GRAPHFLOW outperforms two recently proposed models on the CoQA benchmark dataset: FLOWQA by 2.3% and SDNet by 0.7% on F1 score, respectively. In addition, visualization experiments show that our proposed model can better mimic the human reasoning process for conversational MC compared to existing models.

1. INTRODUCTION:

Recent years have observed a surge of interest in conversational machine comprehension (MC). Unlike the setting of MC that requires answering a single question given a passage (aka context), the conversational MC task is to answer the current question in a conversation given a passage and the previous questions and answers. The goal of this task is to mimic real-world situations where humans seek information in a conversational manner.

Despite the success existing works have achieved on MC (e.g., SQuAD (Rajpurkar et al., 2016) ), conversational MC has proven significantly more challenging. We highlight two major challenges here. First, in conversational MC the focus usually shifts as the conversation progresses (Reddy et al., 2018; Choi et al., 2018) . Second, many questions refer back to the conversation history via either coreference or ellipsis. Therefore, without fully utilizing the conversation history, one can not understand the current question correctly. In this work, we model the concept of conversation flow as a sequence of latent states associated with these shifts of focus in a conversation.

To cope with the above challenges, many methods have been proposed to effectively utilize conversation history, including previous questions and/or previous answers. Most existing approaches, however, simply prepend the conversation history to the current question (Reddy et al., 2018; Zhu et al., 2018) or add previous answer locations to the passage (Choi et al., 2018; Yatskar, 2018) , and treat the task as a single-turn MC while ignoring the important information from the conversation flow. (Huang et al., 2018) assumed that the hidden representations generated during the previous reasoning processes potentially capture important information for answering the previous questions, and thus provide additional clues for answering the current question. They proposed an Integration-Flow (IF) mechanism to first process sequentially in passage, in parallel of question turns and then process sequentially in question turns, in parallel of passage words. Their FLOWQA model achieves strong empirical results on two benchmarks (i.e., CoQA and QuAC) (Reddy et al., 2018; Choi et al., 2018) .

However, the IF mechanism is not quite natural since it does not mimic how humans perform reasoning. This is because when humans execute such task, they typically do not first perform reasoning in parallel for each question, and then refine the reasoning results across different turns. This may partially explain why this strategy is inefficient because the results of previous reasoning processes are not incorporated into the current reasoning process. As a result, the reasoning performance at each turn is only slightly improved by the hidden states of the previous reasoning process, even though they use stacked IF layers to try to address this problem.

To address the aforementioned issues, we propose GRAPHFLOW, a Graph Neural Network (GNN) based model for conversational MC. As shown in Fig. 1 , GRAPHFLOW consists of three components, Encoding Layer, Reasoning Layer, and Prediction Layer. The Encoding Layer encodes conversation history and the context text that aligns question embeddings. The Reasoning Layer dynamically constructs a question-aware context graph at each turn, and then applies GNNs to process the sequence of context graphs. In particular, the graph node embedding outputs of the reasoning process at the previous turn are used as a starting state when reasoning at the current turn, which is closer to how humans perform reasoning in a conversational setting, compared to existing approaches. The Prediction Layer predicts the answers based on the matching score of the question embedding and the learned graph node embeddings for context at each turn.

We highlight our contributions as follows:

• We propose a novel graph neural network based model, namely GRAPHFLOW, for conversational MC which captures conversational flow in the dialog. • We dynamically construct a context graph that consists of each passage word as a node, which encodes not only the passage itself, but also the question as well as the conversation history. Besides, to our best knowledge, we are the first to treat context as a graph of words in the literature of MC. • We propose a novel flow mechanism to process a sequence of context graphs, which models the temporal dependencies between consecutive context graphs. • On two public benchmarks (i.e., CoQA and QuAC), the proposed model shows superior performance compared to existing state-of-the-art methods. For instance, GRAPHFLOW outperforms two recently proposed models FLOWQA by 2.3% F1 and SDNet by 0.7% F1 on benchmark dataset CoQA, respectively. • Our interpretability analysis shows that our model can better mimic human reasoning process on this task.

2. THE GRAPHFLOW APPROACH:

Figure 1 : Overall architecture of the proposed model. Best viewed in color.

2.1. ENCODING LAYER:

Let us denote C as the context which consists of a sequence of words tc 1 , c 2 , ..., c m u and Q i as the question at the i-th turn which consists of a sequence of words tq piq 1 , q piq 2 , ..., q piq n u. We apply an effective encoding layer to encode the context and the question, which additionally exploits the interactions (i.e., soft alignment) between them as well as conversation history (i.e., previous question-answer pairs). The details of the encoder are given next.

Linguistic features For each context word, we encode linguistic features to a vector f ling pc piq j q concatenating POS (part-of-speech), NER (named entity recognition) and exact matching (which indicates whether the context word appears in Q i ) embeddings.

Pretrained word embeddings We use 300-dim GloVe (Pennington et al., 2014) embeddings as well as 1024-dim BERT (Devlin et al., 2018) embeddings to embed each word in the context and the question. Compared to GloVe embeddings, BERT embeddings better utilize the contextual information and can learn different embeddings for the same word type within different context.

Aligned question embeddings Exact matching only matches words on the surface form, we further apply an attention mechanism to learn the soft alignment between context words and question words. Following (Lee et al., 2016) and recent work, for each context word c j at the i-th turn, we incorporate an aligned question embedding f align pc piq j q " ř k a piq j,k g Qi k where g Qi k is the GloVe embedding of question word q piq k and a piq j,k is an attention score between context word c j and question word q piq k .

Here we define the attention score a piq j,k as,

EQUATION

where W P R dˆ300 is a trainable model parameter, d is the hidden state size, and g C j is the GloVe embedding of context word c j . To simplify notation, we denote the above attention mechanism as AlignpA, B, Cq, meaning that an attention matrix is computed between two sets of vectors A and B, which is later used to get a linear combination of vector set C. Hence we can reformulate the above alignment as f align pC piq q " Alignpg C , g Qi , g Qi q.

Conversation history Conversation history is crucial for understanding the current question in this task. (Choi et al., 2018) found it useful to concatenate a feature vector f ans pc piq j q encoding previous N answer locations to the context word embeddings. We adopt this strategy, and additionally prepend previous N question-answer pairs to the current question. When prepending conversation history, a common choice is to separate the current question from conversation history using certain special token, which does not work well in practice as observed by (Choi et al., 2018) . We find a more effective strategy to do the separation which is to concatenate a turn marker embedding f turn pq piq k q to each word vector in the augmented question. The turn marker is intended to indicate which turn the word belongs to (e.g., i indicates the previous i-th turn).

In summary, at the i-th turn in a conversation, each context word c j is encoded by a vector w piq cj which is a concatenation of linguistic vector f ling pc piq j q, GloVe vector g C j , BERT vector BERT C j , aligned vector f align pc piq j q and answer vector f ans pc piq j q. And each question word q piq k is encoded by a vector w Qi k which is a concatenation of GloVe vector g Qi k , BERT vector BERT Qi k and turn marker vector f turn pq piq k q. We denote W piq C and W Qi as a sequence of context word vectors w piq cj and question word vectors w Qi k , respectively at the i-th turn.

2.2. REASONING LAYER:

When performing reasoning over context, unlike most previous methods that solely regard context as a sequence of words, we opt to treat context as a "graph" of words, and hence it is straightforward to apply GNNs to process the context graph. Our Reasoning Layer consists of the Question Understanding module, Context Graph Learning module and Context Graph Reasoning module.

2.2.1. QUESTION UNDERSTANDING:

For each question Q i , we apply a BiLSTM (Hochreiter & Schmidhuber, 1997) to the raw question embeddings W Qi to obtain contextualized embeddings

Q i P R dˆn . Q i " q piq 1 , ..., q piq n " BiLSTMpW Qi q (2)

Each question is then represented as a weighted sum of word vectors in the question via a simple self attention mechanism.

r q piq " ÿ k a piq k q piq k , where a piq k 9 exppw T q piq k q (3)

where w is a d-dim trainable weight.

Finally, to capture the dependency among questions at different turns, we encode question history sequentially in question turns with a LSTM to generate history-aware question vectors.

p 1 , ..., p T " LSTMpr q p1q , ..., r q pT q q (4)

The output hidden states of the LSTM network p 1 , ..., p T will be used for predicting answers.

2.2.2. CONTEXT GRAPH LEARNING:

The intrinsic context graph structure is unfortunately unknown in the MC task. Moreover, intuitively speaking, the context graph structure might vary across different turns by considering the change of questions and conversation history. Most existing applications of GNNs take as input ground-truth graphs or manually constructed graphs which have some limitations. First, the ground-truth graphs are not always available. Second, the error in the manual construction process can be propagated to the subsequent GNN-based models and cannot be reduced during the learning process. In this work, we choose to automatically construct graphs from raw context, which will be combined with the remaining of the system to make the whole learning system end-to-end trainable. The difference between manual graph construction and automatic graph construction is analogous to the difference between the feature engineering + Machine Learning paradigm and the Deep Learning end-to-end paradigm.

We dynamically build a weighted graph to model semantic relationships among context words at each turn in a conversation. We make the process of building such a context graph depend on not only the semantic meanings of context words, but also on the question being asked, as well as the conversation history, so as to help better answer the question.

Specifically, we first apply an attention mechanism to the context representations W piq C (which additionally incorporate both question information and conversation history) at the i-th turn to compute an attention matrix A piq C , serving as a weighted adjacency matrix for the context graph, defined as,

A piq C " ReLUpUW piq C q T ReLUpUW piq C q (5)

where U is a d ˆdc trainable weight and d c is the embedding size of w piq cj . Considering that a fully connected context graph is not only computationally expensive but also makes little sense for reasoning, a simple KNN-style graph sparsification operation is conducted to extract a sparse and weighted graph from the fully connected one. To be concrete, given a learned attention matrix A piq C , we only keep the K nearest neighbors (including itself) as well as the associated attention scores (i.e., the remaining attentions scores are masked off) for each context node, and apply a softmax function to these selected adjacency matrix elements to get a sparse and normalized adjacency matrix r

A piq C . r A piq C " softmaxptopkpA piq C qq (6)

Note that the supervision signal is still able to back-propagate through the KNN-style graph sparsificalation module since the K nearest attention scores are kept and used to compute the weights of the final normalized adjacency matrix. Given the context graphs constructed at each turn, we propose a novel Recurrent Graph Neural Network (RGNN) to sequentially process a sequence of graphs, as shown in Fig. 2 . Conceptually, one can think that it is analogous to an RNN-style structure where the main difference is that each element in a sequence is not a data point, but instead a graph. As we advance in a sequence of graphs, we process each graph using a shared GNN cell and the output of the GNN cell will be used when processing the next graph. Our RGNN module combines the advantages of RNNs which are good at sequential learning (i.e., modeling sequential data), and GNNs which are good at relational reasoning (i.e., modeling graph-structured data).

2.2.3. CONTEXT GRAPH REASONING:

The computational details of RGNNs are as follows. At the i-th turn, before we apply a GNN to the context graph G i , we update node embeddings by fusing both the original node information C l´1 i and the updated node information at the previous turn C l i´1 via a fusion function.

C l i " GNNp s C l´1 i , r A piq C q s C l´1 i,j " FusepC l´1 i,j , C l i´1,j q (7)

where l is the RGNN layer index. Note that we can stack multiple RGNN layers to enhance the performance if necessary. The fusion function is designed as a gated sum of two information sources, Fusepa, bq " z ˚a `p1 ´zq ˚b z " σpW z ra; b; a ˚b; a ´bs `bz q (8)

where σ is a sigmoid function and z is a gating vector.

As a result, the graph node embedding outputs of the reasoning process at the previous turn are used as a starting state when reasoning at the current turn. Note that we set s C l´1 0 " C l´1 0 as we will not incorporate any historical information at the first turn. Notably, even though there is no direct link between c j at the pi ´1q-th turn and c k at the i-th turn, the information flow can be propagated between them as the GNN progresses, which means information can be exchanged among all the objects spatially and temporally.

We use Gated Graph Neural Networks (GGNN) (Li et al., 2015) as our GNN module, but the framework is agnostic to the particular choice of GNN module. In GGNN we can do multi-hop message passing through the graph to capture long-range dependency where the same set of network parameters are shared at every hop of computation. At each hop of GGNN computation, for every node in the graph, we apply an aggregation function which takes as input a set of neighboring node embeddings and outputs an aggregation vector. In our framework, we compute a weighted average for aggregation where the weights come from the normalized adjacency matrices r A piq C . Then, a Gated Recurrent Unit (GRU) (Cho et al., 2014) is used to update the node embeddings by incorporating the aggregation information. We use the updated node embeddings at the last hop as the final node embeddings.

To simplify notation, we denote the above RGNN module as C l " RGNNpC l´1 , r A C q which takes as input a sequence of graph node embeddings C l´1 as well as a sequence of the normalized adjacency matrices r A C , and outputs a sequence of updated graph node embeddings C l .

While a GNN is responsible for modeling the global interactions among context words, modeling local interactions among consecutive context words is also important for the task. Therefore, before feeding the context word representations to a GNN, we first apply a BiLSTM to the context words, that is, C 0 i " BiLSTMpW piq C q, and we then use the output C 0 i as the initial context node embedding. Inspired by recent work (Wang et al., 2018) on modeling the context with different levels of granularity, we choose to apply one RGNN layer on low level representations of the context and another RGNN layer on high level representations of the context, as formulated in the following.

C 1 " RGNNpC 0 , r A C q H C i " rC 1 i ; g C ; BERT C s H Q i " rQ i ; g Qi ; BERT Qi s f 2 align pC piq q " AlignpH C i , H Q i , Q i qs r C 1 i " BiLSTMprC 1 i ; f 2 align pC piq qsq C 2 " RGNNp r C 1 , r A C q (9)

2.3. PREDICTION LAYER:

We predict answer spans by computing the start and end probabilities P S i,j and P E i,j of the j-th context word for the i-th question. To predict the start index, the probability is calculated by,

P S i,j 9 exppc 2 i,j T W S p i q (10)

where W S is a d ˆd trainable weight. Next, p i is passed to a GRU by incorporating context summary and converted to r p i .

r p i " GRUpp i , ÿ j P S i,j c 2 i,j q (11)

Then, the end probability is calculated by,

P E i,j 9 exppc 2 i,j T W E r p i q (12)

where W E is a d ˆd trainable weight.

We apply an answer type classifier to handle unanswerable questions and questions whose answers are not text spans in the context. The probability of the answer type (e.g., "unknown", "yes" and "no") is calculated as follows,

r C 2 i " rf mean pC 2 i q; f max pC 2 i qs P C i " σpf c pp i qp r C 2 i q T q (13)

where f c is a dense layer which maps a d-dim vector to a pnum class ˆ2dq-dim vector. Further, σ is a sigmoid function for binary classification and a softmax function for multi-class classification. We use r C 2 i to represent the whole context at the i-th turn which is a concatenation of average pooling and max pooling outputs of C 2 i .

2.4. TRAINING AND INFERENCE:

The training objective is defined as the cross entropy loss of both text span prediction (if the question requires it) and answer type prediction.

L " ´ÿ i I S i plogpP S i,si q `logpP E i,ei qq `log

EQUATION

where I S i indicates whether this question requires text span prediction, s i and e i are the ground-truth start and end positions of the answer span, and t i indicates the ground-truth answer type.

During inference, we first use P C i to predict whether the question requires text span prediction. If yes, we predict the span to be s i , e i with maximum P S i,si P E i,ei subject to the length constraint on the span.

3. EXPERIMENTS:

In this section, we conduct an extensive evaluation of our proposed model against state-of-the-art conversational MC models. We use two popular benchmarks, described below. The implementation of the model will be publicly available soon. Detailed description of model settings is provided in Appendix A. Although CoQA is released as an abstractive CMC dataset, Yatskar (2018) shows that the extractive approach is also effective for CoQA. Thus, we also use our extractive approach on CoQA. To handle answer types in CoQA, we predict the probability distribution of the answer type (SPAN, YES, NO, and UNANSWERABLE) and replace the predicted span with yes, no, or unknown tokens ex-cept for the SPAN answer type. In QuAC, the unanswerable questions are handled as an answer span (P contains a special token), and the type prediction for yes/no questions is not evaluated on the leaderboard. Therefore, we skip the answer type prediction step.

CoQA CoQA contains 127k questions with answers, obtained from 8k conversations. In CoQA, answers are in free-form and hence are not necessarily text spans from the context (i.e., 33.2% of the questions have abstractive answers). Although this calls for a generation approach, following previous work (Yatskar, 2018; Huang et al., 2018; Zhu et al., 2018) , as detailed in Section 2.3, we adopt an extractive approach with additional answer type classifiers to handle non-extractive questions. The average length of questions is only 5.5 words (i.e., 70% questions have coreference or ellipsis), which means conversation history is important for better understanding those questions.

The average number of turns per dialog is 15.2. Notably, in the testing set, there are two out-ofdomain datasets which are reserved for testing only.

QuAC QuAC contains 98k questions with answers, obtained from 13k conversations. Unlike CoQA, all the answers in QuAC are text spans from the context. The average length of questions is 6.5 and there are on average 7.2 questions per dialog. The average length of QuAC context is 401 which is longer than that of CoQA which is 271. The average length of QuAC answers is 14.6 which is also longer than that of CoQA which is 2.7.

The main evaluation metric is F1 score which is the harmonic mean of precision and recall at word level between the predicted answer and ground truth. In addition, for QuAC the Human Equivalence Score (HEQ) is used to judge whether a system performs as well as an average human. HEQ-Q and HEQ-D are model accuracies at question level and dialog level, respectively. Please refer to (Reddy et al., 2018; Choi et al., 2018) for details of these metrics.

Under review as a conference paper at ICLR 2020

3.2. MODEL COMPARISON:

Baseline methods We compare our approach with the following baseline methods: PGNet (See et al., 2017) , DrQA (Chen et al., 2017) , DrQA+PGNet (Reddy et al., 2018) , BiDAF++ (Yatskar, 2018) , FLOWQA (Huang et al., 2018) and SDNet (Zhu et al., 2018) . We will discuss the details of these methods in Section 4.1.

Results:

As shown in Section 3 and Section 3, our model consistently outperforms these state-of-theart baselines in terms of F1 score. In particular, GRAPHFLOW yields improvement over all existing models on both datasets by at least +0.7% F1 on CoQA and +0.8% F1 on QuAC, respectively.

Compared with FLOWQA which is also based on the flow idea, our model improves F1 by 2.3% on CoQA and 0.8% on QuAC, which demonstrates the superiority of our RGNN mechanism over the Integration-Flow mechanism. Compared with SDNet which relies on sophisticated inter-attention and self-attention mechanisms, our model improves F1 by 0.7% on CoQA1 . We conduct an extensive ablation study to further investigate the performance impact of different components in our model. Here we briefly describe ablated systems: -RecurrentConn removes temporal connections between consecutive context graphs, -RGNN removes the RGNN module, -PreQues does not prepend previous questions to the current turn, -PreAns does not prepend previous answers to the current turn, -PreAnsLoc does not mark previous answer locations in the context, and -BERT removes pretrained BERT embeddings. We also show the model performance with no conversation history GRAPHFLOW (0-His) or one previous turn of the conversation history GRAPHFLOW (1-His).

3.3. ABLATION STUDY:

Section 3.3 shows the contributions of the above components on the CoQA development set. We find that the pretrained BERT embedding has the most impact on the performance, which again demonstrates the power of large-scale pretrained language models. Our proposed RGNN module also contributes significantly to the model performance (i.e., improves F1 score by 7.2%). In addition, within the RGNN module, both the GNN part (i.e., 1.1% F1) and the temporal connection part (i.e., 6.1% F1) contribute to the results. We also notice that explicitly adding conversation history to the current turn helps the model performance by comparing GRAPHFLOW (2-His), GRAPHFLOW (1-His) and GRAPHFLOW (0-His). We can see that the previous answer information is more crucial than the previous question information. And among many ways to use the previous answer information, directly marking previous answer locations seems to be the most effective. We conjecture this is partially because the turn transitions in a conversation are usually smooth and marking the previous answer locations helps the model better identify relevant context chunks for the current question.

3.4. INTERPRETABILITY ANALYSIS:

Here we visualize the memory bank (i.e., an m by d matrix) which stores the hidden representations (and thus reasoning output) of the context throughout a conversation. While directly visualizing the hidden representations is difficult, thanks to the flow-based mechanism introduced into our model, we instead visualize the changes of hidden representations of context words between consecutive turns. We expect that the most changing parts of the context should be those which are relevant to the questions being asked and therefore should probably be able to indicate shifts of the focus in a conversation.

Following (Huang et al., 2018) , we visualize this by computing the cosine similarity of the hidden representations of the same context words at consecutive turns, and then highlight the words that have small cosine similarity scores (i.e., change more significantly)2 . Fig. 3 highlights the most changing context words between consecutive turns in a conversation from the CoQA dev. set. As we can see, the hidden representations of context words which are relevant to the consecutive questions are changing most and thus highlighted most. We suspect this is in part because when the focus shifts, the model finds out the context chunks relevant to the previous turn become less important but those relevant to the current turn become more important. Therefore, the memory updates in these regions are the most active. Obviously, this makes the model easier to answer follow-up questions. As we observe in our visualization experiments, in conversations extensively involving coreference or ellipsis, our model can still perform reasonably well. We refer the interested readers to the supplementary material for complete visualization examples.

4. RELATED WORK:

4.1 CONVERSATIONAL MC Many methods have been proposed to utilize conversation history in the literature of conversational MC. (Reddy et al., 2018; Zhu et al., 2018) concatenate previous questions and answers to the current question. (Choi et al., 2018) concatenate a feature vector encoding the turn number to the question word embedding and a feature vector encoding previous N answer locations to the context embeddings. However, as claimed by (Huang et al., 2018) , these methods ignore previous reasoning processes performed by the model when reasoning at the current turn. Instead, they propose the idea of Integration-Flow to allow rich information in the reasoning process to flow through a conversation.

Another challenge of this task is how to handle abstractive answers. (Reddy et al., 2018) propose a hybrid method DrQA+PGNet, which augments a traditional extractive RC model with a text generator. (Yatskar, 2018) propose to first make a Yes/No decision, and output an answer span only if Yes/No was not selected.

4.2. GRAPH NEURAL NETWORKS:

Over the past few years, graph neural networks (GNNs) (Kipf & Welling, 2016; Gilmer et al., 2017; Hamilton et al., 2017; Li et al., 2015; Xu et al., 2018a) have drawn increasing attention since they extend traditional Deep Learning approaches to non-euclidean data such as graph-structured data.

Recently, GNNs have been applied to various question answering tasks including visual question answering (VQA) (Teney et al., 2017; Norcliffe-Brown et al., 2018) , knowledge base question answering (KBQA) (Sun et al., 2018) and MC (De Cao et al., 2018; Song et al., 2018; Xu et al., 2018c) , and have shown advantages over traditional approaches. For tasks where the graph structure is unknown, (De Cao et al., 2018; Song et al., 2018; Xu et al., 2018b ) construct a static graph where entity mentions in a passage are nodes of this graph and edge information is determined by coreferences of entity mentions. (Norcliffe-Brown et al., 2018) dynamically construct a graph which contains all the visual objects appearing in an image. In parallel to our work, (Liu et al., 2018 ) also dynamically construct a graph of all words from free text.

:

They did not report the results on QuAC.

For better visualization, we apply an attention threshold of 0.3 to highlight only the dramatically changing context words.

CONCLUSIONWe proposed a novel GNNs-based model, namely GRAPHFLOW, for conversational MC which carries over the reasoning output throughout a conversation. On two recently released conversational MC benchmarks, our proposed model achieves superior results over previous approaches. Interpretability analysis shows that the model can better mimic the human reasoning process for conversational MC compared to existing models.In the future, we would like to investigate more effective ways of automatically learning graph structures from free text and modeling temporal connections between sequential graphs.

Figures and Tables:

Figure 2: Architecture of the proposed Recurrent Graph Neural Network (RGNN) for processing a sequence of context graphs. Best viewed in color.

Figure 3: The highlighted part of the context indicates the QA model's focus shifts between consecutive turns.

Model and human performance (% in F1 score) on the CoQA test set.

Table content: <table><tr><td/><td colspan="6">Child. Liter. Mid-High. News Wiki Reddit Science</td><td>Overall</td></tr><tr><td>PGNet</td><td>49.0</td><td>43.3</td><td>47.5</td><td>47.5</td><td>45.1 38.6</td><td>38.1</td><td>44.1</td></tr><tr><td>DrQA</td><td>46.7</td><td>53.9</td><td>54.1</td><td>57.8</td><td>59.4 45.0</td><td>51.0</td><td>52.6</td></tr><tr><td>DrQA+PGNet</td><td>64.2</td><td>63.7</td><td>67.1</td><td>68.3</td><td>71.4 57.8</td><td>63.1</td><td>65.1</td></tr><tr><td>BiDAF++</td><td>66.5</td><td>65.7</td><td>70.2</td><td>71.6</td><td>72.6 60.8</td><td>67.1</td><td>67.8</td></tr><tr><td>FLOWQA</td><td>73.7</td><td>71.6</td><td>76.8</td><td>79.0</td><td>80.2 67.8</td><td>76.1</td><td>75.0</td></tr><tr><td>SDNet</td><td>75.4</td><td>73.9</td><td>77.1</td><td>80.3</td><td>83.1 69.8</td><td>76.8</td><td>76.6</td></tr><tr><td>GRAPHFLOW</td><td>77.1</td><td>75.6</td><td>77.5</td><td>79.1</td><td>82.5 70.8</td><td>78.4</td><td>77.3</td></tr><tr><td>Human</td><td>90.2</td><td>88.4</td><td>89.8</td><td>88.6</td><td>89.9 86.7</td><td>88.1</td><td>88.8</td></tr></table>

Model and human performance (in %) on the QuAC test set.

Table content: <table><tr><td/><td>F1</td><td colspan="2">HEQ-Q HEQ-D</td></tr><tr><td>BiDAF++</td><td colspan="2">60.1 54.8</td><td>4.0</td></tr><tr><td>FLOWQA</td><td colspan="2">64.1 59.6</td><td>5.8</td></tr><tr><td>GRAPHFLOW</td><td colspan="2">64.9 60.3</td><td>5.1</td></tr><tr><td>Human</td><td colspan="2">80.8 100</td><td>100</td></tr><tr><td>3.1 DATA AND METRICS</td><td/><td/></tr></table>

Ablation study: model performance (in %) on the CoQA dev. set.

Table content: <table><tr><td/><td>F1</td></tr><tr><td>GRAPHFLOW (2-His)</td><td>78.3</td></tr><tr><td>-PreQues</td><td>78.2</td></tr><tr><td>-PreAns</td><td>77.7</td></tr><tr><td>-PreAnsLoc</td><td>76.6</td></tr><tr><td>-BERT</td><td>76.0</td></tr><tr><td>-RecurrentConn</td><td>69.9</td></tr><tr><td>-RGNN</td><td>68.8</td></tr><tr><td>GRAPHFLOW (1-His)</td><td>78.2</td></tr><tr><td>GRAPHFLOW (0-His)</td><td>76.7</td></tr></table>

Peer review (iclr2020):
The paper introduces a fairly sophisticated model for conversational QA. The authors describe it as a combination of self-attention, BiLSTM encoders, and graph neural networks. BiLSTMs are used to encode local contexts, self-attention predicts edges of a graph (using a top-k mechanism) and graph neural networks / GNN perform message passing within this predicted graph. Note that these are gated GNNs, so, in practice, another set of gates is used to control the flow of the information (on top of gates in self-attention). GNNs do not start in each conversation turn from scratch but rather integrate information from the previous turn (using forget gates). Some improvements over external baselines have been shown on QuAC and CoQA. I do not find the motivation for the model particularly convincing and the results exciting enough to justify accepting a paper with an over-complicated architecture. To motivate the approach, the authors attempt to link their model to how humans reason (see the introduction) which I find forced. One issue which seems especially confusing is the distinction between self-attention and GNNs in the paper. In practice, they are very similar. GGNNs used in this work, when applied to a fully-connected graph will end up being quite similar to 1-head Transformer. I do not quite see why we need 2 sets of gates: one to select edges, do top-k and then another one to again guide message passing. The intuition for what it buys you needs to be more clear. I could imagine that this just adds more parameters to the model. Considering that this is a long paper, analyses are also limited or anecdotal (see, e.g., *Interpretability analysis*). Minor: 1. The attention scores are computed using the dot product between two ReLU-transformed vectors. Is there a reason to force the attention scores to be non-negative? 2. The passage *difference between manual graph construction and automatic graph construction is analogous to the difference between the feature engineering + Machine Learning paradigm ...* is very speculative. (Also, since much of the previous GNN/GCN QA work used gates, you can think of the manual construction as providing candidates for the edges.) There some types, e.g., in the intro: *parallel of question turns* and *parallel of passage words* -- typos / unclear (page 1) *improved by the hidden states* (page 2) interpretability analysis -> qualitative analysis (?)