Title: Dynamic Coattention Networks For Question Answering
Authors: Caiming Xiong
Victor Zhong
Richard Socher
Venue: ICLR 2017
Keywords: Natural language processing
Deep learning
Applications

Abstract: Several deep learning models have been proposed for question answering. How- ever, due to their single-pass nature, they have no way to recover from local maxima corresponding to incorrect answers. To address this problem, we introduce the Dynamic Coattention Network (DCN) for question answering. The DCN first fuses co-dependent representations of the question and the document in order to focus on relevant parts of both. Then a dynamic pointer decoder iterates over potential answer spans. This iterative procedure enables the model to recover from initial local maxima corresponding to incorrect answers. On the Stanford question answering dataset, a single DCN model improves the previous state of the art from 71.0% F1 to 75.9%, while a DCN ensemble obtains 80.4% F1.

TL;DR: An end-to-end dynamic neural network model for question answering that achieves the state of the art and best leaderboard performance on the Stanford QA dataset.

Review discussion:
Dear Authors,
Could you please clarify how do you combine the cross entropy loss of the start and end points across all iterations? A mathematical formulation of the final objective would be helpful.
Thanks.
Hi,
Thank you for your thoughtful comments!
During each iteration i of the decoder, we compute the start scores (alpha in the paper) and the end scores (beta in the paper) for each position of the document. For each score, we compute the softmax cross entropy losses using the ground truth answer start position and ground truth answer end position. We sum up these losses over each iteration. If the model does not stop, then we collect losses for a maximum of T iterations. If the model stops after R iterations, then we only collect losses for the R iterations that occurred. Mathematically (please excuse my formatting here), the loss is then:
sum_i^{min(T, R)} softmax_crossentropy(alpha_i, ground_truth_start) + softmax_crossentropy(beta_i, ground_truth_end)
The paper mentioned the dynamic decoder is able to escape the initial local maxima several times. I wonder how serious the issue is? If the encoder learns good enough representations, it is able to predict the answer span in one time. If it really needs several iterations to localize, why it must be done in decoder?
Hi,
Thank you for your thoughtful comments!
In our ablation study in Table 2 (page 8), we show the same model but with the decoder forced to a single iteration. As you suggested, most of the time, the coattention encoder learns good enough representations and the model is able to predict the correct answer in one pass.
However, we found that allowing more passes in the decoder helped us achieve better performance.
Namely, allowing the model a maximum of four passes as opposed to forcing it to a single pass increased our development F1 by 1.6%, thereby achieving the state of the art result.
Regarding your point that given the model needs several iterations to localize, why must the iterations occur in the decoder? This is purely a design decision to reduce computation complexity.
Indeed, we are interested in exploring an iterative encoder (e.g. the entire model is iterative) in future work.
I have a small question to your ablation study. You say, that you are forcing the model to a single pass in the ablation study, but is it still the model trained on 4 passes? If that is the case, it might be interesting to see what happens when you train on a single pass model, since the training would become different.
Hi Dirk,
When we say that the model is forced to a single pass, we mean that it is forced to a single pass during both training and evaluation.
1) What is the reason behind allowing some variation between the question encoding space and the document encoding space? What happens if the non-linear projection layer on top of the question embedding is removed?
2) How does the performance (EM and F1) change with respect to iterations in dynamic decoder?
3) Related work talks about previous Statistical QA and Neural QA works. How does the proposed approach relate to / different from all the mentioned approaches?
Hi,
Thank you for your thoughtful comments!
1. We chose to use the same encoder module for the question and the document to avoid overfitting on the limited amount of rather short training questions.
However, since the grammatical structure of question sentence is different from the structure of sentences in the document, we would like to allow for some variations between the representation space of the hidden state of questions and the space of documents.
Therefore, we add nonlinear transformation at the output of the hidden state of questions. Empirically we found that it yields 0.5%~1% F1 improvement on the development set.
2. As shown in our ablation study, allowing for iterations in the dynamic encoder increases our development F1 by 1.6%, thereby achieving the state of the art result.
In our ablation study in Table 2 (page 8), we show the same model but with the decoder forced to a single iteration.
3. Different from statistical QA model, our model is a end-to-end system without any hand-crafted features, which is a key part in statistical QA models.
Similar to neural QA models, we use an attention mechanism in the encoder module and feed the encoder representation to decoder for answer output. The whole framework is an attention-based encoder-decoder neural framework.
But different from prior work, we propose a new co-attention encoder which compute the attention maps both from document to question and from question to document, and use the two directional attention maps to learn a co-dependent representation of the question and documen (e.g. the coattention context). We also proposed a novel dynamic decoder which allow for several iterations to locate the final answer, because given a question, there may exist several intuitive answer spans within the document, each corresponding to a local maxima. This iterative procedure provides opportunity for the model to recover from initial local maxima corresponding to incorrect answer spans. There are some qualitative results shown in Figure 5. Our ablation study and table 2 also show the improvement with our proposed co-attention model and dynamic decoder respectively.
Finally, Table 1 shows that our model achieves state-of-the-art performance .
This paper proposed a dynamic coattention network for the question answering task with long contextual documents.
The model is able to encode co-dependent representations of the question and the document, and a dynamic decoder iteratively pointing the potential answer spans to locate the final answer.
Overall, this is a well-written paper.
Although the model is a bit complicated (coattention encoder, iterative dynamic pointering decoder and highway maxout network), the intuitions behind and the details of the model are clearly presented.
Also the performance on the SQuAD dataset is good.
I would recommend this paper to be accepted.
Thank you very much for your thoughtful review.
Summary: The paper proposes a novel deep neural network architecture for the task of question answering on the SQuAD dataset. The model consists of two main components -- coattention encoder and dynamic pointer decoder. The encoder produces attention over the question as well as over the document in parallel and thus learns co-dependent representations of the question and the document. The decoder predicts the starting and the end token of the answer iteratively, with the motivation that multiple iterations will help the model escape local maxima and thus will reduce the errors made by the model. The proposed model achieved state-of-art result on SQuAD dataset at the time of writing the paper. The paper reports some analyses of the results such as performance across question types, document, question, answer lengths, etc. The paper also performs some ablation studies such as performing only single round of iteration on decoder, etc.
Strengths:
1. The paper is well-motivated with two main motivations -- co-attending to the document and the question, and iteratively producing the answer.
2. The proposed model architecture is novel and the design choices made seem reasonable.
3. The experiments show that the proposed model outperforms the existing model (at the time of writing the paper) on the SQuAD dataset by significant margin.
4. The analyses of the results and the ablation studies performed (as per someone's request) provide insights into the various modelling design choices made.
Weaknesses/Questions/Suggestions:
1. In order to gain insights into how much each additional iteration in the decoder help, I would like to see the following -- for every iteration report the mean F1 for questions that converged in that iteration along with the number of questions that converged in that iteration.
2. Example of Question 3 in figure 5 is an interesting example where the model is unable to decide between multiple local maxima despite several iterations. Could authors please report how often this happens?
3. In order to estimate how much modelling of attention in the encoder helps, it would be good if authors could report the performance of the model when attention is not modeled at all in the encoder (neither over question, nor over document).
4. I would like to see the variation in the performance of the proposed model for questions that require different types of reasoning (table 3 in SQuAD paper). This would provide insights into what are the strengths and weaknesses of the proposed model w.r.t the type reasoning required.
5. In Wang and Jiang (2016), the attention is predicted over question for each word in the document.
But in table 2, when performing ablation study to make the proposed model similar to Wang and Jiang, C^D is set to C^Q.
But isn’t C^Q attention over document for each word in the question?
So, how is this similar to Wang and Jiang’s attention? I think QA^D will be similar to Wang and Jiang's attention since QA^D is attention over question for each word in the document. Please clarify.
6. In section 2.1, “n” and “m” are swapped when explaining the Document and Question encoding matrix. Please fix it.
Review Summary: The paper presents a novel and interesting model for the task of question answering on SQuAD dataset and shows that the model outperforms existing models.
However, to gain more insights into the functioning of the model, I would like see more analyses of the results and one more ablation study (see weaknesses section above).
Thank you very much for your thoughtful reviews.
>>>>1. In order to gain insights into how much each additional iteration in the decoder help, I would like to see the following -- for every iteration report the mean F1 for questions that converged in that iteration along with the number of questions that converged in that iteration.
We do not have strong support (number of examples) for high iteration (e.g. 3-4 iterations) examples. We did find in our ablation study that allowing the model multiple iterations, as opposed to forcing it to a single iteration, improves performance.
Below, we show the 4-iteration model’s performance across iterations needed to converge.
iterations   mean f1                            frequency
>>>>2. Example of Question 3 in figure 5 is an interesting example where the model is unable to decide between multiple local maxima despite several iterations. Could authors please report how often this happens?
This occurs very rarely.
For example, on the development, only 58 examples reach the maximum of 4 iterations.
>>>>3. In order to estimate how much modeling of attention in the encoder helps, it would be good if authors could report the performance of the model when attention is not modeled at all in the encoder (neither over question, nor over document).
We attempted a model without attention, in which the encoder first ingests the question then the document. The decoder then points over the document encoding as usual. The best validation performance we obtained for this model is 41.9% F1 and 33.3% exact match.
>>>>4. I would like to see the variation in the performance of the proposed model for questions that require different types of reasoning (table 3 in SQuAD paper). This would provide insights into what are the strengths and weaknesses of the proposed model w.r.t the type reasoning required.
I emphasize with your position that we analyze the model with respect to the reasoning required to gain more insights into what it is doing.
In particular, I would be very interested in seeing how each top model submitted to SQuAD differs in this regard.
That said, we contacted Rajpurkar et al, the original authors of the SQuAD paper, who declined to share the 200 sampled used to estimate the statistics in Table 3 of the SQuAD paper.
In particular, Pranav offered that releasing the 192 examples would have very little value of information of tuning and comparing models.
That said, they are considering including in the next iteration of the dataset a more thorough analysis for the purposes of analyzing the strengths and weaknesses of each model. We will update our results when the dataset is updated.
>>>>5. In Wang and Jiang (2016), the attention is predicted over question for each word in the document.
But in table 2, when performing ablation study to make the proposed model similar to Wang and Jiang, C^D is set to C^Q.
But isn’t C^Q attention over document for each word in the question?
So, how is this similar to Wang and Jiang’s attention? I think QA^D will be similar to Wang and Jiang's attention since QA^D is attention over question for each word in the document. Please clarify.
You are correct, this is indeed a typo (it’s suppose to be QA^D as opposed to C^Q).
>>>>6. In section 2.1, “n” and “m” are swapped when explaining the Document and Question encoding matrix. Please fix it.
Thanks! We will fix it!
Thanks for carrying out the ablation studies. For the analysis of the model w.r.t different types of reasoning as shown in table 3 in SQuAD paper, it would be great if you could present the outputs of the model for the examples shown in table 3. You could also check-out Appendices B and C in the most-revised version of "Machine Comprehension Using Match-LSTM and Answer Pointer" paper (also an ICLR 2017 submission) to see how they have done the same analysis.
Thanks for the suggestion! I will certainly run our model on these examples and add them to our paper.
Hi,
We have produced predictions using our model for the examples in Table 3 of the original SQuAD paper. Please find them below:
Document 1:
The Rankine cycle is sometimes referred to as a practical Carnot cycle because, when an efficient turbine is used, the TS diagram begins to resemble the Carnot cycle.
Question:
What is the Rankine cycle sometimes called?
Ground truth:
practical Carnot cycle
Model response:
practical Carnot cycle
Document 2:
While the Commision has a monopoly on initiating legislation, the European Parliament and the Council of the European Union have powers of amendment and veto during the legislative progress.
Question:
Which two governing bodies have legislative veto power?
Ground truth:
the European Parliament and the Council of the European Union
Model response:
European Parliament and the Council of the European Union
Document 3:
Current faculty include the anthropologist Marshall Sahlins, historian Dipesh Chakrabarty, ... Shakespeare scholar David Bevington, and renowned political scientists John Mearsheimer and Robert Pape.
Question:
What Shakespeare scholar is currently on the university’s faculty?
Ground truth:
David Bevington
Model response:
David Bevington
Document 4:
The V&A Theatre & Performance galleries, formerly the Theatre Museum, opened in March 2009. The collections are stored by the V&A, and are available for research, exhibitions and other shows. They hold the UK's biggest national collection of material about live performance in the UK since Shakespeare's day, covering drama, dance, musical theatre, circus, music hall, rock and pop, and most other forms of live entertainment.
Question:
What collection does the V&A Theatre & Performance galleries hold?
Ground truth:
Material about live performance
Model response:
UK's biggest national collection of material about live performance in the UK since Shakespeare's day
Document 5:
Along with giving the offender his "just deserts", achieving crime control via incapacitation and deterrence is a major goal of crime punishment.
Question:
What is the main goal of criminal punishment of civil disobedients?
Ground truth:
achieving crime control via incapacitation and deterrence
Model response:
achieving crime control via incapacitation and deterrence
Thanks for the above results. It is informative to know that the model can produce reasonable responses for questions that require reasoning across multiple sentences. I am changing the review score to "Top 50% of accepted papers, clear accept".
Thank you!
Paper Summary:
The paper introduces a question answering model called Dynamic Coattention Network (DCN). It extracts co-dependent representations of the document and question, and then uses an iterative dynamic pointing decoder to predict an answer span. The proposed model achieves state-of-the-art performance, outperforming all published models.
Paper Strengths:
-- The proposed model introduces two new concepts to QA models -- 1) using attention in both directions, and 2) a dynamic decoder which iterates over multiple answer spans until convergence or maximum number of iterations.
-- The paper also presents ablation study of the proposed model which shows the importance of their design choices.
-- It is interesting to see the same idea of co-attention performing well in 2 different domains -- Visual Question Answering and machine reading comprehension.
-- The performance breakdown over document and question lengths (Figure 6) strengthens the importance of attention for QA task.
-- The proposed model achieves state-of-the-art result on SQuAD dataset.
-- The model architecture has been clearly described.
Paper Weaknesses / Future Thoughts:
-- The paper provides model's performance when the maximum number of iterations is 1 and 4. I would like to see how the performance of the model changes with the number of iterations, i.e., the model performance when that number is 2 and 3. Is there a clear trend? What type of questions is the model able to get correct with more iterations?
-- As with many deep learning approaches, the overall architecture seems quite complex, and the design choices seem to be driven by performance numbers. As future work, authors might try to analyze qualitative advantages of different choices in the proposed model. What type of questions are correctly answered because of co-attention mechanism instead of attention in a single direction, when using Maxout Highway Network instead of a simple MLP, etc?
Preliminary Evaluation:
Novel and state-of-the-art question answering approach. Model is clearly described in detail. In my thoughts, a clear accept.
Thank you very much for your thoughtful review.
>>>>I would like to see how the performance of the model changes with the number of iterations, i.e., the model performance when that number is 2 and 3. Is there a clear trend?
Yes, the model consistently performs better with with more iterations, with diminishing returns as the number of iteration increases. With more iterations, the model performs better on examples in which there exists multiple potential answers (e.g. multiple people for a "who question").
>>>>As future work, authors might try to analyze qualitative advantages of different choices in the proposed model. What type of questions are correctly answered because of co-attention mechanism instead of attention in a single direction, when using Maxout Highway Network instead of a simple MLP, etc?
We agree, we will analyze the advantages of different choices in the proposed model across different question types.
Also as suggested by Reviewer 1, we think it would be interesting to analyze the model (e.g. across encoder types, decoder types, and iteration counts) with respect to the type of reasoning involved once the dataset is updated with such information.
We sincerely thank all reviewers for your feedback and comments! We have updated our paper, making minor adjustments such as fixing typos. We have also included some additional ablation studies, requested during the review process, in the appendix.

Meta-review:
role: PC
data: comment: The program committee appreciates the authors' response to concerns raised in the reviews. All reviewers agree that this is a good piece of work that should be accepted to ICLR. Authors are encouraged to incorporate reviewer feedback to further strengthen the paper.

Decision: Accept (Poster)