Title: DCN+: Mixed Objective And Deep Residual Coattention for Question Answering
Authors: Caiming Xiong
Victor Zhong
Richard Socher
Venue: ICLR 2018
Keywords: question answering
deep learning
natural language processing
reinforcement learning

Abstract: Traditional models for question answering optimize using cross entropy loss, which encourages exact answers at the cost of penalizing nearby or overlapping answers that are sometimes equally accurate. We propose a mixed objective that combines cross entropy loss with self-critical policy learning, using rewards derived from word overlap to solve the misalignment between evaluation metric and optimization objective. In addition to the mixed objective, we introduce a deep residual coattention encoder that is inspired by recent work in deep self-attention and residual networks. Our proposals improve model performance across question types and input lengths, especially for long questions that requires the ability to capture long-term dependencies. On the Stanford Question Answering Dataset, our model achieves state of the art results with 75.1% exact match accuracy and 83.1% F1, while the ensemble obtains 78.9% exact match accuracy and 86.0% F1.

TL;DR: We introduce the DCN+ with deep residual coattention and mixed-objective RL, which achieves state of the art performance on the Stanford Question Answering Dataset.

Review discussion:
In Equation 17 (page 5), we made a typo in that we did not include the regularization terms $$\log \sigma_{ce}^2 + \log \sigma_{rl}^2$$.
Summary:
This paper proposed an extension of the dynamic coattention network (DCN) with deeper residual layers and self-attention. It also introduced a mixed objective with self-critical policy learning to encourage predictions with high word overlap with the gold answer span. The resulting DCN+ model achieved significant improvement over DCN.
Strengths:
The model and the mixed objective is well-motivated and clearly explained.
Near state-of-the-art performance on SQuAD dataset (according to the SQuAD leaderboard).
Other questions and comments:
The ablation shows 0.7 improvement on EM with mixed objective. It is interesting that the mixed objective (which targets F1) also brings improvement on EM.
Thanks for your comments.
In practice the F1 and EM metrics are closely correlated. We chose to use the F1 score as a metric because it offers fine grain signals as to how well the span predicted matches the ground truth span, whereas the EM score only rewards exact gloss matches.
This paper proposed an improved version of dynamic coattention networks, which is used for question answering tasks.
Specifically, there are 2 aspects to improve DCN: one is to use a mixed objective that combines cross entropy with self-critical policy learning, the other one is to imporve DCN with deep residual coattention encoder. The proposed model achieved STOA performance on Stanford Question Asnwering Dataset and several ablation experiments show the effectiveness of these two improvements.
Although DCN+ is an improvement of DCN, I think the improvement is not incremental.
One question is that since the model is compicated, will the authors release the source code to repeat all the experimental results?
Thanks for your comments. We can try to release the source code after the decision process.
The authors of this paper propose some extensions to the Dynamic Coattention Networks models presented last year at ICLR.
First they modify the architecture of the answer selection model by adding an extra coattention layer to improve the capture of dependencies between question and answer descriptions. The other main modification is to train their DCN+ model using both cross entropy loss and F1 score (using RL supervision) in order to  reward the system for making partial matching predictions. Empirical evaluations conducted on the SQuAD dataset indicates that this architecture achieves an improvement of at least 3%, both on F1 and exact match accuracy, over other comparable systems. An ablation study clearly shows the contribution of the deep coattention mechanism and mixed objective training on the model performance.
The paper is well written, ideas are presented clearly and the experiments section provide interesting insights such as the impact of RL on system training or the capability of the model to handle long questions and/or answers. It seems to me that this paper is a significant contribution to the field of question answering systems.
Thank you for your comments!

Meta-review:
role: PC
data: comment: This is an interesting paper that provides modeling improvements over several strong baselines and presents SOTA on Squad.  One criticism of the paper is that it evaluates only on Squad, which is somewhat of an artificial task, but we think for publication purposes at ICLR, the paper has a reasonable set of components.

Decision: Accept (Poster)