Title: A Context-aware Attention Network for Interactive Question Answering
Authors: Huayu Li
Martin Renqiang Min
Yong Ge
Asim Kadav
Venue: ICLR 2017
Keywords: Deep learning
Natural language processing
Applications

Abstract: We develop a new model for Interactive Question Answering (IQA), using Gated-Recurrent-Unit recurrent networks (GRUs) as encoders for statements and questions, and another GRU as a decoder for outputs. Distinct from previous work, our approach employs context-dependent word-level attention for more accurate statement representations and question-guided sentence-level attention for better context modeling. Employing these mechanisms, our model accurately understands when it can output an answer or when it requires generating a supplementary question for additional input. When available, user's feedback is encoded and directly applied to update sentence-level attention to infer the answer. Extensive experiments on QA and IQA datasets demonstrate quantitatively the effectiveness of our model with significant improvement over conventional QA models.

TL;DR: A self-adaptive QA model aware of what it knows and what it does not know for interactive question answering.

Review discussion:
1. It is said a end-to-end model for IQA, but in the training IQA, it is not just given question and answer. For each step, it is supervised with the feedback questions, which is similar to the Memory Network paper, not the End-to-end one.
2. In the attention model, at equ 6 and 9, it used the question representation U to calculate both word-level and sentence -level attention, why U could represent information in both word-level and sentence level, or word-leve and sentence-level information are in the same space?
Thanks for your comments.
1. Different from Memory Networks, our model does not use supporting facts during the training of both QA and IQA. For standard QA, we train the model end-to-end in the form of statements-question-answer triplets. For IQA, we use additional information obtained from users during training to update sentence-level attention. From this perspective, we agree that our model is not trained end-to-end for IQA because of interactivity. We will revise our paper to improve the model description to avoid confusions.
2. \mathbf{u} is encoded in sentence-level space. Word-level representation space is different from sentence-level space.
Therefore, we use a two-layer MLP to transform a word representation into a sentence-level space and get a context-aware vector in equation 5. This vector is then used to calculate the attention weight over a word shown in equation 6 by incorporating question and context information.
This paper proposes an "interactive" version of the bAbI dataset by adding supporting questions/answers to the dataset in cases where there is not enough information to answer the question. Interactive QA is certainly an interesting problem and is well-motivated by the paper.
However, I don't feel like the bAbI extension is adequately explained.
For example, the baseline DMN and MemN2N models on the IQA task are "take both statements and question as input and then
estimate an answer." Their task is then fundamentally more difficult from the CAN's because they do not distinguish "feedback" from the original context; perhaps a more fair approach would be to treat **every** question (both supporting and original questions) as individual instances.
Also, how were the supporting questions and the user feedback generated? How many templates / words were used to create them? The dataset creation details are missing, and if space is an issue, a lot of basic exposition on things like GRU / sentence encodings can be cut (or at least greatly shortened) and replaced with pointers to the original papers.
Another issue I had is that the model attempts to generate these synthetic questions; if there are just one or two templates, why not just predict the values that fill these templates?
So instead of generating "Which bedroom, master one or guest one?" with an RNN decoder, just predict "which" or "which bedroom"... isn't this sufficient?
In the end, these just seem like more supporting facts, not actual interaction with users, and the fact that it is run on only three of the original twenty tasks make the conclusions hard to trust.
In conclusion, I think the paper has a strong idea and motivation, but the experiments are not convincing for the paper to be accepted at ICLR.
Thanks a lot for your review.
1. Thanks a lot for your suggestion of providing more details about the bAbI extension. We will improve our paper to talk more about this new dataset. Basically, the extended datasets follow standard bAbI settings, but the difference is that we added some supporting question templates, such as “Who is REFERENCE?”, “Which OBJECT, CHOICE_1 or CHOICE_2 one?”, “What objects are you referring to?”. The feedback is randomly picked from the objects in the statements.
For DMN+ and MemN2N on IQA tasks, they do not use supporting question and feedback and we will make it more clear in the paper. We did this because they cannot handle the feedback by design.
2. We agree that the supporting question like "which" or "which bedroom" is also sufficient for most users.
But generating the template "Which OBJECT, CHOICE_1 or CHOICE_2 one?” can increase the diversity of supporting questions and at the same time can test the capability of the decoder to output a statement-dependent question.
3. Our model is aware of what it knows and what it does not know.
When it does not know, it outputs a question for additional support. The user then provides a feedback. This feedback will be used to update the model and output the answer. We think this interactive process is novel, natural and effective for solving real-world QA problems.
4. Our three tasks simulate different scenarios with incomplete information.
Specifically, task 1 focuses on ambiguous actor problem. Task 4 represents ambiguous object problem. Task 7 is to ask further information that assists answer prediction. We think all of these three task are representative for IQA problems. We do not need to modify each of the 20 bAbI task to make it interactive, because some extensions are not natural.
5. Creating a new task and a new dataset is one of the major contributions of this paper. The purpose is to inform the research community of the importance of modeling unknown state in QA systems.
This work describes
1: a two stage encoding of stories in bAbI like setups, where a GRU is used to encode a sentence, word by word, conditioned on a sentence level GRU, and the sentence level GRU keeps track of a sentence level encoding.  Each is used
2: modifying the bAbI tasks so it is necessary to ask a question to correctly solve the problem
I am not convinced by the papers results:
1:   The new architecture does not do significantly better than DMN+, and in my view, is similar to DMN+.   What problem with DMN+ does your architecture solve?
2:  There are now several papers doing the second thing, for example "Dialog-based Language Learning" by Weston and  "Learning End-to-End Goal-Oriented Dialog" by Bordes and Weston, and I think doing it more carefully and in more compelling ways.   In the current work, the correct answer to the question seems given independent of the what the agent asks, so any model that can output "unknown" and then input the extra response has an advantage.  Essentially all of the architectures that are used to solve bAbI can be modified to do this...
Indeed, the enc-dec* accuracies in appendix A show that this sort of module can be appended to any other model.  All of the standard models can be trained to output questions as a sequence of words.
Furthermore, I suspect you could generate the questions  in the authors' setting just by enumerating all the questions that occur in training, and taking a softmax over them, instead of generating word-by-word.
Thanks a lot for your review.
1. In general, most QA systems can be described to have input module, question module, and answer module in a cartoon style as in Figure 1 (the interactive module is new).
However, the technical difference between our model CAN and previous model DMN+ is very clear, which will be described as follows and further emphasized in the revision of our paper.
1) Completely different attention mechanisms: CAN has a well-motivated two-level context-aware attention mechanism including context-dependent word attention and question-dependent sentence attention, which requires transformations between two different embedding spaces. DMN+ only has attention calculated at a standard fact/sentence level. This is the key difference between CAN and DMN+.
2) Just due to different attention mechanisms, CAN does not need multi-hop attention and calculation, but DMN+ does require it. It is hard to determine the optimal number of hops for each task, which is clearly an overhead for DMN+. Just because our model CAN does not have multi-hop attention, we don't need to worry about complex issues such as ReLU with parameter sharing/nonsharing for computing episode memories in DMN+.
3) Significant performance improvement: Out of 20 bAbI tasks (using 1% error rate as cutoff), our model CAN only failed on 1 task, but DMN+ failed on 5 tasks.
In details, in task 7 (CAN is 0.3% while DMN+ is 2.4%), task 17 (CAN is 0.2% while DMN+ is 4.2%), and task 18 (CAN is 0.5% while DMN+ is 2.1%).
In summary, from a technical perspective, our model CAN is very different from DMN+.
2. Our IQA task is different from the dialog task in (Weston, 2016) and (Bordes et al., 2016). Our IQA focuses on conventional textual statement-question-answer triplets and effectively solves real-world QA problems with incomplete or ambiguous information.
In other words, our system behaves in a way that it knows what it knows and knows what it does not know.
Please note that (Weston, 2016) was just published at NIPS this December and (Bordes et al., 2016) is a concurrent submission to ICLR 2017. We will cite these two papers in the revision, but will mention that our research was performed at the same time.
3. To the best of our knowledge, our work is the first to augment encoder-decoder framework for IQA with incomplete/ambiguous information and use the IQA concept to improve QA accuracy.
This new framework could be extended to other models, but we are the first to propose/conduct this line of research.
Although most models can be trained to output question, how to effectively utilize user’s feedback to update model and output answer will lead to completely different papers.
It is true that the baseline model, EncDec*, can be easily extended to generate supporting questions, however, it cannot use the feedback to generate answers.
It is not clear that how DMN+ can be simply extended to solve the IQA problem proposed by us here. CAN is context-aware and readily models contextual information by design, which is natural for assessing whether there is any ambiguity or missing information. There is no motivation to consider how DMN+ should be extended for modeling ambiguous/incomplete information.
4. Enumerating all the questions in the training and taking softmax over them to generate question can be regarded as one classification solution.
But this kind of solution cannot address statement-dependent supporting questions.
For example, for supporting question “which bedroom, master or guest one?“, ‘master’ and ‘guest’ are generated according to statements. If the testing instance includes new examples, the classification solution is incapable of classifying a new supporting question.
1. the QA model is not novel, very similar to the existing model.
2. The IQA model is very confusing. If it needs human interactive in the training process, how could it be practical to ask human to join the training in each iteration? It sounds impractical. If the human interactive questions are predefined, then it is not interactive at all, since it is not based on the current state of model output.
Thanks a lot for your review.
1. In general, most QA systems can be described to have input module, question module, and answer module in a cartoon style as in Figure 1 (the interactive module is new).
However, the technical difference between our model CAN and previous model DMN+ is very clear, which will be described as follows and further emphasized in the revision of our paper.
1) Completely different attention mechanisms: CAN has a well-motivated two-level context-aware attention mechanism including context-dependent word attention and question-dependent sentence attention, which requires transformations between two different embedding spaces. DMN+ only has attention calculated at a standard fact/sentence level. This is the key difference between CAN and DMN+.
2) Just due to different attention mechanisms, CAN does not need multi-hop attention and calculation, but DMN+ does require it. It is hard to determine the optimal number of hops for each task, which is clearly an overhead for DMN+. Just because our model CAN does not have multi-hop attention, we don't need to worry about complex issues such as ReLU with parameter sharing/nonsharing for computing episode memories in DMN+.
3) Significant performance improvement: Out of 20 bAbI tasks (using 1% error rate as cutoff), our model CAN only failed on 1 task, but DMN+ failed on 5 tasks.
In details, in task 7 (CAN is 0.3% while DMN+ is 2.4%), task 17 (CAN is 0.2% while DMN+ is 4.2%), and task 18 (CAN is 0.5% while DMN+ is 2.1%).
In summary, from a technical perspective, our model CAN is very different from DMN+.
2. We use IQA datasets to train our model. Our IQA training datasets have both supporting questions and user’s feedbacks.
When testing, we use the trained model to handle test examples which are in the form of statements-question. As our model is aware of what it knows and it does not know, it outputs a question for additional support if it does not know. The user then provides a feedback. This feedback will be used to update the model and output an answer.
For example, in the test phase, when the system outputs a supporting question, “which bedroom, master or guest one?”. The user will provide either “master bedroom” or “guest bedroom”.  After receiving this feedback, the system will update the attention weights over sentences and output an estimated answer. Table 2 provides an example of the interactive process.

Meta-review:
role: PC
data: comment: The program committee appreciates the authors' response to concerns raised in the reviews. Unfortunately, all reviewers are leaning against accepting the paper. Authors are encouraged to incorporate reviewer feedback in future iterations of this work.

Decision: Reject