Title: Query-Reduction Networks for Question Answering
Authors: Minjoon Seo
Sewon Min
Ali Farhadi
Hannaneh Hajishirzi
Venue: ICLR 2017
Keywords: Natural language processing
Deep learning

Abstract: In this paper, we study the problem of question answering when reasoning over multiple facts is required. We propose Query-Reduction Network (QRN), a variant of Recurrent Neural Network (RNN) that effectively handles both short-term (local) and long-term (global) sequential dependencies to reason over multiple facts. QRN considers the context sentences as a sequence of state-changing triggers, and reduces the original query  to a more informed query as it observes each trigger (context sentence) through time. Our experiments show that QRN produces the state-of-the-art results in  bAbI QA and dialog tasks, and in a real goal-oriented dialog dataset. In addition, QRN formulation allows parallelization on RNN's time axis, saving an order of magnitude in time complexity for training and inference.  


Review discussion:
About revision:
- improved result on bAbI QA 10k (3.2% -> 0.7%)
- retouched the wording of some paragraphs, fixed typos
I am impressed by the results compared to the simplicity of the model.
You mention that the difference between your update gates and the softmax gating in previous work is that it gives local sigmoid attention computed only on the current memory.
However when you unfold the recursion in eqn 3, multiplication of z_t or (1-z_t) values throughout time will act like weights to a weighted average across all memory (summing up to 1). Is that not very similar to a softmax gating, just that it uses a different function to compute the attention weights? And uses the same parameters that the softmax would use? I am not sure if that is a significant difference. Please clarify if I'm missing anything here.
Thank you for your question!
We first define a term “relevance”, to be distinguished from “attention”, to better explain the difference between sigmoid attention and softmax attention.
“Relevance” between the query and a sentence is the absolute measure of whether the sentence is relevant to the query.
The “attention” weight of a sentence by the query is whether the sentence is ultimately used to get the correct answer of the query with regards to the entire story.
An “attended” sentence is always a “relevant” sentence to the query, but a “relevant” sentence might not be “attended” for the final answer.
For instance, consider the top-left table (task 2) of Figure 3. The first three sentences are all relevant to the query, but only the third sentence is useful for the final answer.
With these terms defined, the update gate in QRN only measures the relevance between the query and each sentence. The sigmoid “attention” of each sentence is controlled by the recurrent update of the hidden state with the update gate. This means that sigmoid attention can perform time-based (address-based) attention (memory access) by differentiating similar sentences at different time steps.
Hence the sigmoid attention is able to just attend on the third sentence even though they are all equally relevant to the query.
On the other hand, vanilla softmax attention mechanism is unable to distinguish between “relevance” and “attention”.
That is, the first three sentences in task 2 (top left) of Figure 3 will be equally attended (e.g. 0.3 each).
In other words, vanilla softmax attention cannot distinguish similar sentences at different time steps.
1. The paper mentioned that 'reduces the original query to an easier-to-answer query as it observes each trigger (context sentence) through time', I do not get the point why the model could have this ability.
2. The model is similar to strong-typed RNN with question as another input in ICML 2016, you also should cite this paper -- "Strongly-Typed Recurrent Neural Network".
Thank you for your questions!
First, please note that we have shown in ‘interpretations’ in Section 5.3 that we could decode the reduced query vectors to the desired natural language sentences in some tasks. We are not sure if we completely understand reviewer’s question.
If the reviewer means “can we guarantee that the reduced query is easier to answer”, we agree that the word “easier” could be misleading and we used it for high-level intuition. It is probably more accurate to say the reduced query becomes “more informed” of the story as it observes each trigger through time.
If the reviewer means “is the query being updated by each trigger at all”, this must be the case since we only use the last reduced query (hidden state) of QRN to answer.
Hence the reduced query must have been updated to incorporate all necessary information from the story.
If the reviewer means “how such a simple MLP can semantically reduce the query”, please note that our reduce function can be an arbitrary function depending on the application, and we just show one simple example that works well on the datasets. QRN is not about how to define the reduce function, but rather how to connect the components (reduce function, update gate, previous hidden state, etc.) together so that each component is inclined to learn its role correctly. We hypothesize that the simplification of the recurrent update helps each component to learn more easily.
Thank you for suggesting an important citation that we missed.
We just updated the related work, and here we write what we added to it.
QRN can be distinguished from Strongly-Typed Recurrent Neural Networks (STRNN) in three ways.
First, QRN’s update gate simulates attention mechanism, measuring the relevance between the input sentence and query.
On the other hand, the gates in STRNN can be considered as the simplification of LSTM/GRU by removing their dependency on previous hidden state.
Second, QRN is an RNN that is natively compatible with context-based QA tasks, where the QRN unit accepts two inputs, i.e. each context sentence and query. This is different from STRNN which has only one input.
Third, we show that QRN is timewise-parallelizable on GPUs. Our parallelization algorithm is also applicable to STRNN.
Our work is similar to STRNN in that the gates and the candidate hidden states do not depend on the previous hidden state, which simplifies the recurrent update. We believe many of the benefits due to the simplification mentioned in STRNN paper are also applicable to QRN (some of them are already discussed in the paper).
What have changed:
- Related work now includes discussions about previous work that R2 mentioned
- "easier-to-answer" to "more informed" in abstract, after R2's comment
- bAbI 10k results average error improved from 0.7 to 0.3 with repeating the training procedures 10+ times (previously only 5 times, see 'Training' in Section 5.2).
Do you train one model for all of the tasks, or is it the same architecture for each task, but a different model?
We trained the same architecture for each task independently. Thank you for pointing this out and we will explicitly mention this in our next revision of the paper.
Sukhbaatar et al. (2015) try both taskwise and joint training and the better result is taken; in bAbI QA 1k, joint training performs better (by 1.5%), and in 10k, taskwise training performs better (by 0.9%). Xiong et al. (2016a) and Perez and Liu (2016) report taskwise results only. Graves et al. (2016) report joint result on 10k only.
In case the reviewer is interested in how our model performs in joint training setting, we are currently running the experiments and will report them once they are available. Please note that 10k joint training will take a considerable amount of time.
This work introduces a new RNN model and shows how it can be used for QA in bAbI-like settings.  I think the model is simple and interesting, and the results are strong.  I would like the authors to also show results training on all tasks simultaneously, but other than that, I think it is an interesting model and it is nicely described and tested.
The paper proposes a simplistic recurrent architecture that has a very small temporal dependence in its definition (which, in turn, allows it to be parallelized across the temporal axis). Paper is clear to follow, the architecture has appropriate motivation and the experiments seem to be very thorough and over an appropriate range of synthetic and real datasets. Results are very impressive on its own especially considering the simplicity of the model. Activation and gate visualizations are informative.
Overall, the work is sufficient as a conference contribution.
I appreciate the effort by authors in responding to questions and incorporating revisions.
The paper proposed a simple and effective model for QA. The paper is easy to read and result is impressive on the synthetic and real dataset. The one question is the paper is called query-reduction, but there is no place to show this reduction explicitly.
We thank all reviewers for supporting our paper! We also thank them for their insightful suggestions and comments that have improved our paper.
R1: We will add the training results for all tasks simultaneously in the upcoming revision.
R2: We will explicitly clarify the “reduction” in the upcoming revision.
- Added joint training result for bAbI QA 1k (in Appendix A Table 2)
- Retouched the explanations of "query-reduction" in Section 1 and 2.
Final version uploaded:
- fixed typos
- more spacious formatting
- added acknowledgments

Meta-review:
role: PC
data: comment: The paper proposes a simple recurrent architecture for question-answering, achieving good performance on several reasoning benchmarks. All three reviewers agree on the merit of the contribution.

Decision: Accept (Poster)