Venue: ICLR 2017 conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: First I would like to apologize for the delay in reviewing.

Summary : this work introduces a novel memory based artificial neural network for reading comprehension. Experiments show improvement on state of the art.
The originality of the approach seems to be on the implementation of an iterative procedure with a loop testing that the current answer is the correct one.

In order to get a better sense of the reason for improvement it would be interesting to have a complexity and/or a time analysis of the algorithm. I might be mistaken but I don't see you reporting anything on the actual number of loops necessary in the reported experiments.

The dataset description in section 2.2, should be moved to section 4 where the other datasets are described.
review_title: Shows improvement on state of the art; complexity ?

final_score: rating: 7: Good paper, accept
review_content: This paper proposed an iterative query updating mechanism for cloze-style QA. The approach is novel and interesting and while it is only verified in the paper for two Cloze-style tasks (CBT and WDW), the concept of read/compose/write operations seem to be more general and can be potentially applied to other reasoning tasks beyond Cloze-style QA. Another advantage of the proposed model is to learn when to terminate the iteration by the so-called adaptive computation model, such that it avoids the issue of treating the number of iterations as another hyper-parameter, which is a common practice of iterative models/multi-hop reasoning in previous papers.

There are a couple places that this paper can improve. First, I would like to see the results from CNN/Daily Mail as well to have a more comprehensive comparison. Secondly, it will be useful to visualize the entire M^q sequence over time t (not just z or the query gating) to help understand better the query regression and if it is human interpretable.
review_title: A novel and interesting approach

final_score: rating: null
review_content: This paper proposes a novel approach at cloze form question answering in which the model iteratively proposes queries to the question until it converges on a query and answer.

Question: what is the relationship between eqn 10 and eqn 13? Is 13 an alternatively way of updating the query? Where is z_t in the query gating approach? For the adaptive computation approach, do you stop based on sampling from the normalized stopping score? Is this stopping decision supervised (it doesn't seem to be)? I would like to see results of T=1 and T=2 of the proposed models to see how important the dynamic aspect of the model is. Although machine performance on the CNN/Daily News dataset is approaching human performance as the authors noted, I would nevertheless like to see the author demonstrate the effectiveness of this model on the CNN/Daily News dataset, as it has seen more interest in the community and is thus more competitive.

review_title: Few questions

final_score: rating: 6: Marginally above acceptance threshold
review_content: Thie paper proposed an iterative memory updating model for cloze-style question-answering task. The approach is interesting, and result is good. For the paper, I have some comments:

1. Actually the model in the paper is not single model, it proposed two models. One consists of "reading", "writing", "adaptive computation" and " Answer module 2", the other one is "reading", "composing", "writing", "gate querying" and "Answer module 1". Based on the method section and the experiment, it seems the "adaptive computation" model is simpler and performs better. And without two time memory update in single iteration and composing module, the model is similar to neural turing machine.

2. What is the MLP setting in the composing module? 

3. This paper tested different size of hidden state:[256, 368, 436, 512], I do not find any relation between those numbers, how could you find 436? Is there any tricks helping you find those numbers?

4. It needs more ablation study about using different T such as T=1,2..

5. According to my understanding, for the adaptive computation,  it would stop when the P_T <0. So what is the distribution of T in the testing data?
review_title: null

Meta-review: This paper proposes a memory-enhanced RNN in the vein of NTM, and a novel training method for this architecture of cloze-style QA. The results seem convincing, and the training method is decently novel according to reviewers, although the evaluation seemed somewhat incomplete according to reviewers and my own reading. For instance, it is questionable whether or not the advertised human performance on CNN/DM is accurate (based on 100 samples from a 300k+ dataset), so I'm not sure this warrants not evaluating or reporting performance on it. Overall this looks like an acceptable paper, although there is room for improvement.
Decision: Accept (Poster)