Venue: ICLR 2017 conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper proposed an integration of memory network with reinforcement learning. The experimental data is simple, but the model is very interesting and relatively novel. There are some questions about the model:

1. how does the model extend to the case with multiple variables in a single sentence?

2. If the answer is out of vocabulary, how would the model handle it?

3. I hope the authors can provide more analysis about the curriculum learning part, since it is very important for the RL model training.

4. In the training, in each iteration, how the data samples were selected, by random or from simple one depth to multiple depth? 

review_title: null

final_score: rating: 7: Good paper, accept
review_content: This paper introduces a nice dataset/data generator that creates bAbI like tasks, except where the questioning answering agent is required to clarify the values of some variables in order to succeed.  I think the baselines the authors use to test the tasks are appropriate.   I am a bit worried that the tasks may be too easy (as the bAbI tasks have been), but still, I think locally these will be useful.  If the generation code is well written, and the tasks are extensible, they may be useful for some time.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: This paper investigates a set of tasks that augment the basic bAbI problems. In particular, some of the people and objects in the scenarios are replaced with unknown variables. Some of these variables must be known to solve the question, thus the agent must learn to query for the values of these variables. Interestingly, one can now measure both the performance of the agent in correctly answering the question, and its efficiency in asking for the values of the correct unknown variables (and not variables that are unnecessary to answer the question). This inferring of unknown variables goes beyond what is required for the vanilla version of the bAbI tasks, which are now more or less solved.

The paper is well-written, and the contributions are clear. Due to the very limited vocabulary and structure of the bAbI problems in general, I think these tasks (and variants on them) should be viewed more as basic reasoning tasks than natural language understanding. I’m not convinced by the claim of the paper that this really tests the ‘interaction’ capabilities of agents – while the task is phrased as a kind of interaction, I think it’s more aptly described by simply ‘inferring important unknown variables’, which (while important) is more related to reasoning. I’m not sure whether the connection of this ability to ‘interaction’ is more a superficial one.

That being said, it is certainly true that conversational agents will need basic reasoning abilities to converse meaningfully with humans. I sympathise with the general goal of the bAbI tasks, which is to test these reasoning abilities in synthetic environments, that are just complicated enough (but not more) to drive the construction of interesting models. I am convinced by the authors that their extension to these tasks are interesting and worthy of future investigation, and thus I recommend the acceptance of the paper.

review_title: Review

final_score: rating: null
review_content: The proposed task is interesting in that it makes headways towards realistic, goal-oriented AI systems. However, I am skeptical regarding some of the evaluation metrics. For example, it seems like the model can achieve very high answer accuracy without good trajectory accuracy, thereby making it dubious as to whether the model is actually "learning" a policy that is actually reasoning over the relevant variables. In particular, counterintuitively, it seems that the trajectory accuracy isn't correlated with higher answer accuracy. This leads one to question whether reasoning over the relevant variables is even required to obtain good final performance (eg. answer accuracy) on the task. Another concern is that there is significant jump in query accuracy in the last column of the "Loc" results. What's the intuition behind this? I would like to see more analysis of the numbers in the results table from the authors.
review_title: some concerns

final_score: rating: null
review_content: 
I think the stated contributions of the paper are quite clear. I had a couple of general questions:

1) Have the authors considered also using locations as unknown variables, rather than just people names?
2) What thoughts do the authors have on scaling these kinds of approaches to more realistic language settings? The bAbI tasks are notoriously vocabulary/ structure constrained, making the task significantly easier.
review_title: paper is clear

final_score: rating: null
review_content: I would be curious to see how well a "greedy" question-asker would fair in terms of reward and the evaluations you show.  More generally, it would be interesting to see the tradeoffs between different query costs and the success metrics you compute.    How did you choose .05 --> -.05 ? 
review_title: reward vs accuracy

Meta-review: The program committee appreciates the authors' response to concerns raised in the reviews. While there are some concerns with the paper that the authors are strongly encouraged to address for the final version of the paper, overall, the work has contributions that are worth presenting at ICLR.
Decision: Accept (Poster)