Venue: ICLR 2017 conference

Reviews:

final_score: rating: null
review_content: 1. What is the memory size used in all the three tasks?
2. For the synthetic task, what is the size of the training data?
3. In the synthetic task, the big performance gap between baseline extended neural GPU with and without memory module is surprising. It would be interesting if authors can also report results on seq2seq+attention with and without memory module.
4. Last few lines in Translation section is not clear. What do you mean by providing the whole test context? Model sees all the test sentences and translations and then asked to translate the same?
5. I do not agree with the comment in the related work section that hard access queries have limited success. See for example [1] 

[1] Gulcehre, Caglar, Chandar, Sarath, Cho, Kyunghyun, and Bengio, Yoshua. Dynamic neural turing
machines with soft and hard addressing schemes. CoRR, abs/1607.00036, 2016. URL http:
//arxiv.org/abs/1607.00036.
review_title: Few questions

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper proposes a new memory module for large scale life-long and one-shot learning. The module is general enough that the authors apply the module to several neural network architectures and show improvements in performance.

Using k-nearest neighbors for memory access is not completely new. This has been recently explored in Rae et al., 2016 and Chandar et al., 2016. K-nearest neighbors based memory for one-shot learning has also been explored in [R1]. This paper provides experimental evidence that such an approach can be applied to a variety of architectures.

Authors have addressed all my pre-review questions and I am ok with their response.

Are the authors willing to release the source code to reproduce the results? At least for omniglot experiments and synthetic task experiments?

References:

[R1] Charles Blundell, Benigno Uria, Alexander Pritzel, Yazhe Li, Avraham Ruderman, Joel Z. Leibo, Jack Rae, Daan Wierstra, Demis Hassabis: Model-Free Episodic Control. CoRR abs/1606.04460 (2016)

review_title: Review

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: A new memory module based on k-NN is presented.
The paper is very well written and the results are convincing. 

Omniglot is a good sanity test and the performance is surprisingly good.
The artificial task shows us that the authors claims hold and highlight the need for better benchmarks in this domain.
And the translation task eventually makes a very strong point on practical usefulness of the proposed model.

I am not a specialist in memory networks so I trust the authors to double-check if all relevant references have been included (another reviewer mentioned associative LSTM). But besides that I think this is a very nice and useful paper. I hope the authors will publish their code.
review_title: null

final_score: rating: null
review_content: As I understand it, the value $v$ is the supervised desired target value which you use during training.
Then every memory operation will store this supervised value $v$?
So memory updates are only done during training? Or what value $v$ do you use during evaluation / inference?

The output of the memory module $V[n]$ is always only a scalar, or even an scalar integer?
As well as $v$?

In Convolutional Network with Memory, the output is the nearest neighbor from the memory module, i.e. the output of the network is also just a single scalar?

In GNMT with added memory module, how exactly is $\hat{y}_{m+1}$ defined?
For translation if you don't know the target sequence, how would you update and use the memory module?

review_title: memory module update

final_score: rating: 7: Good paper, accept
review_content: The paper proposes a new memory module to be used as an addition to existing neural network models.

Pros:
* Clearly written and original idea.
* Useful memory module, shows nice improvements.
* Tested on some big tasks.

Cons:
* No comparisons to other memory modules such as associative LSTMs etc.

review_title: interesting new memory module

final_score: rating: null
review_content: Thank you for a very interesting paper.

I am not sure if I understand the translation experiment correctly.
Can you briefly elaborate on the purpose of even and odd splitting and why it improves performance?
review_title: Context Set

Meta-review: The primary contribution of this paper is showing that k-nearest-neighbor method based memory can be usefully incorporated in a variety of architectures and supervised learning tasks. The presentation is clear, and results are good. I like the synthetic task and analysis. For the Omniglot task, running and reporting results on the original splits used by Lake would be good, as the splits used in matching nets are considerably easier and result in ceiling effects.
Decision: Accept (Poster)