Venue: ICLR 2017 conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: The authors proposed a dynamic neural Turing machine (D-NTM) model that overcomes the rigid location-based memory access used in the original NTM model. The paper has two main contributions: 1) introducing a learnable addressing to NTM. 2) curriculum learning using hybrid discrete and continuous attention. The proposed model was empirically evaluated on Facebook bAbI task and has shown improvement over the original NTM.

Pros:
+ Comprehensive comparisons of feed-forward controllers v.s. recurrent controllers
+ Encouraging results on the curriculum learning on hybrid discrete and continuous attentions

Cons:
- Very weak NTM baseline (due to some hyper-parameter engineering?) in Table 1, 31% err. comparing to the NTM 20% err. reported in Table 1 in(Graves et al, 2016, Hybrid computing using a neural network with dynamic external memory). In fact, the NTM baseline in (Graves et al 2016) is better than the proposed D-NTM with GRU controller. Maybe it is worthwhile to reproduce their results using the hyper-parameter setting in their Table2 which could potentially lead to better D-NTM performance?
- Section 3 of the paper is hard to follow. The overall clarity of the paper needs improvement.
review_title: Review

final_score: rating: 7: Good paper, accept
review_content: The paper extends the NTM by a trainable memory addressing scheme.
The paper also investigates both continuous/differentiable as well as discrete/non-differentiable addressing mechanisms.

Pros:
* Extension to NTM with trainable addressing.
* Experiments with discrete addressing.
* Experiments on bAbI QA tasks.

Cons:
* Big gap to MemN2N and DMN+ in performance.
* Code not available.
* There could be more experiments on other real-world tasks.

review_title: interesting extension to NTM

final_score: rating: null
review_content: I think the footnote is missing.
review_title: footnote page 2

final_score: rating: null
review_content: You introduce the address matrix, so I expected that you only use the address vectors $a$ for the addressing mechanism but in chapter 3 where you describe the addressing mechanism, you use the whole memory $m$ and not just $a$. Is that right? If so, I don't see where you use the address matrix at all and how this is different to the normal NTM. Can you explain? If you meant $a$ instead of $m$ in chapter 3, the update $\overline{m}$ is still meant to update both address vector and content vector, right?

I think for better understandability, you should state the vector space of all the variables, e.g. $\overline{m}^t \in \R^{d_h}$, etc. Also, $m_i$ is maybe confusing, why not call it $M_i$ or $M[i]$?
In equation (3), the $h^t$ and $x^t$ are not bold, are that different variables?

For erasing and writing, you use $e^t$ and $u^t_j$.
So it means that the erase is global over all memory cells?
However, in chapter 3, you say that the erase ($e$) vector is just the same as the read ($w$) and write ($u$) vector. So should it be $e^t_j$ here?
And is it $e^t \in \R^{d_h}$ or is it a scalar?
And is it $u^t_j \in \R^{d_h}$ or is it a scalar?

In chapter 3, you write $w^t_i$ which is a scalar. In equation (5), it is $w^t$ in bold, so that is over all $i$, i.e. $w^t \in \R^N$?

In chapter 7.1.2, what is CBA?
In table 1, what is LBA?
In Table 1 & 2, you call it "Soft D-NTM" and in Table 3 & 4, you call it "D-NTM cont.". Is that the same?

Have you tried to use a NTM with multiple steps? I only see 1-step NTMs.

Why do you think the discrete D-NTM is worse than the cont. D-NTM for pMNIST?
As well as with FF controller (Table 2).

The curriculum strategy of chapter 4 was only applied for bAbI trained with FF controler (Table 2) and not for the other experiments?

review_title: formulas, notations, address matrix usage, etc

final_score: rating: null
review_content: The model described in his paper is quite complicated and reproducing the method from its description may be challenging: Are you planing to release the code and what is your estimate release date?

Different cost functions and regularization are introduce in the paper. Would it be possible to summarize the overall cost function minimized by this model? 

Some variables seems to have different definition. In particular w_t and b, would it be possible to clarify this?

“gamma_t is a shallow MLP”: What does it mean? Is gamma_t a function or a variable? It seems from eq. (10) that it is a vector (or a scalar?).

"curriculum learning for the discrete attention": Can you compare this to simpler schemes? Like rounding the continuous attention?

In your introduction, you state that "it is possible to use the discrete non-differentiable attention mechanism", referencing Zaremba & Sutskever, 2016 that use REINFORCE and a simple controller. However, in your sec. 4, you are stating that  "Training discrete attention with feed-forward controller and REINFORCE is challenging". It seems that these two statements contradict each other, could you comment on it? 

In the introduction, you state that "Memory network (Weston et al. 2015b) [..] uses an attention-based mechanism to index them". To the best of my knowledge, it is actually Sukhbaatar et al., 2015 that has introduce the attention mechanism to memory networks. 

In the introduction, the authors state that “memory networks […] [are] used in real tasks (Bordes et al. 2015, Dodge et al. 2015)”. However in both papers, they use memory network systems different from Weston et al. The current statement is quite misleading, would it be possible to clarify it?

review_title: Code available? + other questions

final_score: rating: null
review_content: Is sole FF being compared in table 1? 

sec 7.1.2 mentioned that FF controller are harder to train then why is table 2 number are better than table 1?

how comparable are the D-NTM vs MemN2N in table 1 in terms of the number of parameters and representation power?

 
review_title: Question about the experiments

final_score: rating: 4: Ok but not good enough - rejection
review_content: This paper introduces a variant of the neural Turing machine (NTM, Graves et al. 2014) where key and values are stored. They try both continuous and discrete mechanisms to control the memory.

The model is quite complicated and seem to require a lot of tricks to work. Overall it seems that more than 10 different terms appear in the cost function and many different hacks are required to learn the model. It is hard to understand the justification for all of these tricks and sophisticated choices. There is no code available nor plan to release it (afaik).

The model is evaluated on a set of toy problems (the “babi task”) and achieves performance that are only slightly above those of a vanilla LSTM but are much worse than the different memory augmented models proposed in the last few years.  

In terms of writing, the description of the model is quite hard to follow, describing different blocks independently, optimization tricks and regularization. The equations are hard to read, using non standard notation (e.g., “softplus”), overloading notations (w_t, b…), or write similar equations in different ways (for example, eq (8-9) compared to (10-11). Why are two equations in scalar and the other in vectors? Why is there an arrow instead of an equal?…).

Overall it is very hard to put together all the pieces of this model(s), there is no code available and I’m afraid there is not enough details to be able to reproduce their numbers. Finally, the performance on the bAbI tasks are quite poor compared to other memory augmented models.

review_title: Review

Meta-review: This paper proposes some novel architectural elements, and the results are not far from published DNC results. However, the main issues of this paper are the complexity of the model, lack of justification for certain architectural choices, gaps with reported DNC numbers on BABI, and also a somewhat toy-ish task.
Decision: Reject