Venue: ICLR 2018 conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: Much of the work on neural computation has focused on learning from input/output samples.  This paper is a study of the effect of adding additional supervision to this process through the use of loss terms which encourage the interpretable parts of the architecture to follow certain expected patterns.

The paper focuses on two topics:
1.  Developing a general formalism for neural computers which includes both the Neural Turing Machine (NTM) and the Neural Random Access Machine (NRAM), as well as a model for providing partial supervision to this general architecture.

2.  An experimental study of providing various types of additional supervision to both the NTM and the NRAM architecture.

I found quite compelling the idea of exploring the use of additional supervision in neural architectures since oftentimes a user will know more about the problem at hand than just input-output examples.  However, the paper is focused on very low-level forms of additional supervision, which requires the user to deeply understand the neural architecture as well as the way in which a given algorithm might be implemented on this architecture.  So practically speaking I don't think it's reasonable to assume that users would actually provide additional supervision in this form.

This would have been fine, if the experimental results provided some insights into how to extend and/or improve existing architectures.  Unfortunately,  the results were simply a very straight-forward presentation of a lot of numbers, and so I was unable to draw any useful insights.  I would have liked the paper to have been more clear about the insights provided by each of the tables/graphs.  In general we can see that providing additional supervision improves the results, but this is not so surprising.

Finally, much of the body of the paper is focused on topic (1) from above, but I did not feel as though this part of the paper was well motivated, and it was not made clear what insights arose from this generalization.  I would have liked the paper to make clear up front the insights created by the generalization, along with an intuitive explanation.  Instead much of the paper is dedicated to introduction of extensive notation, with little clear benefit.  The notation did help make clear the later discussion of the experiments, but it was not clear to me that it was required in order to explain the experimental results.

So in summary I think the general premise of the paper is interesting, but in it's current state I feel like the paper does not have enough sufficiently clear insights for acceptance.

review_title: Insights not clear enough

final_score: rating: 4: Ok but not good enough - rejection
review_content: The authors introduce the general concept of a differential neural computational machine, dNCM. It can apply to any fully differentiable neural programming machine, such as the Neural Turing Machine (NTM) or NRAM or the Neural GPU, but not to non-fully-differentiable architecture such as NPI. The author show how partial traces can be used to improve training of any dNCM with results on instantiations of dNCM for NTM and NRAM.

On the positive side, the paper is well-written (though too many results require looking into the Appendix) and dNCM is elegant. Also, while it's hard to call the idea of using partial traces original, it's not been studied in this extent and setting before. On the negative side, the authors have chosen weak baselines and too few and easy tasks to be sure if their results will actually hold in general. For example, for NTM the authors consider only 5 tasks, such as Copy, RepeatCopyTwice, Flip3rd and so on (Appendix E) and define  "a model to generalize if relative to the training size limit n, it achieves perfect accuracy on all of tests of size ≤ 1.5n and perfect accuracy on 90% of the tests of size ≤ 2n". While the use of subtraces here shows improvements, it is not convincing since other architectures, e.g., the Improved Neural GPU (https://arxiv.org/abs/1702.08727), would achieve 100% on this score without any need for subtraces or hints. The tasks for the NRAM are more demanding, but the results are also more mixed. For one, it is worrysome that the baseline has >90% error on each task (Appendix J, Figure 12) and that Merge even with full traces has still almost 80% errors. Neural programmers are notoriously hard to tune, so it is hard to be sure if this difference could be eliminated with more tuning effort. In conclusion, while we find this paper valuable, to be good enough for acceptance it should be improved with more experimentation, adding baselines like the (Improved) Neural GPU and more tasks and runs.
review_title: Interesting synthesis for neural programming machines lacking strong baselines.

final_score: rating: 5: Marginally below acceptance threshold
review_content: Summary

This paper presents differentiable Neural Computational Machines (∂NCM), an abstraction of existing neural abstract machines such as Neural Turing Machines (NTMs) and Neural Random Access Machines (NRAMs). Using this abstraction, the paper proposes loss terms for incorporating supervision on execution traces. Adding supervision on execution traces in ∂NCM improves performance over NTM and NRAM which are trained end-to-end from input/output examples only. The observation that adding additional forms of supervision through execution traces improves generalization may be unsurprising, but from what I understand the main contribution of this paper lies in the abstraction of existing neural abstract machines to ∂NCM. However, this abstraction does not seem to be particularly useful for defining additional losses based on trace information. Despite the generic subtrace loss (Eq 8), there is no shared interface between ∂NCM versions of NTM and NRAM that would allow one to reuse the same subtrace loss in both cases. The different subtrace losses used for NTM and NRAM (Eq 9-11) require detailed knowledge of the underlying components of NTM and NRAM (write vector, tape, register etc.), which questions the value of ∂NCM as an abstraction.

Weaknesses

As explained in the summary, it is not clear to me why the abstraction to NCM is useful if one still needs to define specific subtrace losses for different neural abstract machines.
The approach seems to be very susceptible to the weight of the subtrace loss λ, at least when training NTMs. In my understanding each of the trace supervision information (hints, e.g. the ones listed in Appendix F) provides a sensible inductive bias we would the NTM to incorporate. Are there instances where these biases are noisy, and if not, could we incorporate all of them at the same time despite the susceptibility w.r.t λ?
NTMs and other recent neural abstract machines are often tested on rather toyish algorithmic tasks. I have the impression providing extra supervision in form of execution traces makes these tasks even more toyish. For instance, when providing input-output examples as well as the auxiliary loss in Eq6, what exactly is left to learn? What I like about Neural-Programmer Interpreters and Neural Programmer [1] is that they are tested on less toyish tasks (a computer vision and a question answering task respectively), and I believe the presented method would be more convincing for a more realistic downstream task where hints are noisy (as mentioned on page 5).

Minor Comments

p1: Why is Grefenstette et al. (2015) an extension of NTMs or NRAMs? While they took inspiration from NTMs, their Neural Stack has not much resemblance with this architecture.
p2: What is B exactly? It would be good to give a concrete example at this point. I have the feeling it might even be better to explain NCMs in terms of the communication between κ, π and M first, so starting with what I, O, C, B, Q are before explaining what κ and π are (this is done well for NTM as ∂NCM in the table on page 4). In addition, I think it might be better to explain the Controller before the Processor. Furthermore, Figure 2a should be referenced in the text here.
p4 Eq3: There are two things confusing in these equations. First, w is used as the write vector here, whereas on page 3 this is a weight of the neural network. Secondly, π and κ are defined on page 2 as having an element from W as first argument, which are suddenly omitted on page 4.
p4: The table for NRAM as ∂NCM needs a bit more explanation. Where does {1}=I come from? This is not obvious from Appendix B either.
p3 Fig2/p4 Eq4: Related to the concern regarding the usefulness of the ∂NCM abstraction: While I see how NTMs fit into the NCM abstraction, this is not obvious at all for NRAMs, particularly since in Fig 2c modules are introduced that do not follow the color scheme of κ and π in Fig 2a (ct, at, bt and the registers).
p5: There is related work for incorporating trace supervision into a neural abstract machine that is otherwise trained end-to-end from input-output examples [2].
p5: "loss on example of difficulties" -> "loss on examples of the same difficulty"
p5: Do you have an example for a task and hints from a noisy source?
Citation style: sometimes citation should be in brackets, for example "(Graves et al. 2016)" instead of "Graves et al. (2016)" in the first paragraph of the introduction.

[1] Neelakantan et al. Neural programmer: Inducing latent programs with gradient descent. ICLR. 2015. 
[2] Bosnjak et al. Programming with a Differentiable Forth Interpreter. ICML. 2017.
review_title: Useful abstraction?

Meta-review: While the reviewers considered the basic idea of adding supervision intermediate to differentiable programming style architectures to be interesting and worthy of effort, they were unsure if
1: the proposed abstractions for discussing ntm and nram are well motivated/more generally applicable
2: the methods used in this work to give intermediate supervision are more generally applicable
 
Decision: Reject